Testing: Are Some Tests Better than Others?
by Alan Kurtz
Many researchers investigating the efficacy of FC have regarded testing as a straightforward task. However, until recently, no systematic analysis of the various testing protocols and how they influenced outcome had been conducted.
At the annual FC conference, sponsored by the FC Institute at
Syracuse University in Syracuse, Don Cardinal and Doug Biklen
reported on the preliminary results of their meta-analytic study of
FC. which revealed that the way in which persons who are using FC.
are tested significantly influences observed outcomes. Cardinal and
Biklen identified 14 procedural testing conditions that they felt
were most likely to positively influence the success of FC. users in
demonstrating their ability to communicate independent messages.
Decisions to include the various procedural conditions were based on
Cardinal and Biklen's own extensive experience with FC and on
Cardinal's experience in developing a large controlled evaluation.
Biklen and Cardinal assigned weights to each procedural condition so
that the relative influence of each condition on outcome could be
quantified and interpreted. The procedural conditions identified
were:
-Extensive user experience with FC;
-User has opportunity to practice with the protocol over multiple
sessions;
-FC users consult on the design of the test and format;
-The degree of familiarity of the facilitators to the FC user;
-Monitoring for FC user "style" (so that individualized supports can
be provided;
-Perception on the part of the FC user that testing had no or low
risk (the user's communication system will not be taken away if s/he
fails);
-The extent to which the testing included the capacity to build the
users confidence (typically, facilitators provide more structure when
an FC user has difficulty, so that the FC user has limited
opportunities to fail);
-Naturally controlled testing conditions (conditions approximating
those used in normal communication)
-Ongoing feedback to FC user during the test on performance;
-Minimize word retrieval tasks so the user is not required to produce
a specific work on demand;
-Information presented to the FC user in the test is presented in
multiple modalities;
-Content appropriate to the age of the FC. user during the test;
-Content relevant to the FC. user during the test;
-Adequate time for the FC. user to respond to questions during the
test.
Cardinal and Biklen examined six existing studies to determine the extent to which each used the procedural conditions. Scores, computed for each study to reflect the extent to which the study contained the desired procedural testing conditions, were correlated with the observed incidence that the FC users in each of the studies passed independent messages. A strong, positive association was revealed suggesting that the degree to which each study conformed to the desired testing conditions was related to successful outcome on the part of the FC. user. Therefore, the data strongly suggest that the presence of these procedural conditions in the testing protocol increases the likelihood that participants will pass their own messages in blind tests.
An interpretation of Cardinal and Biklen's results suggests that some FC. testing procedures may be more accurate measures than others of an individual's ability to communicate with FC. Thus, as identified by Stanley and Campbell (1963), the testing procedures may be in large part, responsible for explaining the huge disparity in results of various FC validation studies. It is curious to note that while most of the early controlled FC validation research appeared to show that almost none of the FC users tested were capable of authoring authentic messages, several, later, controlled studies revealed very different conclusions (Cardinal , Hanson & Wakeham, in press; Weiss, Wagner, & Bauman, in press; Sheehan & Matouzzi, 1996; Vasquez, 1994). In contrast to much of the earlier research, most or all of the participants in these studies passed validation tests. It certainly is possible that their success may have been due to the presence of many of the procedural conditions described by Cardinal and Biklen. Similarly, many FC users may have failed in earlier studies because of the absence of these conditions.
Cardinal and Biklen plan to expand their examination of FC. research to an additional 12 studies. This research not only will illuminate possible reasons for the equivocal results on FC. studies, but can assess the value of the procedural conditions in predicting outcome, and provide a foundation for establishing guidelines for accurate validation testing of FC.
Cardinal and Biklen's work can help researchers design
validations
tests that most accurately measure FC. users outcome. This line of
inquiry may also help us learn more about the kinds of support that
facilitators need to provide FC users whenever they communicate.
Reference
Campbell, D., & Stanley, J. (1963). Experimental and
quasi-experimental designs for research. Boston: Houghton
Mifflin.
The purpose of Facilitated Communication In Maine is to promote the appropriate use of facilitated communication through education, technical assistance, and support to people with disabilities, parents, educators, speech and language pathologists support providers, and other interested individuals. The project provides up-to-date information on current best practices, introductory and advanced workshops on the technique, resources regarding theoretical and practical components of facilitated communication and ongoing support to a network of resource persons who provide local education and support to other facilitators.
In complying with the letter and spirit of applicable laws and in pursuing its own goals of pluralism, the University of Maine shall not discriminate on the grounds of race, color, religion, sex, sexual orientation, national origin or citizen status, age, disability, or veterans status in employment education, and all other areas of the University. The University provides reasonable accommodations to qualified individuals with disabilities upon request. This publication is available on audiotape and upon request this material will be made available in other alternative formats to accommodate the needs of individuals with disabilities.