THE ACCURACY OF MOTOR AND SENSORY SCALES OF THE AMERICAN SPINAL INJURY ASSOCIATION STANDARDS WHEN ASSESSED BY TRAINED PHYSICAL THERAPISTS AND PHYSICIANS
Jones, Linda A.; Marino, Ralph J.; Kirshblum, Steven; Tal, Joseph

Purpose/Hypothesis : The aim was to assess reliability of trained examiners using the Motor and Sensory scales of the American Spinal Injury Association Standards. This reliability study was in preparation for the Proneuron Phase II autologous incubated macrophage study for the treatment of acute SCI patients.

Number of Subjects : Sixteen examiners (8 MDs, 8 PTs) with SCI experience and 16 patients with SCI (9 Complete-C, 6 Incomplete-I, 1-data unavailable) participated.

Materials/Methods : After a Standards training session, each examiner evaluated three patients for Motor (M), Light Touch (LT) and Pin Prick (PP) Scales. The following day, 15 examiners reevaluated one of the three patients. We computed intraclass correlation (ICC; one-way, random model) for inter-rater reliability and Pearson Correlation for Intra-rater. Percent Relative Error (PRE) was computed as the average absolute difference of examiners relative to the mean, multiplied by 100. PRE is provided with its 95% Confidence Interval (CI) by: a) Scale, b) Scale and Completeness of Injury c) Examiner Type (MD or PT). Statistical testing was done using independent and paired t-tests and repeated measures.

Results : Inter-rater reliability, as measured by ICC, was high for all total scores: M=0.97(95% CI: 0.94-0.99); LT=0.96(95% CI: 0.90-0.98), PP=0.88(95% CI: 0.76-0.95). Intra-rater reliability, as measured by Pearson correlation, was high as well: M=0.96, LT=0.96, PP=0.98. Inter- and intra-rater reliabilities did not differ significantly between the Scales. Combining all Scales, PTs and MDs demonstrated an intra-rater reliability of 0.98.

Average PRE by Scale: M=4.2%(95% CI: 0.2-8.2), LT=7.1%(95% CI: 3.8-8.3), PP=9.5%(95% CI: 4.7-14.3). There was a trend for PRE to be higher for PP than M(p =0.09). Otherwise, there was no significant difference between Scales. Average PRE by Completeness overall all scales: C=5.1%(95% CI: 2.6-7.6), I=10.6%(95% CI: 6.2-15.0). This difference was significant (p=0.02).

The PRE of PTs and MDs were very similar (6.3% and 6.4% respectively; 95% CIs: 3.8-8.7, 4.4-8.5 respectively) and not significantly different.

Conclusions : When conducted by trained examiners, Motor and Sensory examination of the standards is highly reliable for both PTs and MDs. Error is slightly greater when scoring PP relative to M and when scoring patients with I injuries relative to C. While expected, we quantify this difference as about 5% error for C and 11% for I.

Clinical Relevance : We demonstrated high reliabilities on the Scales of the Standards, which indicates that: a) Motor and Sensory Scales are accurate and b) Examiners - both PTs and MDs - can, when trained, use these scales properly. The level of inaccuracy we can expect for any given examiner differs by Scale but will, in most cases, not differ by more than 10% from a patient's true score. The error levels for patients with I injuries are approximately twice those of C,therefore further investigation is warranted to ascertain if this error can be reduced.
________________________________________