Reliability of the Blessed Telephone Information-Memory-Concentration Test

In-person cognitive evaluations can be costly and labor intensive in geographically widespread populations. Reliable telephone instruments that screen for cognitive status would greatly facilitate epidemiologic and other longitudinal studies. We evaluated the reliability of the Blessed Information-Memory-Concentration (IMC) test when administered by telephone. Eighty-four subjects with a wide range of cognitive abilities were administered the Blessed IMC twice over a 3-week interval. Forty-nine of the subjects were administered the test both by telephone and in-person, and 35 of the subjects were tested twice by telephone. Spearman's rank correlation was used to compare scores of the different administrations (.96; P < .001) and to examine test-retest reliability (.96; P < .001). The Blessed Telephone IMC (TIMC) test exhibits excellent reliability both when compared to in-person administration as-well as in test-retest results. The Blessed TIMC appears to be a practical instrument for population and longitudinal studies when in-person assessment is not feasible.

The need for reliable telephone measurements of cognition has become increasingly apparent in recent years. Hence, several instruments have been developed to evaluate twin or to follow inaccessible subjects.4~~ Some of these procedures necessitate mailing materials ahead of time and require up to an hour for completio11.33~ Several briefer instruments, such as the Telephone Interview for Cognitive Status (TICS) and the modified version (TICS-M),5-7 correlate well with the Mini-Mental State Examination (MMSE),B and can prpvide useful information. In our investigations of a geographically dispersed cohort, we needed an instrument that could be administered over the telephone and that would permit direct comparison with in-person longitudinal data.
In this study, therefore, we examined the reliability of a telephone-administered Blessed Information-Memory-Concentration test (TIMC) as compared to inperson administration, and of repeat-telephone administration (test-retest reliability) in subjects with a range of Blessed scores.

Subjects
Thirty-nine men and 45 women, aged 50 to 98 years (mean, 74 years; SD, 9.4) were recruited from neurology clinics, longitudinal studies, and clinical trials at the Johns Hopkins School of Medicine. The mean education level for the subjects was 12 years (range, 4-20 years; SD, 3.7).
Fifty-nine (70%) of the subjects met criteria for dementia,g primarily Alzheimer's disease (n = 55). The remaining subjects with dementia had multi-infarct dementia (n = 2) and normal pressure hydrocephalus (n = 2). Twenty-five subjects were considered normal, although two of these individuals were diagnosed with depression according to DSM-111-Rg criteria.

Instrument and Procedures
The Blessed Information-Memory Concentration (IMC) Testlo is a brief mental status instrument that has been used widely in clinical populations and research studies.11-13 The Blessed IMC test lends itself to telephone administration, because it contains no visual prompts or written instructions, and administration time is under 10 minutes. A l l participants were interviewed by a registered nurse or a trained psychometrist with experience testing Alzheimer patients. During the Blessed TIMC administration, subjects were requested not to use clocks or calendars to assist them in answering questions. In addition, caregivers were asked to provide no cues or answers to test items even if the participant requested help.
To evaluate the reliability of telephone administration as compared to in-person measurements, 26 subjects were administered the Blessed IMC test followed by the telephone procedure (TIMC), while 23 subjects received the tests in reverse order. The order of administration depended on scheduled visits to the center. Subjects who had been tested in person were retested by phone within 4 weeks; subjects were tested by phone first if they were scheduled to return to the center within the same amount of time. The average interval between administrations was 21 days (range, 10-38 days). There were approximately equal numbers of subjects in each of the following Blessed score intervals as determined by initial score: M,5-9,10-14,1519, and 20 and above.
To determine test-retest reliability, 35 participants were administered the Blessed TIMC test on two occasions separated by an average interval of 20 days (range, 12-31 days). Equal numbers of subjects were recruited for each of the Blessed score intervals as described above.

Analysis
The correlation between in-person and telephone scores was assessed using the Spearman's Rank Correlation Coefficient.14 Analyses were performed to determine if there were significant differences between the Blessed TIMC and IMC scores using a paired t test. Separate analyses were performed by gender, age, education, and the order of the test (telephone or in-person first). A multiple-regression analysis was also performed to determine if the difFerences between the Blessed TIMC and IMC scores could be predicted by any of these covariates. A 95% prediction interval based on a normal distribution was also calculated, lvvhich would give the likely difference between the Blessed TIhIC and IMC scores for an individual. Test1* was used to determine whether one instrument tended to favor a correct response more than the other instrument. If neither instrument favors a particular response, correct or incorrect, we would expect about an equal number of people giving the two types of discordant answers (i.e., correct in persodincorrect by phone vs. incorrect by phondcorrect in person). Similar analyses were performed to evaluate the association between the test and retest Blessed TIMC scores.

RESULTS
As shown in Figure 1, Blessed IMC scores were highly correlated with Blessed TIMC scores (Spearman's rank correlation = .96; P < .OOl). Table 1 shows average differences between Blessed TIMC and IMC scores for gender, age, education, and order of administration. 'Age and education were dichotomized using the median as a cutoff.

Distribution of score differences between Blessed
A multiple-regression analysis was performed including these covariates, and none were significant. The average score difference of the TIMC-IMC was -0.05, and not statistically different from zero (to,, = -0.11; P > .5). The distribution of score differences, shown in Figure 2, can be approximated by a Normal distribution with mean = -0.05 and variance = 6.75. A95% prediction interval for the difference in scores is -5.16 to 5.06 points, as suggested by Figure 2.
Exact percent agreement for each one of the-test items is shown in Table 2. Agreement of 80% or more was obtained for most of the items (20/27). The following items had agreement of less than 80% -item 5: Name of this place; 6: What street is it on; 7: How long have you been here; 13: Part of the day; 15: Season; 24: Months backwards; and 27: Recall of name and address. For 24 of the items, the observed agreement was significantly higher (P < .05) than the expected agreement as calculated from the kappa statistic.15 Additional analysis of discordant answers (correct vs. incorrect) found systematic differences between the TIMC and IMC responses. Using McNemar's test, asymmetry was found in item 5: Name of this place, and item 6: What street is it on, which were answered incorrectly by a significantly higher number of subjects on the Blessed IMC test than on the TIMC. The exact P values for these two items based on McNemar's test were .013 and .006, respectively ( Table 2).
The Blessed TIMC test-retest correlation was also high, as shown on Figure 3 (Spearman's rank correlation = .96; P c .001). Age, gender, education, and time 1 score were not predictive of score differences in the two telephone administrations in a multiple-regression analysis.

DISCUSSION
The Blessed TIMC provides results comparable to the Blessed IMC test, exhibits excellent test-retest reliability, and can be a practical alternative when in-person evaluations are not feasible or more frequent assessments are desirable. Most instruments developed for telephone ~s a g e~.~J~J~ are either abbreviations of longer instruments or are unrelated to procedures generally administered in the clinic. A strength of the Blessed TIMC is that it allows administration of the same instrument in person or over the telephone, allowing item-by-item comparison of data. Although the majority of items were answered in the same way on the Blessed IMC and TIMC tests, two items, item 5: Name of this place and item 6: What street is it on? showed discrepancies between in-person and telephone administrations. Incorrect responses were more frequent at the research center, presumably due to the subjects'relative unfamiliarity with the center as compared to their homes. Omitting these two items may further strengthen the comparability of test scores for larger studies, but did not significantly affect the results of our study when excluded from the analyses.
The Blessed TIMC test has limitations common to all telephone instruments. While all subjects were able to complete testing, the procedure was sometimes difficult in the severely demented subjects, who had poor telephone communication skills and short attention spans. We could not always be sure that the subject was not using clocks or calendars to assist in answering orientation questions. On a few occasions, we perceived possible "cheating," including help by caregivers. When this occurred, the test was interrupted and the caregiver reinstructed not to give further assistance before proceeding. Lastly, impaired hearing can certainly limit test interpretation. We did not screen for hearing loss in our subjects, but none had obvious impairment that appeared to affect testing. In some situations, however, it may be difficult t o assess adequacy of hearing on the phone. The Telephone Blessed was highly acceptable to the subjects as a method for increasing frequency of data collection and following subjects who no longer return for study. One reason for its acceptance was the short amount of time needed for administration: average administration time was 5 minutes. Although generalizability to more diverse population samples remains to be determined, its brevity and reliability also make it potentially useful as a screening instrument. Studies of its sensitivity and specificity for detecting dementia are in progress a t our center. The Blessed TIMC test and other telephone instruments can be an efficient approach to some of the difficulties in longitudinal and epidemiologic studies of cognition in aging.