The Direct Assessment of Functional Abilities (DAFA): A Comparison to an Indirect Measure of Instrumental Activities of Daily Living 1

The Direct Assessment of Functional Abilities (DAFA) was designed as a direct measure of instrumental activities of daily living (lADLs) that could be compared with an indirect assessment of lADLs by the Pfeffer Functional Activities Questionnaire (PFAQ). The DAFA (28 demented and 15 control subjects) and PFAQ (subjects and informants) were administered twice, together with a brief cognitive battery. Demented subjects performed significantly worse on direct assessment (DAFA) than predicted by self-report (PFAQ), and overestimation of abilities increased with severity of dementia. In contrast, informants tended to underestimate abilities of demented subjects, but not to a significant degree. Control (nondemented) subjects had comparable results with the two methods. The DAFA may provide a more objective measure of functional status in demented subjects than do indirect methods of assessment.

theless, it is uncertain whether indirect methods are as accurate as direct methods for assessing lADLs.
Studies comparing the accuracy of indirect measures of lADLs with various direct methods are few in number. Loewenstein et al. (1989) compared patients' performance on the Direct Assessment of Functional Status (DAFS) to an indirect measure of lADLs (Blessed Dementia Rating Scale). Their results indicated that functional information obtained through direct assessment was superior to information obtained through indirect measures, since it was less likely to be prone to biases inherent in indirect measures, such as overestimation of patients' functional abilities. Reuben, Valle, Hays, and Siu (1995) and Rozzini, Frisoni, Bianchetti, Zanetti, and Trabucchi (1993) compared patients' performance on the Physical Performance Test (PPT), a direct measure of lADLs, to an indirect measure of lADLs (Lawton and Brody IADL Scale) in communitydwelling elderly populations. Results from Reuben et al. (1995) suggested that these subjects tended to underestimate their functional abilities and that there were weak to moderate associations among the different methods of assessment. Results from Rozzini et al. (1993) indicated that direct methods were more sensitive to functional impairment than were indirect methods of assessment.
Since relatively few studies have compared direct and indirect methods of assessing lADLs, and to our knowledge, those conducted have not had item to item correspondence between measures, we designed the Direct Assessment of Functional Abilities (DAFA) study to address this issue. The Pfeffer Functional Activities Questionnaire (PFAQ) was selected as the IADL measure for this study because it includes additional domains not typically assessed in IADL scales, making it more sensitive to early dementia. In addition, the PFAQ was selected because it is easily administered.
The DAFA was designed to measure lADLs queried in the PFAQ, with a direct correspondence between DAFA and PFAQ items. The instrument was designed to be administered to mild to moderately demented individuals in a clinic-based setting. This study compares the responses of subjects to the DAFA with those from both the subjects and informants to the PFAQ. In addition, the DAFA was administered twice to examine test-retest reliability.

DAFA Test Development
In designing the DAFA we adapted the ten items in the PFAQ, which encompasses seven functional domains of lADLs, for application to patients in the clinic. As shown in Table 1, the DAFA and the PFAQ have an exact item to item correspondence, and differ primarily in that the DAFA is a direct measure while the PFAQ is an indirect measure of assessment. The DAFA test is given in the Appendix.
Definitions of functional abilities used in scoring the DAFA are equivalent to those used in scoring the PFAQ. Items on the DAFA and PFAQ are both evaluated by using an integer score from 0 (independent functioning) to 3 (dependent functioning) as shown in Table 2.
For each item on the PFAQ, six possible choices are provided. Each response corresponds to an integer score from 0 to 3. The total score (0-30) on the PFAQ is the sum of the integer scores for the ten individual test items.
Each item on the DAFA is scored by first observing the component parts of each task. (See Appendix for the specific components of each item.) Each component part is assigned an integer score from 0 to 3. The overall score for each item (0-3) is the average of these component scores rounded to the nearest integer. The total score (0-30) for the DAFA is the sum of the integer scores for the ten individual test items.

Subjects
Forty-three subjects participated in this study. This sample size was based on the feasibility of recruiting and testing a group of subjects in approximately a one year period. The 28 demented subjects that completed the protocol were recruited from outpatient geriatric neurology clinics at The Johns Hopkins University School of Medicine and Bayview Medical Center in Baltimore, Maryland. Medical chart reviews were used to identify potential participants. Subjects were selected to capture a broad range of cognitive impairment as measured by the Mini-Mental State Exam (MMSE) (Folstein, Folstein, & McHugh, 1975). Control subjects were Making out insurance or Social Security forms, handling business affairs or papers, assembling tax records.
Shopping alone for clothes, household necessities and groceries.
Playing a game of skill such as bridge, or other card games, chess, working a hobby such as painting, photography, woodwork, stamp collecting.
Heat the water, make a cup of coffee or tea, and turn off the stove.
Prepare a balanced meal.
Keep track of current events, either in the neighborhood or nationally.
Pay attention to, understand, and discuss the plot or theme of a one-hour television program, get something out of a book or magazine.
Travel out of neighborhood, driving, walking, arranging to take or change buses, trains, planes.
Writing a check, recording it in check ledger, subtracting for correct balance.
Making out insurance form.
Shopping alone for basic necessities.
Playing bingo or checkers.
Fill pot with water, heat water, unplug pot, make coffee.
Make a sandwich.
Comment on current events in politics, sports or entertainment.
Summarize three main points of a passage from a story.
Report birth date, next national holiday, number and schedule of medications.
Locate cafeteria using directions provided.

114
The Gerontologist 1 Does without difficulty or advice, but more difficult than used to be, or never did and would find it difficult to start now.

2
Requires frequent advice or assistance which was not previously necessary.

3
Someone has taken over this activity completely or almost completely.
Performs without any difficulty or assistance.
"Difficulty," completes task successfully without cues, but verbalizes that it is difficult, or becomes agitated and frustrated, self-corrects.
"Assistance," verbal, visual or other cues required, directions repeated a third time, requires physical aid, requires three or fewer cues to complete task.
"Dependent," incorrectly performs, is unable, or refuses to continue a task, or fails to complete a task despite maximum number (  the first 15 volunteers from the Hopkins community who responded to posted advertisements regarding the study. The MMSE was also used to ensure normal cognitive status among controls. Seven additional demented patients and caregivers were contacted but declined participation. Eight additional demented subjects completed a first visit but did not return for the second visit due to illness or lack of interest in completing the protocol. All study participants were required to provide a reliable informant and to be at least 50 years of age. Table 3 summarizes the demographic information of study participants, and Table 4 summarizes the mental status scores of demented subjects. All control subjects were cognitively normal, and there was no evidence of depression in any of the subjects as measured by the Center for Epidemiological Studies-Depression (CES-D) scale (Radloff, 1977).

Procedures for Demented Subjects and Their
Informants.-Initially, the subjects' informants underwent the MMSE and Blessed Information-Memory-Concentration (Blessed IMC) test (Blessed et al., 1968) to document their normal cognitive status, followed by the Clinical Dementia Rating (CDR) scale (Hughes, Berg, Danziger, Coben, & Martin, 1982) and the PFAQ to obtain an informant-based assess- ment of the subjects' memory and functional abilities. They provided information to verify the subject's responses. Subjects were then administered a battery of tests according to the following protocol. First, to assess mental status, the MMSE, Blessed IMC, CDR, and CES-D were administered. Next, the PFAQ was given as a self-rated test of the subjects' functional ability. Lastly, a research assistant, blind to the results of the cognitive testing, administered the DAFA. Informants.-The procedures for control subjects were similar to those for demented subjects except for the informant's participation. Informants of control subjects completed the protocol by telephone and through mailings rather than in person. Instead of the MMSE, control subjects received the Blessed Telephone-Information-Memory-Concent ration (TIMC) test (Kawas, Karagiozis, Resau, Corrada, & Brookmeyer, 1995).

Procedures for Control Subjects and Their
Study Protocol.-A summary of the procedures listed above, in the order in which they were administered, is given in Table 5. The test battery was administered twice, approximately four weeks apart (range: 1.9-14.6 weeks), in an outpatient clinical setting with all tests completed in a single session. Control subjects, together with their informants, completed the protocol in approximately 1.25 hours (45 minutes for the DAFA). Together it took demented subjects and their informants longer to complete the protocol, with the most severely demented subjects taking up to 2.5 hours (approximately 1.5 hours for the DAFA). Informed consent was obtained from all control subjects. Consent for the severely demented subjects was provided by their informants.
Statistical Analysis. -Since each subject completed the test battery twice, the resulting scores from the subject's two visits were likely to be correlated. Random effects regression models (Laird & Ware, 1982) were used to account for such correlation. In addition to exploratory plots, random effects models were used to estimate the average difference between DAFA and PFAQ scores and to determine if this difference was significant. The models used the difference between DAFA and PFAQ scores as the response variable, and included a random intercept for each individual to account for the correlation. Random effects models also were used to determine whether such differences depended upon covariates (e.g., informant's relationship to subject or subject's dementia severity).
In an item-wise comparison of the DAFA and PFAQ, McNemar's Test was used to determine which specific items favored either overestimation or underestimation by indirect assessment as com-pared to direct assessment. Test-retest reliability of the DAFA was assessed by Pearson's correlation coefficient and the intraclass correlation coefficient. A paired t-test was used to determine if the average difference between DAFA scores from first and second visits was significant.

Results
Completion Rates of Direct Assessment -Occasionally subjects did not perform all of the items on the DAFA: 16% of subjects had one or more missing items at the first visit and 23% at the second visit. However, all subjects completed at least 70% of the DAFA items. Missing values were generally due to time constraints rather than a subject's inability to perform a task. When missing values occurred, mean substitution was used in calculating the total DAFA score.
Direct Versus Indirect Assessment. - Figure 1 shows plots of DAFA versus PFAQ scores from both subjects ( Figure 1A) and informants ( Figure 1B) at the first visit. For both the DAFA and PFAQ, a higher score indicates more impaired functional ability in lADLs. Direct assessment of demented subjects generally yielded higher scores than self-assessment, implying that demented subjects typically overestimated their functional abilities ( Figure 1A). In contrast, Figure 1B shows that informant-based assessment of demented subjects yielded scores near or higher than direct assessment scores. Thus, informants tended to underestimate slightly the functional abilities of demented subjects. Direct assessment of normal subjects was comparable to both self-and informant-based assessment, with scores clustered near perfect functional ability on both the DAFA and PFAQ, perhaps reflecting a ceiling effect.
On the basis of data from both visits and adjusting for the repeated measurements, the average amount by which demented subjects overrated their functional abilities was estimated to be 8.1 points (s.e. = 1.4, p < 0.01), while the average amount by which informants underestimated the functional abilities of demented subjects was estimated to be 0.9 point (s.e. = 1.1, p = 0.41). Control subjects showed no significant difference between DAFA and PFAQ scores,  with the average amount by which control subjects overestimated their functional abilities estimated to be 0.5 point (s.e. = 0.4, p = 0.21), while the average amount by which informants overestimated the functional abilities of control subjects was estimated to be 0.9 point (s.e. = 0.7, p = 0.07). In addition, the amount of over-and underestimation did not differ significantly at the two visits for any of the subject groups.

Effect of Relationship of Informant to Demented
Subject.-As a group, informants underestimated the functional abilities of demented subjects by an average of 0.9 point. When informants were separated into two groups, spouses and others (child, friend, paid caregiver), spouses slightly overestimated (mean = 0.4 point, s.e. = 1.4, p = 0.77) while the "other" group underestimated (mean = 2.6 points, s.e. = 1.6, p = 0.11) the functional abilities of the demented subjects. Neither the overestimation by spouses nor the underestimation by the "other" group was statistically significant. In addition, the average difference between the two informant groups (mean = 3.0 points, s.e. = 2.1, p = 0.17) was not significant.
Relationship Between Accuracy of Functional Ratings and Severity of Dementia.-\Ne next examined the relationship between MMSE score and the accuracy of self and informant-based assessment of functional ability, where accuracy is defined as the difference between DAFA and PFAQ scores (DAFA minus PFAQ). A positive value implies overestimation of abilities, while a negative accuracy implies underestimation of abilities. Figure 2 contains data from the first visit and shows that subjects' overestimation of functional abilities increased with dementia severity. In contrast, the accuracy of informants' ratings generally was not related to the severity of the subjects' dementia.
Although the MMSE score was found to have a statistically significant effect on the subject's accuracy (p < 0.01), the subject's MMSE score did not affect the informant's accuracy significantly (p = 0.66). Furthermore, the relationship between the subject's MMSE score and the subject's accuracy was found to be nonlinear. For example, the average increase in accuracy for subjects with MMSE scores of 15 as compared to MMSE scores of 10 was estimated to be 5.3 points (s.e. = 1.0), while the average increase in accuracy for subjects with MMSE scores of 20 versus 25 was estimated to be 2.5 points (s.e. = 0.5).
Examination of the relationship between Blessed IMC scores and accuracy, as well as between CDR scores and accuracy, yielded the same conclusion. Namely, subjects' accuracy significantly decreased with greater dementia severity, but informants' accuracy was not significantly affected by the severity of the subjects' dementia. Table 6 presents a comparison of indirect versus direct method of IADL assessment in demented subjects. Values represent the percentage of participants who, relative to direct assessment by the DAFA, overestimated or underestimated the functional abilities of demented subjects by the PFAQ at Visit 1. Items are marked for which  *Percentage of overestimation significantly differed from percentage of underestimation (p < 0.05). Note: Values are percentage of participants who rated functional abilities of demented subjects at Visit 1 either correctly or incorrectly (overestimated or underestimated). there was significant asymmetry; i.e., there was a significant difference between the percentage of subjects who overestimated and the percentage of subjects who underestimated. While overall, demented subjects overestimated their abilities, fewer subjects overestimated their skills in money management (items 1 and 2), shopping (item 3), and hobbies (item 4). A greater percentage of demented subjects overestimated their abilities in the domains of awareness (item 7) and reading (item 8). Similarly, a greater percentage of informants overestimated subjects' abilities in reading (item 8), and underestimated subjects' skills in meal preparation (items 5 and 6) and awareness (item 9).

Comparison of Direct Versus Indirect Assessment for Each Item.-
Test-Retest Reliability of DAFA.-lhe DAFA had excellent test-retest reliability as illustrated in Figure  3. The correlation between Visit 1 and Visit 2 DAFA measurements was estimated to be 0.95 (p < 0.01) using Pearson's correlation coefficient, and 0.95 (p < 0.01) using the intraclass correlation coefficient. Furthermore, the average difference between DAFA scores at first and second visits (second minus first) was estimated to be -0.05 point (s.e. = 0.48, p = 0.92), and not statistically different from zero.

Discussion
This study compared direct and indirect methods of IADL assessment to determine whether the two methods provided similar results. The Direct Assessment of Functional Ability (DAFA) was designed to evaluate lADLs in subjects with a range of cognitive impairments, and had a direct item to item correspondence to the Pfeffer Functional Activities Questionnaire (PFAQ), an indirect measure of lADLs. For comparison purposes, Table 7 shows additional characteristics of the DAFA and other direct measures of lADLs that have been developed.
DAFA scores were compared to both subject and informant responses to the PFAQ. The results indicate that demented subjects significantly overestimated their functional abilities when measured by indirect methods of assessment, whereas there was a trend for informants to underestimate the subjects' functional abilities. Although informants tended to underestimate subjects' functional abilities, their ratings were more accurate than the subjects' selfratings. Subjects with greater cognitive impairment showed poorer judgment of their functional status, while informants' accuracy was not significantly affected by subjects' cognitive ability. Although the results suggested that spouses were better informants for demented subjects than "other" individuals, this study did not allow for a more detailed comparison of informant types, as few informants were friends, siblings, or paid caregivers.
Demented subjects tended to overestimate their abilities most often on an item in the domain of awareness and on an item evaluating reading comprehension. They overestimated their abilities less frequently on items involving money management, shopping, and hobbies. It is conceivable that subjects were more optimistic about their awareness and reading comprehension because there was no concrete way for them to measure their loss of abilities in these domains. In comparison, domains such as money management and shopping offer the subjects more tangible evidence of their declining abilities. Informants may have overestimated subjects' abilities on reading comprehension for the same reasons that subjects overestimated them. Items measuring subjects' abilities in the domain of meal preparation and an item in the domain of awareness were underestimated by informants possibly because they may have assumed complete responsibility for these tasks prematurely.  (Reuben & Siu, 1990); b (Mahurin, DeBettignies, & Pirozzolo, 1991); c (Loewenstein, et al., 1989).

Vol. 38, No. 1,1998
Although we compared self-reported and informant-based scores to direct assessment scores, the direct method of assessment does not necessarily represent the gold standard. In fact, it may be true that DAFA scores detected functional disability that was imperceptible to subjects and informants. This notion has been referred to as "preclinical" disability and has been studied by Fried et al. (1996). Their studies indicate that direct measures of assessment may identify individuals at risk of developing functional disability despite failure to report difficulty on self-rated assessments. Therefore, the DAFA has potential for detecting disability that will ultimately have clinical significance. Myers, Holliday, Harvey, and Hutchinson (1993) suggest that one of the advantages of self-ratings is that they are influenced by a longer time frame and performance in a variety of environments. Self-ratings may be affected by adaptations subjects have made which allow them to perform functional activities. The DAFA, a direct method of assessment of lADLs conducted in an outpatient setting, removes the influence of a familiar environment. While standardizing the environment does not account for adaptations, a strength of direct methods of assessment is that they allow for easier comparison of scores between subjects.
The DAFA was administered reliably to subjects with different degrees of dementia. Occasionally subjects did not complete all of the items included in the DAFA. Since using this direct method of IADL assessment required more time and effort from study subjects, it is not unexpected that more data were missing than with the indirect method of assessment. For this study, it took demented subjects approximately 1.5 hours to complete the DAFA. However, we administered DAFA items in different locations (e.g., cafeteria, gift shop, exam room) which accounted for a portion of the time to complete the test. This length of time could be reduced by creating a test environment where all of the DAFA items could be administered in a central location.
Since the sample size for this study was relatively small, our results need to be confirmed in a larger, more diverse sample of cases and controls. Moreover, future studies might be conducted to observe the effect of informant relationship on assessment of functional abilities in greater detail. The effect of the administrator of the direct assessment on subjects' IADL scores could also be examined. In addition, the effect of such demographic variables as education and race on accuracy of functional ability assessment could be of interest. Furthermore, from this study it was difficult to determine if direct methods of assessment were more accurate than indirect methods for control subjects because of ceiling effects. Adding a timed measurement to each item might provide further information to distinguish among control subjects' functional abilities. Also, a more diverse socioeconomic background among control subjects might increase the variability in their scores. This would increase the sensitiv-ity to detect differences between the direct and indirect methods of assessment in control subjects.
More accurate assessment of IADL functioning may be gathered by direct methods of assessment than by either self-report or informant-based methods. In addition to avoiding the potential biases associated with indirect measures of lADLs, direct methods of assessment conducted in an outpatient clinical setting, such as the DAFA, have the advantage of removing the influence of a familiar environment and potentially detecting preclinical disability. In this regard, direct methods of assessment can provide additional information about subjects' functional abilities that may not be reported by subjects or informants.
Often, however, direct methods of assessment conducted in an outpatient clinical setting are not feasible due to time and financial constraints. When direct methods of assessment are not feasible, the results of the comparison presented between direct and indirect methods of IADL assessment should be taken into consideration.