Gauging the Transdisciplinary Qualities and Outcomes of Doctoral Training Programs

A key assumption underlying recent investments toward establishing transdisciplinary research centers and training programs is that cross-disciplinary research and training provide a stronger basis for achieving scientific and societal advances than unidisciplinary programs. It is necessary to develop reproducibleand reliable criteria for identifying the distinctive qualities of cross-disciplinary research and training programs, especially in the field of urban and regional planning. The current study provides an exploratory first step toward that goal. A composite scale designed to measure the transdisciplinary qualities of doctoral dissertations as an important product of one's intellectual development and graduate training was constructed and administered in the present study. Dissertations completed over a twenty-five-year period by Ph.D. candidates within an interdisciplinary doctoral training program were rated by two independent reviewers across multiple dimensions of transdisciplinary integration and scope. Departmental as well as gender differences were found on several dimensions of transdisciplinarity.

environmental law courses. Schmelzkopf (2002) discusses the benefits of interdisciplinary courses in the geography of tourism. Inam (2002) points out the need for an interdisciplinary, in-depth, and problem-driven pedagogical approach to urban design. Both the American and European associations of planners (ACSP and AESOP, respectively) emphasize the collaborative nature of planning practice and the cross-disciplinary background and expertise of planning students in the guides to planning schools in North America and Europe (Rosenbloom 2000;Bergstrom and Cars 1998).
Planning scholars have recognized the need for interdisciplinary research in the planning profession as well. Longley (2000), in an editorial published in Computers, Environment and Urban Systems, emphasizes the interdisciplinary nature of the journal, with its emphasis on linking computer technology, spatial analysis, and decision making. Fulong (2002) reviews three books on the city. All three offer interdisciplinary approaches, acknowledging the great complexity that characterizes the future prospects of cities in the era of globalization and new global-local relations. Ellemor (2003) discusses place and identity and concludes that ongoing interdisciplinary and theoretically informed empirical research is necessary to understand the complex context of people-place relationships in settler societies. Edwards and Steins (1999) advocate an interdisciplinary, holistic approach to natural resource policy analysis, while Burgi and Russell (2001) suggest that the study of landscape changes requires an interdisciplinary approach bridging landscape ecology, history, and urban design.
Whereas many scholars believe that cross-disciplinary approaches to complex problems are superior to unidisciplinary ones, the empirical evidence for this assumption is quite limited (cf. Kahn 1993;Kessel, Rosenfield, and Anderson 2003;Stokols, Fuqua, et al. 2003). Very few studies have been conducted to compare the collaborative processes and scientific and training outcomes associated with cross-disciplinary versus unidisciplinary research (Latucca 2001b;Nash et al. 2003;Younglove-Webb et al. 1999). Moreover, before the outcomes of these alternative scientific approaches can be evaluated, explicit criteria for operationalizing the cross-disciplinary scope of research projects and for distinguishing among those that are of narrow versus broad in scope must be articulated and refined.
The major purpose of this investigation is to develop reproducible and reliable criteria for distinguishing among research projects-particularly, doctoral dissertations-in terms of their relative transdisciplinary qualities. Our use of the term, transdisciplinary follows Rosenfield's (1992) discussion of three different types of cross-disciplinary researchnamely, multidisciplinar y, interdisciplinar y, and transdisciplinary projects. Multidisciplinary projects are those in which researchers representing different fields contribute methods and ideas from their respective disciplines toward the analysis of a particular research question. Interdisciplinary projects involve closer and more frequent collaborative exchanges among researchers drawn from different fields who are working together on a common problem. Transdisciplinary projects are those in which researchers from different fields not only work closely together on a common problem over an extended period but also create a shared conceptual model of the problem that integrates and transcends each of their separate disciplinary perspectives. Rosenfield views transdisciplinary projects as the most robust form of cross-disciplinary collaboration since each researcher sets aside his or her own disciplinary perspectives during the process of collaborating with colleagues to develop a more integrative conceptualization of the problem.
In the present study, we focus on developing reproducible criteria to assess the transdisciplinary qualities of doctoral training programs and dissertation research projects. Doctoral research training and completion of the dissertation project constitute an important and formative phase of a scientist's intellectual development. Planning-training programs, in particular, aim at preparing future planners for the interdisciplinary nature of the profession. It is through doctoral training and completion of the requirements for the Ph.D. degree that young scientists acquire the core intellectual values and research strategies that provide the foundation for their postgraduate careers. And in the context of increasing calls for cross-disciplinary, integrative research on a wide range of complex problems, some scholars have suggested that doctoral training programs can play an important role in developing a new generation of transdisciplinary scientists (Latucca 2001a(Latucca , 2001bNash et al. 2003).

᭤ Challenges and Opportunities Associated with Transdisciplinary Doctoral Training Programs
Earlier studies concerning the development and evaluation of cross-disciplinary training programs have emphasized multidisciplinary curricula and the teaching of varied subjects from two or more disciplinary perspectives (e.g. , Bradbeer 1999;Davis 1995;Clarke and Agne 1997;Pezzoli and Howe 2001;Wineburg and Grossman 2000). Recent analyses of the processes and outcomes associated with transdisciplinary scientific collaboration highlight certain components that are believed to play a key role in transforming disciplinary-oriented researchers into transdisciplinary ones (cf. Fuqua 2002;Thompson-Klein 1996;Rosenfield 1992;Nash et al. 2003;Stokols 1998;Stokols, Grzywacz, et al. 2003). Among those influential components are required interdisciplinary courses (taught by an interdisciplinary team or by interdisciplinarytrained professors), intellectual exchanges with faculty members and fellow students, internship experiences that afford students the opportunity to collaborate with practitioners from several different fields, and research mentorship during the development and completion of theses and dissertations (Brooks et al. 2002;Mitrany and Stokols 2003;Schon 1987). These as well as other analyses of cross-disciplinary training programs propose and evaluate explicit tools for promoting interdisciplinary education (e.g., Stubblefield, Houston, and Haire-Joshu 1994;Shepard, Yeo, and McGann 1985;Nash et al. 2003).
The international congress Universities' Responsibilities to Society (Nicolescu 1997) offered proposals for promoting transdisciplinary training in universities. One proposal was for university administrators to ensure that 10 percent of the teaching time provided by faculty members in their institutions be devoted to covering transdisciplinary issues, although they do not offer explicit details about the content of such teaching. Latucca (2001a) gathered information on the processes, contexts, and outcomes of interdisciplinary teaching and research in several universities in the United States. She was able to identify certain characteristics of transdisciplinary courses but emphasized that they are scarce in U.S. colleges and universities (multidisciplinary and interdisciplinary courses are much more common).
A common assumption of earlier analyses was that the content and structure of research training programs can play a substantial role in cultivating a transdisciplinary orientation among young researchers. For instance, by introducing students to an interdisciplinary worldview and broadening their conceptualization of particular research topics, instructors can help cultivate a transdisciplinary scientific orientation; several interdisciplinary courses are based on this assumption. Latucca (2001b) noted that there is little evidence to support the claims made about the outcomes of interdisciplinary courses for students. She is concerned about the lack of concrete evidence of improved student learning or increased capacity for integration and synthesis of concepts, theories, and methods from disparate disciplines.
One model of cross-disciplinary education is the transdisciplinary doctoral training program. Graduate students in such programs develop knowledge and skills about how their research interests can be addressed from the vantage point of several different disciplinary perspectives. Students enrolled in these programs develop professional identities as transdisciplinary scientists rather than establishing narrower identities anchored in a single, traditionally defined discipline (Nash et al. 2003). Graduate students enrolled in masters and Ph.D. training programs in urban planning typically are encouraged to expand their intellectual horizons far beyond their earlier baccalaureate degree training by cultivating a more holistic, transdisciplinary perspective on research and community problem solving (Bradbeer 1999;Brooks et al. 2002;Schon 1987).
Global megatrends, including economic restructuring, migration, and environmental degradation, have profoundly transformed planning practice; this reality needs to be reflected in planning education. Pezzoli and Howe (2001) conducted a content analysis of sixty-nine planning syllabi. Their analysis highlighted the interdisciplinary nature of globalized planning pedagogy and the greater emphasis on planning theory and history and economic development.
Transdisciplinary scientists, as compared to their unidisciplinary counterparts, tend to be inclusive rather than exclusionary thinkers, broad-gauged and contextually oriented in their theorizing and research, methodologically eclectic, optimistic in pursuing transdisciplinary research goals, open-minded and respectful of divergent viewpoints, and adept at promoting good will and cross-disciplinary tolerance (Stokols 1998). Roland et al. (2002) suggested that the transdisciplinary research orientation, once acquired by a trainee, is manifested at multiple levels: knowledge, attitudes and values, and behavior.
Several barriers to transdisciplinary research and training are mentioned in the literature (Stokols, Fuqua, et al. 2003;Thompson-Klein 1996). Among them are communication constraints posed by discipline-specific jargon. White (1998) reflected on five years' experience of interdisciplinary research on planning-related issues and suggested that entrenched academic territories, derived from disciplinary and data differences, make managing an interdisciplinary team of researchers a daunting task. Disciplinary protection of turf is another barrier (Campbell 1969a;Pellmar and Eisenberg 2000). Pride in one's discipline and its methods are often instilled during graduate training. As a consequence, other disciplines may come to be viewed as less rigorous or less important than one's own. To work together, people must understand and appreciate the value and limitations of both their own and others' methods. Career challenges are another potential constraint on the development of a transdisciplinary research orientation. The length of time required to complete training in more than one field can be substantial. Acquiring a depth of knowledge in one field during graduate school sometimes precludes obtaining breadth in other disciplines within a reasonable period (Sigma Xi 1988). Junior scientists also are concerned that training in transdisciplinary fields will not prepare them for long-term success in their careers (Pellmar and Eisenberg 2000).
Measuring the essential attributes, let alone the tangible outcomes, of a transdisciplinary training program is a complicated issue (Stubblefield, Houston, and Haire-Joshu 1994). Too little is known about ways to quantify the qualities and outcomes of such programs. Field, Lee, and Field (1994) note the lack of concise assessment criteria tailored to interdisciplinary studies, although there is no shortage of theoretical and practical material on evaluating other types of higher education. Furthermore, the indicators of successful transdisciplinary training may not be realized until several years after the training period has ended. Nash et al. (2003) suggest that the degree of transdisciplinarity of a student's work is partly reflected in research that is coauthored by collaborators from different disciplines and publications that incorporate theoretical frameworks and methodological approaches from different fields. Other indicators of success are employment in multidisciplinary centers and academic appointments outside the primary discipline. Nash et al. recommend that longitudinal evaluations of training outcomes be conducted by following the career path of each transdisciplinary trainee over several years and even decades. Assessment of transdisciplinary projects by traditional means and structures falls short in several respects. Russell (2000) points out that research across disciplines rarely satisfies the criteria and standards of each of the disciplines involved. Moreover, transdisciplinary research generally creates its own criteria and standards because of its unique emergent qualities (Thompson-Klein 1990, 1996. These qualities are not explicitly taken into account by current assessment strategies. Researchers trying to assess the quality of transdisciplinary research face this challenge, and currently, there is no one agreed on set of measures that has been used to gauge the transdisciplinary qualities of Ph.D. training programs and research products. As a result, it is difficult to compare different transdisciplinary programs and research projects, let alone evaluate their relative quality.

᭤ Assessing the Processes and Products of Transdisciplinary Training Programs
At least two methodological strategies can be used to assess the transdisciplinary qualities and tangible outcomes of doctoral training programs and research projects. One approach is to assess the ongoing processes of transdisciplinary training as it occurs within specific academic settings. Process measures of the qualities and outcomes of transdisciplinary training include assessments of the ways in which particular curricula, intellectual exchanges with faculty and fellow students, and research mentorship influence the development of students' attitudes, values, and behavior, which, taken together, reflect their intellectual orientation and their adoption or rejection of a transdisciplinary scientific perspective. For instance, doctoral trainees can be interviewed and surveyed before and after they participate in particular courses or research projects that emphasize a transdisciplinary scientific orientation, and they can be asked to evaluate both the quality and influence of those courses and projects on their own intellectual development (Fenwick 2001;Pezzoli and Howe 2001). Such interviews and surveys include a variety of qualitative and quantitative measures that are administered during the time that students are enrolled in the training program. Examples of such measures are provided by Field, Lee, and Field (1994) and Mitrany and Stokols (2003). One limitation associated with process measures of transdisciplinary training is that they are locally designed and highly customized to the specific circumstances of the target program and therefore may not generalize widely to other academic settings beyond the immediate training context.
A second approach is to assess the transdisciplinary qualities and scope of tangible research projects as reflected in published papers, theses, and dissertations. Although product assessments do not capture the dynamics of transdisciplinary training processes as they occur over time, the development of criteria for evaluating the transdisciplinary qualities and scope of research projects has the advantage of developing standardized measurement tools that can be applied reliably and validly across a wide range of training programs and research projects.
In this article, we focus on the second approach, namely, the development of objective indices for assessing the transdisciplinary qualities and scope of doctoral dissertation projects. The doctoral dissertation is an important milestone in the trainee's intellectual development and provides important evidence regarding the individual's scientific orientation and the directions of research that will be pursued during the next postgraduate phase of this person's career. The development of reliable and valid criteria for assessing the transdisciplinary qualities of doctoral dissertations and other published research products can provide an objective basis for evaluating the short-and long-term scientific and educational outcomes associated with transdisciplinary training programs ("Editorial Note" 2001). For example, future investigations might examine the scientific citation rates and relative scholarly and public policy effects of transdisciplinary versus unidisciplinary dissertation projects in an effort to calibrate the "value-added" or "return on investment" attributable to transdisciplinary training programs (Nash et al. 2003;Stokols, Fuqua, et al. 2003).
For purposes of the immediate study, our major goal was to develop objective measurement criteria that can be used to assess the transdisciplinary qualities of diverse training programs and research projects. This study is a first step for establishing valid and reliable measures of the transdisciplinary characteristics of research. Because of the exploratory nature of the study, we did not emphasize validation of the measures or their generalizability across multiple graduate training programs. Rather, we tried to develop reliable and reproducible measures within one particular interdisciplinary training program rather than across multiple graduate programs in planning.

Identifying the Transdisciplinary Qualities of Doctoral Dissertations
Prior evaluations of cross-disciplinary scientific collaboration and training suggest certain criteria for gauging the transdisciplinary qualities of a doctoral dissertation and other research projects (Kahn 1993;Nash et al. 2003;Pellmar and Eisenberg 2000;Rosenfield 1992;Stokols, Fuqua, et al. 2003). For instance, the creation of integrative conceptual frameworks is a hallmark of transdisciplinary science (Rosenfield 1992). Additional qualities of a dissertation that appear to be closely associated with a transdisciplinary approach to research include the multidisciplinary composition of the dissertation committee and the diversity of research methods used in the study (Nash et al. 2003). Still other criteria for judging the transdisciplinarity qualities of a dissertation (or other research products) include the contextual scope of the research (Stokols 1987), the number of analytical levels bridged in the study (e.g., from molecular to molar levels of analysis), and the degree to which the researcher makes an effort to translate his or her concepts, methods, and findings into guidelines for community problem solving (Turkkan, Kaufman, and Rimer 2000). Efforts to translate research concepts, methods, and findings into guidelines for community problem solving promote transdisciplinary integration because they expose researchers to greater opportunities for integrating their scholarly perspectives with the diverse professional orientations of community practitioners (Altman 1995;Stokols, Grzywacz, et al. 2003).

Research Setting
The present study focused on the doctoral training program in the School of Social Ecology at the University of California, Irvine. This academic setting was chosen for the present study for several reasons. First, the academic mission of the school and its predecessor, the Program in Social Ecology, is to educate undergraduate and graduate students to analyze research and community problems from a broad ecological perspective that integrates several different disciplines. Second, the School of and Program in Social Ecology have been in existence for more than thirty years and have produced several thousand undergraduate alumni and hundreds of masters and Ph.D. graduates during that time. The Program in Social Ecology initially was founded at the University of California Irvine in 1970 as an experimental academic unit. The program grew progressively over the next two decades and was formally designated as the School of Social Ecology in 1992 by the University of California Regents. The school currently incorporates four multidisciplinary departments that span more than fifteen different disciplines; it enrolls approximately 2,500 undergraduate majors and 170 graduate students. The four departments are Planning, Policy, and Design; Criminology, Law, and Society; Environmental Health Science and Policy; and Psychology and Social Behavior.

Participants and Procedures
An archival survey of dissertations completed by Ph.D. students in the school was conducted to determine the degree of transdisciplinarity reflected in each dissertation. This entailed the creation of a composite scale for measuring the transdisciplinary qualities of a research project and applying this scale to gauge the transdisciplinary scope of each dissertation produced by doctoral candidates enrolled in the Ph.D. programs within the School of Social Ecology. One hundred forty-four dissertation theses out of the 156 written in the school were surveyed; 53 percent of them were written by females, and 47 percent were written by males. Twelve dissertations completed by doctoral candidates in the school were not included in the analyses because they lacked certain identifying information that was crucial for assessing the transdisciplinary qualities of the research project. Those dissertations included in the analyses were assessed independently by two different reviewers and rated on several dimensions or qualities of transdisciplinarity. Prior to the actual data collection, reviewers were given detailed instructions about the operational criteria associated with each dimension of transdisciplinarity and interrater reliability was gathered on twelve randomly chosen dissertations (from among the 144 dissertations included in the sample) to ensure that adequate levels of agreement were obtained among the ratings provided by multiple reviewers.

Composite Scale for Gauging the Transdisciplinarity of Dissertation Projects
A composite scale for gauging several transdisciplinary qualities of research products (e.g., dissertations) was constructed. This scale incorporates six components or qualities of transdisciplinarity, each of them operationalized as a quantifiable measure. The six components are (1) the transdisciplinary quality of the research topic and its conceptualization, (2) the multidisciplinary composition of the dissertation committee, (3) the diversity of research methods used in the study, (4) the contextual scope reflected in the student's conceptualization of the core research topic, (5) the number of analytic levels incorporated into the research, and (6) the translation of the research concepts, methods, and findings into community problem-solving strategies. After surveying every research project, each subscale measure was multiplied by a standardization weight so that each of the six components had an equal influence on the overall summary index of transdisciplinarity assigned to each dissertation. 1 The six subcomponents of transdisciplinarity are described further below and are summarized in Table 1.
The operational components of transdisciplinarity were chosen to represent key conceptual principles of transdisciplinary science. Transdisciplinarity of the research topic and its conceptualization refers to the degree of cross-disciplinary integration, which bridges the theories and methods of two or more fields, achieved by the student through his or her literature review and theoretical conceptualization of the topic. Following the definitions suggested by Rosenfield (1992) and by Latucca (2001a), researchers can achieve varying levels of transdisciplinary integration in their work. This continuum ranges from unidisciplinary research, where no integration occurs, through a moderate degree of integration (interdisciplinarity), to a full integration of several fields evidenced by the creation of a new conceptual framework that transcends the theories and methods of individual disciplines. The operational measure of a research topic's transdisciplinarity included two separate variables-an index of cross-disciplinary integration and a count of the number of fields brought together within the dissertation project.
The second objectively recorded dimension is the multidisciplinary composition of the dissertation committee. An important facilitator of transdisciplinary integration is the number of disciplinary perspectives represented within a research group (Stokols, Fuqua, et al. 2003). Although it is possible for a single scientist to achieve an integration of several fields found to be essential for understanding the research topic (a noncollaborative form of transdisciplinarity), scientific collaboration occurs when researchers from several fields join together to investigate a problem of mutual interest. In the case of a doctoral dissertation, this component is operationalized as the multidisciplinary composition of the dissertation committee (members representing several different fields)-a key factor that can facilitate the exposure of the student to multiple disciplinary perspectives and encourage the doctoral candidate to approach his or her research topic from a broader, more transdisciplinary perspective and to integrate those perspectives within the dissertation study.
The third dimension is the diversity of research methods used in the dissertation study (e.g., ranging from quantitative to qualitative and from experimental to nonexperimental measures and research designs). The inclusion of multiple research methods and the triangulation of qualitative and quantitative measures within the same study affords greater convergent validity than can be achieved through the use of singular, discipline-specific methods (Bryman 1992;Campbell 1969b;Gaber and Gaber 1997;Greene, Caracelli, and Graham 1989;Kelle 2001). Inclusion of multiple research methods reflects the greater tolerance for multiple research perspectives and inclusive thinking-attributes that are inherent in a scientist's "transdisciplinary ethic," as discussed by Stokols (1998).
The fourth dimension is the contextual scope of the author's conceptualization of the research topic or the extent to which the dissertation study reflects a broad or narrow range of geographical, temporal, and sociocultural phenomena (Stokols 1987). Contextual scope pertains the breadth of spatial, temporal, and sociocultural factors that are encompassed within a conceptual and/or empirical analysis of a particular topic. For instance, human response to environmental

Levels of analysis bridged by the dissertation research Number of levels mentioned
The levels of analysis are (1) molecular/cellular, (2) organismic/ individual, (3) group/inter-personal, (4) organizational/ institutional, (5) community, (6) regional, (7) social/ national, and (8) global 1 to 8 Number of analytic levels integrated within the dissertation If the answer to the previous one was higher the 1 1 to 7 Translation of research concepts, methods, and findings into community problem-solving strategies Community problem solving No mention of it 1 Some mention of possible policy intervention implication in the future 2 Proposed policy/intervention that is not carried out 3 Evaluate an existing one 4 Propose and evaluate a new intervention/ policy 5 a. Every method used is assigned one point. The points are then summed. The higher the number of points, the more inclusive and diverse the research methods. b. The points for temporal, spatial, and sociocultural scope are summed. The higher the number of points, the more contextual the research. stressors such as noise and high density can be analyzed in the context of a single setting or multiple settings (Cohen et al. 1986), thereby varying the spatial or geographic scope of the analysis (e.g., assessing the behavioral effects of exposure to high density within residential settings alone or within multiple settings of the individual's life situation-including residential, employment, and recreational settings). Similarly, the temporal and sociocultural scope can be relatively narrow or broad depending on the breadth of the researcher's conceptualization of the research topic. For purposes of the present study, we assume that the contextual scope reflected in the student's conceptualization of the core research topic is positively associated with the transdisciplinarity of the dissertation-the broader the contextual scope of analysis, the more transdisciplinary the dissertation is assumed to be. The fifth dimension measured in this study is the number of analytic levels bridged by the dissertation study. Conceptual analyses bridging molecular/cellular to macro/community levels of analysis can generate insights and discoveries that might be missed by working within a single analytic level (Miller 1986).
In the present study, both the number of analytic levels mentioned by the researcher and the number of levels actually crossed or integrated within the dissertation research were taken into account by the reviewers of each thesis. The greater the number of analytic levels crossed (e.g., ranging from molecular/genetic, psychological/developmental, small group/organizational, and macro/community levels), the higher the probability of achieving an integrative conceptual analysis of the research topic (Bronfenbrenner 1979).
The last dimension measured is the degree to which the author makes efforts to translate his or her research concepts, methods, and findings into specific recommendations for community problem solving. As noted earlier, these efforts promote transdisciplinary integration by enabling researchers to integrate their scholarly perspectives with the diverse professional orientations of community practitioners (Altman 1995;Butterfoss, Goodman, and Wandersman 1993).

Interrater Reliability
Multiple independent reviewers used the composite scale, incorporating the six operational dimensions shown in Table  1, to assess the transdisciplinary qualities of the doctoral dissertations produced by Ph.D. candidates within the School of Social Ecology. As a basis for estimating the reliability of the scale, independent ratings of twelve dissertations provided by two different reviewers were compared. Interrater reliability across the two sets of ratings was significant (r 2 = .852, p < .01).

Transdisciplinary Qualities of Dissertation Research Projects
Degree of transdisciplinary integration. The Degree of Transdisciplinary Integration component scores revealed three main groups. The first group included 45 percent of the dissertations that showed no transdisciplinary integration at all. The second group included 44 percent of the total dissertations that revealed a moderate level of interdisciplinary integration reflecting linkages between the concepts and methods of two or more fields but without the development of a new and integrative conceptual model. Eleven percent of the dissertations constituted the third group, which achieved a level of integration in which a new conceptual model was created. The second measure of integration was simply the number of fields brought together in the research. Fifty-nine percent of the dissertations brought together two fields, and 12 percent brought together three fields. Only 29 percent of the dissertations stayed within the limits of a single field.
Multidisciplinary composition of the dissertation committee. Most dissertations in the school were written under the supervision of advisors from more than one field. In 42 percent of the dissertations, two disciplines were represented in the committee, and in 28 percent of the dissertations, three or more disciplines were present. In 30 percent of the dissertations, all committee members were from the same field. (Only one-third were dissertations that stayed within the limits of a single field. There was no linkage between number of fields represented by committee members and the number of fields brought together by the student in the dissertation itself.) Diversity of research methods. The diversity of research methods used in the dissertations was rated on a scale ranging from 3 to 7. Three components composed this measure (i.e., qualitative vs. quantitative methods, or both; experiment vs. quasi experiment vs. nonexperiment; and laboratory vs. field observation). The scale for each component was 1, 2, or 3 (e.g., a score of 1 for using either qualitative or quantitative methods or a score of 2 for implementing both). Summing up the score of each component added up to total scale ranging from 3 to 7. A score of 3 was assigned to 68 percent of the dissertations. Only 1 percent of the dissertations incorporated six methodological approaches included in the scale. The degree of transdisciplinarity was similar for the three components of research methods and low for all three. Ninety-seven percent of the dissertations used either laboratory or field observations, 88 percent used either qualitative or quantitative research methods, and 70 percent used either an experimental, quasiexperimental, or nonexperimental method. Only 30 percent of the dissertations used two types of methods.
Contextual scope of the analysis. The survey covered three facets of contextual scope: the spatial, temporal, and sociocultural scope of the analysis. The scale scores for these three facets of contextual scope ranged from 3 to 9 (where 3 indicated a narrow scope of all facets, and 9 indicated a broad scope of all facets). A broad spatial scope was evident in 25 percent of the dissertations, whereas moderate spatial scope was found in 13 percent of the dissertations. The temporal context of the research topic was taken into account to a lesser degree. Broad temporal scope was evident in 17 percent of the dissertations, and moderate temporal scope was shown in 9 percent of the theses. At the same time, 74 percent of the dissertations did not address the temporal context of the research topic at all. The sociocultural context of the research topic was addressed in many more of the dissertations than were the spatial or temporal contexts. Specifically, 48 percent of the dissertations revealed a broad sociocultural scope, whereas another 23 percent incorporated moderate sociocultural scope. Only 29 percent of the dissertations did not address the sociocultural context of the research topic. Another aspect of contextual analysis is the extent to which researchers consider either objective or subjective aspects of the phenomena they study, or the extent to which they alternatively examine both objective and subjective facets of the topic in a combined fashion. Interestingly, 44 percent of the dissertations took into account both objective and subjective facets of the research topic, whereas 56 percent considered only one facet.

Number of analytic levels bridged in the dissertation study.
The dissertations also were rated to assess the number of analytic levels bridged by the conceptual and empirical analysis presented by the author. As shown in Table 1, levels of analysis ranged from molecular/cellular to the macro/global levels. Only 20 percent of the dissertations included more than one analytic level. Of them, 7 percent crossed one level, and another 7 percent crossed two or three levels. Five percent out of the twenty crossed five levels, and one dissertation crossed all seven levels of analysis.
Efforts to translate research concepts and findings into recommendations for community problem solving. Fifty-five percent of the dissertations completed in the School of Social Ecology did not mention issues of community problem solving at all, whereas another 24 percent mentioned the possible implications of the research concepts and findings for community intervention. At the same time, 15 percent of the dissertations explicitly evaluated an existing policy or community intervention.

Personal and Situational Factors that Influence the Transdisciplinary Qualities of Dissertations
In addition to assessing the objective attributes of doctoral theses, we examined certain personal and situational factors associated with the level of transdisciplinarity reflected in the dissertation study. Among the most significant correlates of transdisciplinarity were the doctoral student's gender and the departmental affiliation of the student's dissertation advisor.
Gender. Gender was found to differentiate between levels of sociocultural scope reflected in the dissertation research (t = 1.986, df = 118, p < .05), with females demonstrating broader sociocultural scope in their analyses than males (mean for females = 2.36 as compared to 2.05 for males). Although not statistically significant, there was a tendency for female researchers to be more inclusive of both subjective and objective facets of the research topic. Specifically, 51 percent of the women incorporated both objective and subjective aspects of the phenomenon under investigation, whereas only 36 percent of the men incorporated both facets of the research topic in their dissertation analyses. Interestingly, these findings are consistent with the results from another study (Mitrany and Stokols, 2003), which found that female Ph.D. students were more receptive to the themes of transdisciplinary research, especially the concept of contextualism, than their male counterparts.
Departmental affiliation of the dissertation advisor. The most significant factor influencing the transdisciplinary qualities of the research is the departmental affiliation of the dissertation advisor. Comparisons of Ph.D. dissertations supervised by professors from different departments revealed some striking differences in the transdisciplinary qualities of dissertations. Table 2 summarizes the scores for several dimensions of transdisciplinarity observed in the dissertations completed by students from different departments. Significant differences were found for the number of fields represented among committee members, the contextual scope of the dissertation research, the number of analytic levels bridged in the dissertation study, and degree of emphasis on translating research findings into community problem solving strategies. Higher levels of transdisciplinarity were achieved by those students whose advisors are affiliated with smaller and more diverse, multidisciplinary departments in the school. 2 When the department of origin is smaller yet includes a greater diversity of disciplines among its faculty members, the dissertations were assigned higher scores across several different dimensions of transdisciplinarity. Perhaps in smaller and more diverse departments (as compared to larger and more homogenous ones), faculty and students must interact and cooperate more closely with departmental colleagues and those from other academic units both within and outside the school. It is possible that these conditions within smaller, more diverse departments foster greater open-mindedness and a stronger transdisciplinary orientation among faculty and students.

᭤ Discussion
The present exploratory study demonstrates the feasibility of reliably categorizing doctoral dissertations according to their transdisciplinary qualities. The development and application of a composite scale for assessing the transdisciplinarity of written research products is important for several reasons. First, this research strategy yields a complementary set of measures that can be used in conjunction with process evaluations to assess the effectiveness and outcomes of various scientific collaborations and research training programs. Previous studies of transdisciplinary science and training have placed relatively greater emphasis on process measures rather than product assessments, so the results of this study can contribute toward a more balanced evaluative approach in future studies of transdisciplinary science (Kahn, 1993;Nash et al., 2003). Second, the availability of a research tool for gauging the transdisciplinary qualities of research products can be combined with other kinds of measurements in future studies (e.g., scientific citation rates, peer-evaluated reviews concerning the magnitude of researchers' scientific and public policy contributions) to calibrate the "value added" or "return on investment" associated with transdisciplinary scientific collaborations and training programs (Stokols, Fuqua, et al. 2003).
The specific findings of this study also shed new light on the dynamics of transdisciplinary research training. First, we found that dissertations written under the supervision of advisors from smaller and more diverse (multidisciplinary) departments were rated higher on several transdisciplinary qualities. It is possible that faculty members from smaller multidisciplinary departments are more open-minded and amenable to collaboration with colleagues from other academic units. Collaboration on the basis of shared interests rather than shared institutional affiliations is perhaps more readily achieved in smaller and more diverse departments. Along these lines, Campbell's (1969a) analysis of departmental "ethnocentrism" and "tribalism" suggests that faculty and students based in smaller and less traditional departments may be more inclined to pursue interdisciplinary "fish-scale" topics at the overlapping boundaries of two or more fields. Such departments are more open to new ideas and to the need to cooperate with members from other academic units. Thus, faculty members affiliated with these departments may be more willing to encourage their students to explore these "fish-scale" topics lying at the boundaries of two or more fields and, thereby, to cultivate a transdisciplinary orientation in their research. Overall, the results of our dissertation survey suggest that the student's advisor and departmental affiliation exert a crucial influence on the transdisciplinary qualities of trainees' research. Specifically, the advisor's departmental affiliation directly influenced five of the six components included in our composite measure of transdisciplinarity (i.e., all components except the diversity of research methods used in the dissertation study).
Clearly, transdisciplinary research is shaped in important ways by the personal interests, methodological preferences, and intellectual orientation of students, their advisors, and other members of the dissertation committee. A potentially important strategy for promoting the transdisciplinary development of doctoral students is to foster change in the attitudes of faculty members teaching in graduate programs. Since the advisors who supervise the work of young researchers have such enormous influence on their dissertation projects, efforts should be directed not only toward encouraging a transdisciplinary orientation among graduate students with unidisciplinary backgrounds but also toward encouraging faculty members to be more receptive to new ideas and collaboration with colleagues from other fields. Moreover, the gender differences observed in this study revealed that the dissertations completed by female Ph.D. candidates were rated higher for their transdisciplinary qualities than those completed by male authors. These findings warrant further investigation to determine whether or not they generalize to other settings and populations and, if so, to develop a plausible theoretical account of the observed gender differences. Finally, it should be noted that the doctoral training program assessed in this study (the Ph.D. program in the School of Social Ecology at the University of California, Irvine), has been reasonably successful in promoting an interdisciplinary research orientation among its Ph.D. graduates. Several of the dissertations assessed revealed strong interdisciplinary qualities, yet fewer produced novel conceptual frameworks that integrate theories and methods drawn from two or more fields-the sine qua non of transdisciplinary science, according to Rosenfield (1992). Apparently, this robust form of transdisciplinarity (development of integrative conceptual models) is less commonly achieved in dissertation projects than interdisciplinary integration. Nonetheless, several of the dissertations evaluated by the independent reviewers demonstrated conceptual integration across two or more fields and broad contextual scope-two essential ingredients of a transdisciplinary research orientation.
In sum, the findings from this study provide a basis for developing future investigations of the transdisciplinary qualities and outcomes of research training programs and, especially, the tangible research products contributed by their participants.