Development and validation of the Just Community Gardening Survey: A measure of the social and dietary outcomes of community garden participation

Author(s): Burt, Kate G.; Delgado, Kathleen | Abstract: Our aim was to develop and validate a survey measuring outcomes of community gardening related to food system equity. Face validity was assessed by six community gardening experts and resulted in minor changes to the survey. Weighted kappa and Cronbach’s alpha were used to determine test-retest reliability and internal consistency using a sample of community gardeners in New York City (n=38). Weighted kappa analyses revealed almost perfect agreement (mean = 0.981, range 0.498-1, p≤.001). Cronbach’s alpha was calculated for eight scales, most of which had excellent agreement. The resulting 25-item Just Community Gardening Survey consisted of demographic, garden participation, garden activities, social participation, social cohesion, collective efficacy, perceived impact on dietary intake, food access and security, horticultural and environmental knowledge and diet-related health questions. This validated survey not only measures the social and dietary outcomes of community gardening, it incorporates important aspects of food justice not assessed in previous research.


Introduction
The food justice movement (food movement) seeks to promote equity in the food system by establishing the production, processing, distribution, and access to healthy food as a right to communities (Purifoy, 2014). Issues within the food system addressed by localized food movements include eliminating food insecurity, reducing disparities in diet-related disease, creating equal access to healthy food, and expanding healthy alternatives to processed food (Freudenberg, McDonough, & Tsui, 2011). One proposed strategy to increase community control and access to healthy, culturally appropriate food is community gardening. Community gardens promote important aspects of food justice, providing a physical space to connect with community members and improve community cohesion, organizational capacity, and social networks (Armstrong, 2000;Teig et al., 2009). Communities of color and those with low-incomes, in particular, realize the social benefits of community gardens have used coalitions built in the garden to address neighborhood issues (Armstrong, 2000). Community garden participation (CGP) has also been used for civic engagement as a platform to develop partnerships, conduct education, and as a site to strengthen community members' relationships (Gonzalez, 2015). CGP may also produce dietary benefits to community members. While the purpose of CGP is not necessarily food production, many community gardeners grow food and report increased access to fruits and vegetables and increased food security (Draper & Freedman, 2010).
Evidence establishing community gardens as a successful way to further the food justice movement is weak, as outcome studies are correlational or have other methodological limitations. Many studies exploring the benefits of CGP have low sample sizes (and thus inclusive findings) or they are qualitative in nature and may not be generalizable. Further, no comprehensive measure of the social and dietary outcomes of community gardening exists. More robust research should be done to explore if community gardeners of color or those in low-income neighborhoods are experiencing the food justice-related outcomes in order to determine if community gardening is a viable way to increase food system equity. Due to the lack of a clear communication channel, it may be difficult to reach community gardeners and/or food justice proponents to conduct a large-scale evaluation or distribute a survey; therefore, a valid instrument that is widely available in order to synthesize data across studies is needed. The purpose of this study is to develop and validate a survey measuring the social and dietary outcomes of CGP.

Conceptual Constructs
Social outcomes of CGP increase community control are relevant to the food justice movement and include social capital (individuals' contribution of time and effort toward cooperation with others), cohesion (in part, the accumulation of social capital) and collective efficacy (social cohesion and shared expectations for social control) (Ansari, 2013;Chan, DuBois, & Tidball, 2015;Oxoby, 2009;Whatley, Fortune, & Williams, 2015). We also aimed to measure social relationships developed in gardens, as they may be a precursor to building social capital (Cohen, Underwood, & Gottlieb, 2000). Other studies have found that aspects of social relationships, like neighborhood participation and community sharing are related to community gardening as well (Armstrong, 2000;Hartwig & Mason, 2016;Lanier, Schumacher, & Calvert, 2015). Dietary outcomes of CGP related to the food justice movement include dietary intake, access to high quality produce (most desirable is sustainably grown produce), and food security (Armstrong, 2000;Barnidge et al., 2013;Carney et al., 2012;Hartwig & Mason, 2016;Lanier et al., 2015;Longo, 2016). While improved diet is a clear goal of the food justice movement, it is unlikely that community gardens alone will significantly contribute to improved diet, as produce yield from community gardening may be low. Moreover, garden productivity depends on many climate-related factors making it unpredictable; therefore, we aimed to assess perceived impact on diet more generally.
It was a priority to balance the comprehensive measurement of constructs (social relationships, cohesion, collective efficacy, food access, sustainable eating, food security, and dietary impact) with reducing participant burden. Therefore, we considered the inclusion of previously developed scales to measure individual constructs if they were relatively short and validated in any audience. The measurement of individual constructs is summarized in table 1 and described in more detail below.

Demographic information
Personal identification may not be necessary when using a survey in practice, however, for the purposes of this survey, follow up was required to measure test-retest reliability. Therefore, personal information including name, email, and phone number was collected. As with any survey, demographic questions are important to capture who the audience is. This survey limits the number of demographic questions to age, race/ethnicity, and socioeconomic status. A question was also included about the name and/or location of the garden in which they participate.

Garden participation
Questions regarding length (in years) of garden participation, frequency of attendance during the gardening season, and duration of time spent per garden visit measured individual garden participation. These questions were originally developed for the Community Food Coalition Toolkit (length) and Community Garden Social Impact Assessment Toolkit (frequency and duration). The former was developed by the United States Department of Agriculture Community Food Projects Program's Community Food Security Coalition to help communities conduct evaluation of local food programs (including gardens) (National Research Center, Inc., 2006) and the latter by the University of Minnesota's Center for Regional and Urban Affairs to measure the social impact of community gardens (Miller, 2012). Neither of these measures is validated but both are widely used. One additional question was added to capture what plants or animals were grown or raised in participants' gardens.

Social relationships
Social network scales (Lubben and Stroke Social Network Scales) were considered to measure social relationships; however neither was appropriate to measure the breadth and types of relationships facilitated by the garden. Therefore, questions from the Community Garden Social Impact Assessment Toolkit (CFC Toolkit) were used to measure the number of people gardeners interact with, how comfortable gardeners feel addressing garden-related issues with other gardeners, and how, if at all, gardeners interact with each other for events or activities other than gardening (Miller, 2012). Two additional questions were added to measure what types of events gardeners host or attend in the garden (based on expert suggestion, described more in the Face Validity section below) and interactions among gardeners of different races (based on literature suggesting that interracial interaction is particularly beneficial for people of color, which is related to food system equity) (Jettner, 2017).

Cohesion
Several validated measures of neighborhood cohesion exist; some are based on modified versions of the Neighborhood Cohesion Instrument, which measures attraction to neighborhood and psychological sense of community (Buckner, 1988;Fone, Farewell, & Dunstan, 2006). As the difference between neighborhood cohesion and social cohesion are unclear, for testing purposes, a measure of social cohesion, a sub-scale of the Perceptions of Collective Efficacy Scale was also tested (more on this scale in the next section).

Collective efficacy
The Perceptions of Collective Efficacy Scale includes three sub-scales to measure willingness to intervene, social cohesion, and capacity for social control (Sampson, 1997). Due to methodological limitations, it was revised (and expanded) and the newer version included additional items relevant to our study, so two of the three sub-scales (social cohesion and capacity for social control) were tested in the current survey (Uchida, Swatt, Solomon, & Varano, 2014).

Perceived impact on self
We included items developed for the CFC Toolkit to measure the perceived impact of community gardening on food intake, physical activity, gardening skills, environmental knowledge, neighborhood involvement, and community sharing -all of which have been found by previous studies as outcomes of community gardening (Armstrong, 2000;Barnidge et al., 2013;Carney et al., 2012;Chan et al., 2015;Hartwig & Mason, 2016;Lanier et al., 2015;Teig et al., 2009). As these questions were pilot tested for the CFC Toolkit, they were used as published for the current survey.

Sustainable eating
Sustainable eating was measured using a six-item validated but unpublished scale, the Low Environmental Impact Food (LEIF) intake scale (Arbit, Ruby, Burt, & Rozin, 2017).

Food access and security
The United States Department of Agriculture's Household Food Security Survey Module: Six-Item Short Form best suited this survey, as this shortened version effectively captures household food insecurity ("U.S. Household Food Security Survey Module: Six-Item Short Form," 2012). However, the items were modified from their original version into Likert-style questions, which aligns with the presentation of most other items on the current survey. Item rephrasing was based on language used in the CFC Toolkit. One question from the New York City Community Health Survey, a cross sectional telephone survey conducted in five languages (English, Spanish, Russian, Mandarin, and Cantonese) annually with 10,000 adults residing in New York City was included to capture the distance that individuals have to travel in order to obtain fresh produce ("Community Health Survey," n.d.). The question was modified to be applicable to and inclusive of individuals living outside urban areas.

Personal health
Several questions were included to measure aspects of health relevant to gardening, including questions about chronic disease, physical activity, produce intake, height, weight, and perceived level of activity. All questions were used verbatim from the New York City Community Health Survey.

Questionnaire Development
Given the length of the survey developed, consideration was given to the order of items so as to reduce order effects. In particular, items were ordered for this survey by grouping topics (or related questions) and general items were placed before specific items (McFarland, 1981). For instance, to transition from social impact to personal impact (on health, diet, and attitudes), general items about food and eating habits were presented before specific items about sustainable eating and food security. Items were also ordered based on impact proximity; that is, social relationships may be a more proximal, immediate effect of CGP, whereas impact on food security or access is a more distal effect. Thus, questions were ordered from most proximal to distal in each topic area. Finally, given the importance of demographics in the food justice movement, demographic questions were presented at the beginning of the survey.

Validation Study and Statistical Analyses Face validity
An examination of each survey item was conducted and discussed with community gardening experts as a way to assess the completeness of the survey items to adequately capture outcomes related to CGP (Sireci, n.d.). Four community gardening experts from the New York City (NYC) Parks Department's organization GreenThumb, the largest community gardening program in the United States, were interviewed to determine the face validity of the items ("NYC Parks GreenThumb," n.d.). More than just managers of NYC community gardens, GreenThumb hosts conferences and events on topics such as food justice, community engagement, and health and well-being. First, each expert reviewed survey items independently and provided written, qualitative feedback about the relevance, clarity of items, and made suggestions for any topics that were omitted. Next, each was interviewed to understand their specific suggestions and impression of the survey overall. Specific questions were asked about the completeness of the items to capture all relevant outcomes and appropriateness for the community gardening population.
Participants were recruited from an NYC gardening listserv to pilot the revised survey; the primary researcher (first author) posted instructions and a link to the survey once. One participant e-mailed the researchers to request to disseminate the survey among the other members of the community garden in which he participated; thus, it is unknown if all survey participants were from the listserv or if the survey was disseminated through other lines of communication.
The survey remained open for 37 days during May and June 2018 until the desired sample size was obtained. Consent was obtained from all participants and those who completed the survey twice (for reliability, described more in the next section) received $40 for their participation.

Test-retest reliability
Each participant who completed the survey (time 1) was invited to complete it a second time (via e-mail) six days after their first completion. The target time between time 1 and time 2 was between 7 and 14 days. This time period was preferable so as to minimize variables that may affect participants' responses. For instance, since the weather was becoming more conducive to gardening and work to maintain a garden increases as the weather improves, community gardeners may spend more time in the garden later in the season. In order to minimize changes in responses due to the anticipated increased frequency of participation, a narrow time frame between surveys was desirable. In addition, evidence suggests that the most frequently recommended target time is 2 weeks (Streiner, Norman, & Cairney, 2014). Participants who completed the survey a second time were compensated $40 and the number of eligible participants was capped at 30. This target was selected as 30 is suggested minimum number of participants and capped due to funding limitations (McHugh, 2012).

Internal consistency
The reliability of sub-scales for social relationships, cohesion, collective efficacy, food security, and personal health were assessed using Cronbach's alpha. The surveys of all participants who completed (at minimum) a single survey in the 37 day time period that the survey was open were included. For those participants who completed the survey twice, data from the first completion was used.

Results
A total of 6 experts and 38 community gardeners participated in the validation study. Characteristics of the community gardeners are presented in table 2.

Face validity
Minor revisions were made to the survey based on feedback from expert reviewers. Most suggested revisions were to items measuring social relationships developed in the garden. Table 3 presents a summary of the changes that were made. Table 3. Revisions to survey items based on community gardening expert review

Length of garden work
Original item and answer Options How long have you been working in this garden? • Less than 1 year • 1-2 years • 3-4 years • 5-10 years More than 10 years

Justification for change
Difference between people who have been gardening for a full season (1 year) and those who have just joined.

Revised item and answer options
How long have you been working in this garden?

Frequency of garden work
Original item and answer Options

Revised item and answer options
Do you grow or raise any of the following in your garden?
Test-rest reliability 30 participants completed the survey twice (83.3% of the total number of participants). The average time between completions was 8.8 days and only 1 participant completed the survey outside of the desired 7-14 day window (at day 15). Weighted Cohen's kappa analyses revealed an average of .981 agreement between participants' time 1 and time 2 surveys, indicating almost perfect agreement (range .498-1, p≤.001 for all analyses).

Internal consistency
Internal consistency was measured for eight scales. Cronbach's alpha for social relationships was .574, indicating poor reliability. Of the two cohesion measures (Neighborhood Cohesion Instrument and Perceptions of Collective Efficacy Scale's social cohesion sub-scale), the Neighborhood Cohesion Instrument had greater internal consistency though Chronbach's alpha for both indicated excellent agreement. Cronbach's alpha for the 18-item Neighborhood Cohesion Instrument was .939 and increased to .951 after removing items with low reliability (n=6) while Cronbach's alpha for the social cohesion subscale was .918 and increased to .927 after removing 1 item with low reliability. Cronbach's alpha for the Perceptions of Collective Efficacy Scale's capacity for social control sub-scale was .911, indicating excellent agreement and none of the items were removed. The Perceived Impact on Self items were not developed as a scale, nor were they intended to measure the same construct. Therefore, Chronbach's alpha was only calculated for related questions: those on diet and on environmental knowledge and skills. For the nine items measuring the impact of community garden participation on diet, Cronbach's alpha was .951 and increased to .952 after 1 item was removed. For the four environmental knowledge and skills items, Chronbach's alpha was .891, indicating good agreement. Chronbach's alpha for the LEIF scale, a measure of sustainable eating, was .687, indicating poor reliability. Finally, food security was measured using the United States Department of Agriculture's Household Food Security Survey Module: Six-Item Short Form. Chronbach's alpha was .942; one item was removed but the value remained at .942.

The Survey Instrument
The final survey, the Just Community Garden Survey consisted of 25 items (counting sub-scales and matrix style questions as single items) (Appendix A). General items were included to gauge participant characteristics and garden engagement, including demographic items (gender, age, race/ethnicity, income), items about garden participation (e.g. location of garden, duration and frequency of participation), and garden activities (e.g. types of foods grown or raised, interactions with others). Items aimed to gather information about social outcomes assessed (a) social participation, (b) social cohesion, and (c) collective efficacy. Items aimed to gather information about dietary outcomes assessed (a) perceived impact on dietary intake and (b) food access and security. Items were also included to examine horticultural and environmental knowledge and skills and diet-related health outcomes.

Discussion and Conclusions
This study is the original to develop a valid and reliable measure explicitly connecting outcomes of community gardening to food justice. To date, research about the outcomes of community gardening has not been directly linked to current discourse about the potential for community gardening to be one strategy to establish a more just food system. Yet, there is great need to determine if CGP actually promotes core tenets of the food justice movement (like increasing access to healthy, culturally appropriate food). While some correlational research has been done about the social and dietary outcomes of CGP, studies often use a small sample of participants or are qualitative and do not utilize valid or reliable measures (Draper & Freedman, 2010;Okvat & Zautra, 2011). The survey developed and validated in this study can become an important tool to determine if CGP achieves desired food-justice related outcomes.
Outcome evaluations of community gardening studies often use measures that are specific to an intervention and not generalizable or applicable to use in other research (Draper & Freedman, 2010). Prior to this study, none of the existing scales relevant to measuring social and dietary impacts of community gardening have been validated in the population of interest. Previously developed scales included in the final version of our survey include the Neighborhood Cohesion Instrument and the Perceptions of Collective Efficacy Scale's capacity for social control sub-scale (Buckner, 1988;Sampson, 1997). Social cohesion and collective efficacy help build the capacity of marginalized groups to seek equal opportunities in communities, which is critical to food justice (Agyeman & McEntee, 2014;Fonseca, Lukosch, & Brazier, 2018). Since CGP has been shown to promote these constructs, it was important to validate these scales in the population of interest. Lastly, among community gardeners, only five of the six items from the United States Department of Agriculture's Household Food Security Survey Module: Six-Item Short Form were included in the final version of the survey to measure food security.
In addition to food security, access to purchase or grow healthy food is a critical tenet of the food justice movement (Glennie & Alkon, 2018). Therefore, two additional sets of items measuring perceived dietary impact and horticultural knowledge were included in this survey. The items included in each scale were found to have excellent and good agreement, respectively. Finally, items were included to assess dietrelated health outcomes. Disparate access to healthy food between racial/ethnic groups is strongly linked to poor diet quality and high rates of diet-related chronic disease (Neff, Palmer, Mckenzie, & Lawrence, 2009). Though a proximal goal of food justice is improved food access, a distal goal is to reduce dietrelated health disparities (Glennie & Alkon, 2018), giving merit to the inclusion of health-related items.
The inclusion of items about social relationships with attention to race/ethnicity (and other socioeconomic factors) is crucial, as these factors are directly related to food system inequities, like eliminating poverty and hunger (Glennie & Alkon, 2018;Longo, 2016). However, studies examining the impact of community gardening on different races/ethnicities and interactions between them has yielded mixed results; it is unclear as to whether community garden participation divides or unifies people of different races/ethnicities (Glover, 2004;Shinew, Glover, & Parry, 2004). As white individuals have more power and control over the food system (Reynolds & Cohen, 2016), when examining community gardening through a food justice lens, it is imperative to consider the role of racism and white privilege. Existing social relationship and network scales failed to consider race/ethnicity as a critical factor and thus were insufficient for this survey. Though items about social relationships are included, future research should aim to develop a valid and reliable measure specific to food justice-related issues.
Finally, here are other goals of food justice not addressed by this survey, including job creation, improved wages and working conditions related to agriculture, and environmental sustainability (Glennie & Alkon, 2018;Longo, 2016;Reynolds & Cohen, 2016). As these topics are not necessarily outcomes of CGP, they were not included in this survey.

Strengths and Limitations
This study initialed to develop and validate a scholarship aimed to examine the perceived social and dietary benefits of community gardening. Building on existing scales and previous survey items developed by field experts enhances the quality of the survey. The primary limitation is the reliance of self-reported data. Data collected by this survey will provide insight into participants' perceptions of impact rather than measuring actual impact. Moreover, items measuring perceived impact are subject to response bias. However, the aim of this research was to develop a measure based on perceptions rather than measuring outcomes directly and evidence suggests that positive experiences or perceptions (particularly related to social outcomes) may lead to behavioral change (Devine, Brunson, Jastran, & Bisogni, 2006). If future studies aim to measure actual impact, this survey should be used in tandem with objective measures of social cohesion, collective efficacy, dietary intake, and environmental skill.

Conclusions
Designing a validated measure of the impacts of community garden participation is the first step to conduct more rigorous, causal research. The survey developed in this study, the Just Community Gardening Survey, not only measures the perceived social and dietary outcomes of community gardening, it incorporates important aspects of food justice that have not been assessed in previous research. In order to objectively determine if community gardening is having the intended impact on people of color, casual research should be conducted with population(s) of interest. Though some research indicates that CGP has a positive impact on diet and social outcomes, most research to date has utilized a higher-income, White sample, who may or may not experience the same benefits of community gardening as people of color. The impact of community gardening on people of color has been understudied, despite the plethora of literature indicating that people of color have inequitable access to healthy, affordable food. Future studies should explicitly focus on the impact of community gardening participation on persons of marginalized demographic groups.
This study was approved by the Lehman College Institutional Review Board, protocol #2018-0006.