Appendices

U.S. Department of Agriculture and U.S. Department of Health and Human Services. Dietary Guidelines for Americans, 2020-2025. 9th Edition. December 2020. Available at DietaryGuidelines.gov.

The 2020–2025 Dietary Guidelines for Americans are designed for policymakers and nutrition and health professionals to help all individuals and their families consume a healthy, nutritionally adequate diet. The information in the Dietary Guidelines is used to develop, implement, and evaluate Federal food, nutrition, and health policies and programs. It also is the basis for Federal nutrition education materials designed for the public and for the nutrition education components of USDA and HHS nutrition programs.

The Guidelines provide four overarching Guidelines that encourage healthy eating patterns at each stage of life and recognize that individuals will need to make shifts in their food and beverage choices to achieve a healthy pattern. The Guidelines also explicitly emphasize  that a healthy dietary pattern is not a rigid prescription. Rather, the Guidelines are a customizable framework of core elements within which individuals make tailored and affordable choices that meet their personal, cultural, and traditional preferences. For the first time, the Dietary Guidelines for Americans 2020-2025 provides recommendations for each life stage, from birth to older adulthood.

What is a dietary pattern? It is the combination of foods and beverages that constitutes an individual’s complete dietary intake over time. This may be a description of a customary way of eating or a description of a combination of foods recommended for consumption.

There are 4 Guidelines:

• Follow a healthy dietary pattern at every life stage.
• Customize and enjoy nutrient-dense food and beverage choices to reflect personal preferences, cultural traditions, and budgetary considerations.
• Focus on meeting food group needs with nutrient-dense foods and beverages, and stay within calorie limits.
• Limit foods and beverages higher in added sugars, saturated fat, and sodium, and limit alcoholic beverages.
  • This fourth guideline includes quantitative recommendations on limits that are based on the body of science reviewed. These recommendations include:
    • Limiting added sugars* to less than 10% of calories per day for ages 2 and older and to avoid added sugars for infants and toddlers.
    • Limiting saturated fat to less than 10% of calories per day starting at age 2;
    • Limiting sodium intake to less than 2,300mg per day (or even less if younger than 14);
    • Limiting alcoholic beverages* (if consumed) to 2 drinks or less a day for men and 1 drink or less a day for women.

There are 3 Key Dietary Principles that can help people achieve the Dietary Guidelines. To help improve Americans’ eating patterns, the Dietary Guidelines suggests:

• Meet nutritional needs primarily from nutrient-dense foods and beverages.
• Choose a variety of options from each food group.
• Pay attention to portion size

* The Dietary Guidelines for Americans, 2020-2025 recommend limiting intakes of added sugars and alcoholic beverages, but do not include changes to quantitative recommendations from the 2015-2020 Dietary Guidelines for these two topics, because the new evidence reviewed since the 2015-2020 edition is not substantial enough to support changes to the quantitative recommendations for either added sugars or alcohol.

Below are the key guidelines pulled directly from the Physical Activity Guidelines for Americans, 2nd edition:

Key Guidelines for Preschool-Aged Children

• Preschool-aged children (ages 3 through 5 years) should be physically active throughout the day to enhance growth and development.
• Adult caregivers of preschool-aged children should encourage active play that includes a variety of activity types.

Key Guidelines for Children and Adolescents

• It is important to provide young people opportunities and encouragement to participate in physical activities that are appropriate for their age, that are enjoyable, and that offer variety.
• Children and adolescents ages 6 through 17 years should do 60 minutes (1 hour) or more of moderate-to-vigorous physical activity daily.
  – Aerobic: Most of the 60 minutes or more per day should be either moderate- or vigorous intensity aerobic physical activity and should include vigorous-intensity physical activity on at least 3 days a week
  – Muscle-strengthening: As part of their 60 minutes or more of daily physical activity, children and adolescents should include muscle-strengthening physical activity on at least 3 days a week.
  – Bone-strengthening: As part of their 60 minutes or more of daily physical activity, children and adolescents should include bone-strengthening physical activity on at least 3 days a week.

Key Guidelines for Adults

• Adults should move more and sit less throughout the day. Some physical activity is better than none. Adults who sit less and do any amount of moderate-to-vigorous physical activity gain some health benefits.
• For substantial health benefits, adults should do at least 150 minutes (2 hours and 30 minutes) to 300 minutes (5 hours) a week of moderate-intensity, or 75 minutes (1 hour and 15 minutes) to 150 minutes (2 hours and 30 minutes) a week of vigorous-intensity aerobic physical activity, or an equivalent combination of moderate- and vigorous-intensity aerobic activity. Preferably, aerobic
  activity should be spread throughout the week.
• Additional health benefits are gained by engaging in physical activity beyond the equivalent of 300 minutes (5 hours) of moderate-intensity physical activity a week.
• Adults should also do muscle-strengthening activities of moderate or greater intensity and that involve all major muscle groups on 2 or more days a week, as these activities provide additional health benefits.

Key Guidelines for Older Adults
The key guidelines for adults also apply to older adults. In addition, the following key guidelines are just for older adults:

• As part of their weekly physical activity, older adults should do multi component physical activity that includes balance training as well as aerobic and muscle-strengthening activities.
• Older adults should determine their level of effort for physical activity relative to their level of fitness.
• Older adults with chronic conditions should understand whether and how their conditions affect their ability to do regular physical activity safely.
• When older adults cannot do 150 minutes of moderate-intensity aerobic activity a week because of chronic conditions, they should be as physically active as their abilities and conditions allow.

Key Guidelines for Women During Pregnancy and the Postpartum Period

• Women should do at least 150 minutes (2 hours and 30 minutes) of moderate-intensity aerobic activity a week during pregnancy and the postpartum period. Preferably, aerobic activity should be spread throughout the week.
• Women who habitually engaged in vigorous-intensity aerobic activity or who were physically active before pregnancy can continue these activities during pregnancy and the postpartum period.
• Women who are pregnant should be under the care of a health care provider who can monitor the progress of the pregnancy. Women who are pregnant can consult their health care provider about whether or how to adjust their physical activity during pregnancy and after the baby is born.

Key Guidelines for Adults With Chronic Health Conditions and Adults With Disabilities

• Adults with chronic conditions or disabilities, who are able, should do at least 150 minutes (2 hours and 30 minutes) to 300 minutes (5 hours) a week of moderate-intensity, or 75 minutes (1 hour and 15 minutes) to 150 minutes (2 hours and 30 minutes) a week of vigorous-intensity aerobic physical activity, or an equivalent combination of moderate- and vigorous-intensity aerobic activity. Preferably, aerobic activity should be spread throughout the week.
• Adults with chronic conditions or disabilities, who are able, should also do muscle-strengthening activities of moderate or greater intensity and that involve all major muscle groups on 2 or more days a week, as these activities provide additional health benefits.
• When adults with chronic conditions or disabilities are not able to meet the above key guidelines, they should engage in regular physical activity according to their abilities and should avoid inactivity.
• Adults with chronic conditions or symptoms should be under the care of a health care provider. People with chronic conditions can consult a health care professional or physical activity specialist about the types and amounts of activity appropriate for their abilities and chronic conditions.

Key Guidelines for Safe Physical Activity
To do physical activity safely and reduce risk of injuries and other adverse events, people should:

• Understand the risks, yet be confident that physical activity can be safe for almost everyone.
• Choose types of physical activity that are appropriate for their current fitness level and health goals, because some activities are safer than others.
• Increase physical activity gradually over time to meet key guidelines or health goals. Inactive people should “start low and go slow” by starting with lower intensity activities and gradually increasing how often and how long activities are done.
• Protect themselves by using appropriate gear and sports equipment, choosing safe environments, following rules and policies, and making sensible choices about when, where, and how to be active.
• Be under the care of a health care provider if they have chronic conditions or symptoms. People with chronic conditions and symptoms can consult a health care professional or physical activity specialist about the types and amounts of activity appropriate for them.

There are four main types of evaluation that may be included in State SNAP-Ed Plans. The following table from Addressing the Challenges of Conducting Effective Supplemental Nutrition Assistance Program Education (SNAP-Ed) Evaluations: A Step-by-Step Guide (Cates et al., 2014) provides definitions and SNAP-Ed relevant example questions:

SNAP-Ed providers can use all four types of evaluation to measure indicators in the SNAP-ED Evaluation Framework. Baseline measurement of relevant individual-level indicators could inform the current state of target behaviors and serve to answer formative assessment questions. Additionally, at the environmental settings level, identifying a need for improved access or creating appeal in SNAP-Ed eligible sites and organizations and identifying and measuring the strength of key partnerships are additional examples of relevant indicators for formative assessment.

The framework contains fewer process indicators as most process data for SNAP-Ed are collected through reporting systems that inform the Education and Administrative Reporting System (EARS). Some process indicators included in the framework are reach and resources put towards policy, systems, and environmental (PSE) change activities and sustainability.

Outcome assessments examine the extent to which an intervention achieves its stated goals. The example in the table above depicts outcomes at an Individual level, and the medium- and long-term individual-level indicators of the framework can be measured pre- and post-intervention to assess whether your intervention achieved its behavioral change goals. The framework also specifies outcomes at the Environmental Settings and Sectors of Influence levels. At these levels, for example, we may assess the food and/or physical activity environments in sites, organizations, communities, or other jurisdictions at baseline and follow-up to determine the extent to which those environments changed as a result of PSE changes that were adopted and implemented.

Impact evaluation measures changes in outcomes attributable to the program and includes the use of control or comparison groups. The step-by-step guide by Cates and colleagues (2014) mentioned above focuses on impact evaluation and discusses important considerations for quality impact evaluation studies.

Considerations in SNAP-Ed Evaluation

Both process and outcome evaluation of SNAP-Ed programming is essential to ensure the fidelity and assess the effectiveness of program delivery, respectively. Some SNAP-Ed programs may have internal evaluators on staff who work hand-in-hand with program staff; others may contract with external evaluators from a separate agency; and some may use both. You can find experts with experience in community-based evaluation programs like SNAP-Ed in your state or a neighboring state who can help you evaluate your SNAP-Ed interventions. Many community outreach evaluation services at colleges and universities, including Land-grant institutions, are already evaluating SNAP-Ed services. CDC Prevention Research Centers, state and local health departments, and public health institutes can also assist with community-based evaluation for SNAP-Ed programs.

SNAP-Ed evaluations should focus on specific current interventions or initiatives in your approved SNAP-Ed Plan. Evaluation of projects or initiatives beyond the scope of SNAP-Ed interventions or the low-income population, or projects that intend to generate new knowledge or theory in the field of obesity prevention, are considered research and, therefore, will not be approved for funding. For example, requests to fund the creation or validation of an evaluation tool that is not specific to the SNAP-Ed intervention would not be approved. States interested in broad research may wish to seek alternate sources of funding. SNAP-Ed will pay for the data collection from a low-income control group (no intervention) or comparison groups (different intervention) when such data are necessary and justified to conduct an impact evaluation of the SNAP-Ed intervention. Whenever a state carries out a SNAP-Ed evaluation activity that costs more than $400,000 in total—whether spent in one year or multiple years—FNS strongly recommends that an impact evaluation be conducted. However, SNAP-Ed funds cannot be used to pay for the portion of data collection or surveillance of populations whose incomes exceed 185 percent of the federal poverty level or the general population.

As with most population-based public health programs, SNAP-Ed programs or interventions generally are not designed to establish cause-and-effect relationships between programming or exposure and outcomes as in a randomized control group design due to ethics (programming cannot be withheld from SNAP-Ed participants to serve as a control group) and the multi-level complexity of SNAP-Ed programming. However, SNAP-Ed programming may be evaluated to assess correlations or associations among variables (e.g., programming and outcomes) at one, or preferably more, points in time and at multiple programming levels throughout the funding period.

SNAP-Ed Plans may range from 1- to 3-year plans. One-year SNAP-Ed plans and funding cycles pose a challenge for states who are limited in the length of their program delivery, and evaluation designs, particularly in the evaluation of PSE interventions and social marketing campaigns. Within the Environmental Settings, Sectors of Influence, and Social Norms and Values levels of the framework, medium- and long-term outcomes and population results cannot be adequately measured within a 1- or even 3-year period, depending upon the indicator(s) and/or measure(s).

As the multi-level, multi-sectoral field of SNAP-Ed programming expands, so does the need for the reliable and validated environmental scanning and evaluation tools, measures, and secondary data sources that are referenced throughout this interpretive guide.

Effective evaluation will help to build the evidence base and identify effective and promising or emerging obesity prevention strategies and interventions. Knowledge on effective obesity prevention strategies and interventions is evolving. Examples of success can be found across the nation in states, cities, towns, tribes, and communities. But there is still much to learn, and programs are challenged to stay up-to-date, to be culturally relevant, and to help to establish evidence-based practices that are needed to meet the evidence-based requirement of the Healthy, Hunger-Free Kids Act.

Sampling

As part of your SNAP-Ed evaluation plan, it is helpful to develop a sampling plan, a description of who will participate in the evaluation. Some evaluations may include all participants in an activity or all sites, while others may assess a sample or subset of participants or sites. Sampling in evaluation is useful for programs that do not have the time, funds, or staff to include all participants or sites in their evaluations. If your program does not have internal evaluation expertise, partnering with evaluators or researchers to plan evaluation methods, including sampling plans, will benefit your SNAP-Ed outcomes.

Individual Level
At the Individual level of the framework, when measuring goals, intentions, behavior change, and sustained behaviors, evaluators may wish to collect pre-, post-, and in some cases, follow-up data on a subset or sample of participants. Ideally, this sample will represent the entire group that participated in the SNAP-Ed intervention. Members of the sample are representative of the total population from which they are sampled, when using methods, such as:

• Simple random sampling: All individuals in the population have an equal opportunity of being selected (e.g., a lottery system or draw numbers/names from a hat).
• Systematic sampling: Individuals selected according to a random starting point and a fixed, periodic interval (e.g., from random start point on a list every 8th individual is selected).
• Stratified sampling: The population is divided into groups (e.g., by grade level), then individuals from each group are randomly selected.
• Cluster sampling: Clusters or groups are randomly selected (e.g., classrooms in a school); all individuals within the selected clusters are included in the study (e.g., all students within the classrooms that were randomly selected).

Often in SNAP-Ed evaluation, random sampling is not feasible or practical. For example, if you wanted to administer a survey to people who attended a one-time community nutrition event and you did not have a roster of all attendees, you would not be able to select a random sample of attendees. The conclusions from the sample may not generalize to all participants. Specific sampling methods for non-random samples include

• Convenience sampling: Individuals who are readily available are selected from the population you are studying.
• Purposive sampling: The evaluator selects a sample he or she believes represents the population.
• Snowball sampling: A small group of participants who have the desired characteristics is selected; those participants in turn recommend others with similar characteristics to participate.
• Quota sampling: A sample is selected based on pre-specified characteristics in the same proportion as the population (e.g., if 60% of the population is female then 60% of sample will be female; recruitment stops when 60% is reached).
• Self-selection sampling: Participants volunteer to participate.

Environmental Level
At the environmental or organizational settings level, programs may choose to measure PSE changes in a sample of sites in which they are working to affect change. Based on the stage at which programs are engaged in PSE changes, the sampling will vary. The Needs and Readiness Flow Chart that follows outlines a process to establish readiness to implement PSE interventions.

Needs and Readiness Assessment Flow Chart

Figure Content In Text-only Format:

Needs and Readiness Assessment Flow Chart

Step 1. Has a needs assessment been completed?

• If yes, proceed to step 2.
• If no, conduct a brief need assessment to identify high need areas before moving ahead with a readiness assessment: http://ctb.ku.edu/en/table-of-contents/assessment/assessing-community-needs-and-resources. Then, proceed to step 2.

Step 2. Is there a coalition representing the SNAP-Ed PSE approach?

• If yes, proceed to step 3.
• If no, consider coalition building work before engaging in a formal readiness assessment: http://ctb.ku.edu/en/table-of-contents/assessment/promotion-strategies/start-a-coaltion/main. Then, proceed to step 3.

Step 3. Are the opinions/perspectives of core coalition leaders already clear/well established?

• If yes, proceed to step 4.
• If no, consider conducting key informant interviews or a facilitated discussion with leaders using readiness tool: https://pmc.ncbi.nlm.nih.gov/articles/PMC3904699/. Then, proceed to step 4.

Step 4. Are the opinions/perspectives related to readiness already known from a broader group (e.g. the community or other organizations)?

• If yes, proceed to step 6.
• If no, proceed to step 5.

Step 5. Are members willing to help identify stakeholders and gather information?

• If yes, consider engaging the coalition in gathering information: http://ctb.ku.edu/en/table-of-contents/structure/strategic-planning. Then proceed to step 6.
• If no, consider gathering information through a larger facilitated discussion: http://ctb.ku.edu/en/table-of-contents/leadership/group-facilitation/facilitation-skills/main. Then, proceed to step 6.

Step 6. Is there broad community support related to the need? Are they concerned and willing to support?

• If yes, proceed to step 8.
• If no, proceed to step 7.

Step 7. Are there resources available for gathering information from community members?

• If yes, consider conducting a community readiness survey: http://ctb.ku.edu/en/table-of-contents/overview/models-for-community-health-and-development/community-readiness/main. Then, proceed to step 8.
• If no, there are two choices. Consider grant opportunities to identify or obtain resources for a survey. Alternatively, consider other available sources of public information, such as media analysis. Then proceed to step 8.

Step 8. There is a wealth or readiness information available – use it for analysis, information sharing, and strategic planning.

As indicated on the flow chart, the 5 stages in the readiness process: 1) needs assessment, 2) coalition building, 3) assessing coalition readiness, 4) gaining support from other groups and community members, and 5) assessing site, organization, or community readiness build on one another and require information from different sources, thus the sampling will vary. Below each stage’s sampling recommendation is provided.

1. Needs assessment: For a defined issue, the sampling will be 10 issue-specific sites in the community, or within the relevant domains of environmental settings (i.e., eat, learn, live, play, shop, and work. If the community or domain has fewer than 10 sites then all the issue-specific sites will be assessed.
   • For example, a group wants to address access to parks. Google Maps or other maps could be used to identify public parks in the community. Community A has 20 parks and community B has 5 parks. For community A, 10 parks will be chosen using one of the sampling methods listed in the individual level. For community B, all 5 parks will be assessed.
2. Coalition building: A minimum of 3 partnerships, community groups and/or members actively engaged in the issue. Using the access to parks issue above, the three groups and/or members could be: (1) a representative from the department of parks and recreation; (2) a member of a neighborhood board; and (3) a mother who organizes a playgroup at the park.
3. Assessing coalition readiness to collaborate: One representative from each group in the coalition. For example if a park coalition consists of 5 groups, one member of each of the 5 groups would complete the assessment.
4. Gaining support from other groups and community members: Five key informants active in the issue. For example, the park coalition could get input from the following: local business leader, after-school program director, president of a local running club, principal of a nearby school, and a representative from the local police department.
5. Assessing community readiness: At least 5 key informants who are familiar with the community and the issue.

The sites selected should represent the SNAP-Ed sites where one or more PSE changes are being made (MT5).

When sampling sites that are already actively addressing the issue, you may want to include the sites or organizations that represent the most advanced sites in terms of their progression through the stages of change at the organizational level (e.g., Weiner, 2009). In this case, the sample of sites would not necessarily be representative of all sites in which you are working, but it would represent those sites that have made significant progress in adopting and implementing PSE changes.

Sectors of Influence

As stated in the introduction to the Sectors of Influence chapter, most indicators will be measured using existing or secondary data sources, and you will likely include all relevant jurisdictions in your evaluation. For example, indicator MT8a1 is the total number of farmers markets that accept SNAP benefits per 10,000 SNAP recipients. To calculate the rate, if a state has 178 farmers markets that accept SNAP, and an average monthly SNAP caseload of 950,000 participants, the adjustment would be equal to 178/(950,000/10,000), yielding 1.87 farmers markets for every 10,000 SNAP participants. This is calculated based on state data, not a sample of state data.

Population Results

Most of the data for the indicators at the population level will come from existing data sources at the state level, such as the Behavioral Risk Factor Surveillance System and the Youth Risk Behavior Surveillance System. If states choose to collect data to represent population-level changes, sampling methods described for individual-level sampling would apply. For example, in California, the Champions for Healthy Change study is a 4-year longitudinal survey of mothers, teens, and children from randomly selected SNAP households in 17 local health departments. Telephone interviews include collecting comprehensive information on dietary behaviors during the previous day using the Automated Self-Administered Recall System (ASA24), types and levels of physical activities, and self-reported height and weight to calculated BMI.

Determining the Size of Your Sample

Consider partnering with an evaluator or researcher to assist in your evaluation design and selection of an appropriate sample. An experienced evaluator or researcher will be able to assist with a power analysis, which determines how likely it is to achieve a significant program effect given a specific sample size. Power represents the chance that you will find an effect of your intervention if it’s there. Typically, researchers aim for a power of .80, meaning you have an 80 percent chance of finding a statistically significant difference between two groups when there is one. A similar analysis can be conducted to determine the necessary sample size for different types of studies—one-time surveys, comparisons of pre/post means, comparisons of two proportions, and more sophisticated statistical tests.

Several factors influence sample size, depending on the type of analysis. For example, if you want to determine how many people you should include in a survey in order to report the percentage of the population who intends to eat more fruit, you will need to know the population size, the margin of error, and the desired confidence level to calculate the sample size (see, for example, https://www.surveymonkey.com/mp/sample-size-calculator/). If you want to compare averages, such as the average cups of fruit per day, before and after an intervention, you will need to know the Type I error rate (typically .05), the Type II error rate (typically .20), the effect size, and the standard deviation for the change (see, for example, https://www.sample-size.net/sample-size-study-paired-t-test/). There are other online calculators available for many study designs. Since the body of evidence is building for PSE interventions, associated or anticipated effect sizes may not always be known to accurately calculate a sample size needed to detect significant changes in outcomes.

The following figure depicts the factors that influence sample size:

 Source: http: //toolkit.pellinstitute.org/evaluation-guide/collect-data/determine-appropriate-sample-size/

Figure Content In Text-only Format:

Determining Appropriate Sample Size

Some questions are listed to determine appropriate sample size.

How precise do you want to be? Your sample size can range from a smaller sample size, which is fairly precise, to a larger sample size, which is very precise.

How sure do you want to be our your answer? With a smaller sample size, you will be fairly sure in your answer.  As a sample size grows larger, you will very sure in your answer.

How much variation is there in the population you are studying? In a smaller sample size, there is low variation in the population.  As a sample size grows larger, there is higher variation in the population.

How small of an effect do you want to be able to identify? With a smaller sample size, you can identify a moderately sized effect.  With a larger sample size, you can identify very small effects.

There are other considerations such as time and resources that may influence your sampling decisions. For example, in Maryland, the Text2BHealthy program delivers nutrition and physical activity tips via text messages to parents of elementary children who are receiving classroom-based nutrition education. The program staff is also working with grocery stores and parks near the participating schools. To evaluate behavior change, pre- and post-intervention surveys were conducted with parents in a sample of participating schools. The number of schools selected, 50, was based on available funds to provide one prize for a drawing at each school. Schools that had at least two intervention elements were included in the initial list of eligible schools; schools were randomly selected for participation. All parents at the randomly selected schools were invited to participate in the survey.

Interval: Interval measures are standard units, such as cups of milk. When survey responses include standard units (e.g., ¼ cup, ½ cup, 1 cup), it is preferred to use paired or matched statistical tests to determine whether there are changes in mean (average) scores before and after the program. Use paired or matched statistical tests, such as a t–test, to determine whether the changes are statistically significant.

Ordinal: Assessments of attitudes or agreement with statements using a Likert-type rating scale use ordinal measures. For ordinal levels of measurement, the simplest approach is to compare percentage distributions of responses before and after the program. For instance, before the program, a certain percentage of participants may strongly agree with a statement; at follow-up, a different percentage may strongly agree. Calculate the percentage change from before to after the program. Unlike interval data, calculating means is not appropriate for ordinal responses. However, comparing the median (middle) or mode (most frequent) response before and after the program can be appropriate. The Wilcoxon Signed-Rank statistical test will identify the level of statistical significance.

Nominal: When an outcome measure is nominal (e.g., names of fruit or answers to “yes or no” questions), these are categorical responses. For nominal data, the simplest approach is to compare percentage distributions of responses before and after the program. For instance, before the program, a certain percentage of participants may drink low-fat milk; at follow-up, a different percentage may drink low-fat milk. Calculate the percentage change from before to after the program. The McNemar’s statistical test will identify the level of statistical significance.

Open-ended versus closed-ended questions: A key part of creating an excellent survey or questionnaire is the appropriate use of open-ended and closed-ended questions. An example of the difference between closed-ended and open-ended questions would be the offer of fish or meat for dinner (closed-ended), as opposed to asking, “What would you like for dinner?” (open-ended). Questions that are closed-ended are conclusive in nature as they are designed to create data that are easily quantifiable. The fact that questions of this type are easy to code makes them particularly useful when trying to determine statistical significance. The information gained by closed-ended questions allows evaluators to categorize respondents into groups based on the options. One drawback to the use of closed-ended questions is the possibility that the response options may not be comprehensive enough to reflect the respondent’s true response. For example, a question may ask if the participant takes the bus, bikes or takes the metro to work, but doesn’t include car-pooling as a response option. Open-ended questions are exploratory in nature. Open-ended questions provide rich qualitative data because the respondent can provide any answer. Since questions that are open-ended ask for critical thinking they are ideal for gaining information from specialists in a field, small groups of people, and preliminary research. Although respondents’ answers are rich in information, it takes great effort to distill the information provided.

Pre- and post-tests: Pre- and post-tests are used to measure changes as a result of an intervention. The pre-test is a set of questions given to participants before the training or activity, and the post-test is administered after the training or activity and contains the same questions as the pre-test. Comparing participants’ post-tests and pre-tests enables the evaluator to assess changes in specific outcomes. In the evaluation framework, healthy eating, physical activity, and food security goals, intentions, and behaviors are the main indicators for which pre- and post-tests are recommended. Evaluators should use a unique, anonymous identifier to facilitate matching of pre-post pairs (e.g., MM/DD/YYYY birthdate).

Limitations of surveys: When in the planning stages of creating and implementing a survey, evaluators should keep in mind survey limitations. Some of these limitations include limited access to the population of interest, compressed or limited time schedule to conduct the survey, and lack of funding.

Bias in survey sampling: Bias refers to a sample statistic that either over- or under-estimates the populations’ parameters. This means that the results from the sample of participants that is drawn either over- or under-represent the true population parameter (such as the true population mean). Bias often occurs when the survey sample does not accurately represent the population.
The bias that results from an unrepresentative sample is called selection bias. Some common examples include: nonresponse bias, undercoverage, and voluntary response bias. This type of bias can be reduced through random sampling. There can also be bias that is introduced through problems in the measurement process such as asking leading questions and respondent social desirability.

Statistical Significance (T-tests): With a t-test, statistical significance indicates that the difference between two groups’ averages most likely reflects an actual difference in the populations from which the groups were sampled. A statistically significant t-test means that the difference between two groups is unlikely to have occurred because the sample happened to be atypical. Statistical significance is determined by the size of the difference between the group averages, the sample size, and the standard deviations of the groups.