Frequently Asked Questions - Evidence for ESSA

How can I find the standards and procedures used to create Evidence for ESSA?

How are the new sections of Evidence for ESSA on social emotional learning and attendance different from other sections?

We have recently added sections of Evidence for ESSA on research on social emotional learning (SEL) and attendance. Because there are many distinct measures of SEL and two of attendance, there are several unique features of these sections, as follows:

1. In order to allow for consistency and interpretability across programs reviewed, we have organized SEL measures under four major categories, and within these we have 17 individual variables. The categories and variables are as follows:

Academic
Academic performance
Academic engagement
Problem Behaviors
Aggression/misconduct
Bullying
Disruptive behavior
Drug/alcohol abuse
Sexual/Racial Harassment or Aggression
Early/Risky Sexual Behavior
Social Relationships
Empathy
Interpersonal relationships
Pro-social behavior
Social skills
School climate
Emotional Well-Being
Reduction of Anxiety/depression
Coping skills/stress management
Emotional regulation
Self-esteem/self-efficacy

When we review studies that use other names for these variables, we translate them to our variable names. For example, we would report “externalizing” as “aggression/conduct problems” and “cooperative behavior” becomes “social skills.” We always list key references to articles on each included program, and readers interested in the original variable names can find them in those articles.

Attendance has just two categories, “average daily attendance” and “chronic absenteeism,” defined as the number of students absent more than 10% of school days each year, which are common school accountability metrics many states use under ESSA..

2. SEL and attendance variables include behaviors we want to see more of (such as emotional regulation and average daily attendance) and those we want to see less of (such as bullying, anxiety/depression, and chronic absenteeism). We have reversed the +/- signs on negative behaviors, so that all outcomes (in effect sizes) with a positive sign are desirable. Some articles list, for example, “absenteeism” as a negative number if it is going down more in the experimental group than in the control group. We have recoded “less absenteeism” as a positive number.

3. Because we have multiple, very different outcomes across SEL studies and two for attendance studies, our reports on these topics are more complicated than those for reading and mathematics. The summary pages for SEL show outcomes on the four categories. Following definitions in the ESSA law, the categories are marked in green if at least one variable in that category met the ESSA “Strong” criterion, in orange if at least one variable met the ESSA “Moderate” criterion, and in fuchsia if at least one variable met the ESSA “Promising” criterion. If a category contains variables that met different ESSA levels, the summary page shows the highest level reached across all variables measured. For example, if one variable under Social Relationships had a “Strong” rating and another had a “Promising” rating, the summary page would indicate social relationships in green (i.e., “Strong”).

The summary also shows the average effect size for the category, including all variables in that category. This means that it is possible that a given category is marked in green (“Strong”), but the mean effect size is very low. This could happen if a program had a statistically significant positive outcome on one variable but a zero or negative outcome on one or more other variables in that category. If the category average is zero or negative, however, we do not list the category as meeting ESSA standards, even if it does have one or more outcomes meeting one of the three ESSA criteria.

4. As in the reading and mathematics reviews, programs are ranked on the summary page from highest to lowest. These rankings are based on the number of categories rated “Strong,” “Moderate,” or “Promising,” the average effect sizes, the number of studies, and the number of students involved in the outcomes.

The exact formula for ranking is (Σ(ES)(100)(1+SQRT N/100), where effect sizes are quadrupled if “Strong,” tripled if “Moderate,” and doubled if “Promising.”

5. SEL and attendance can be filtered by specific outcome measures, specific student ages, demographic characteristics of study populations (e.g., race, ELs, percent free lunch, urban/rural), and other study features.

6. Program pages go into more detail, providing a narrative description of each program and of program outcomes, as well as much other information. Costs, contacts for program providers, and other information are listed if we have them from the developers or disseminators.

7. We search exhaustively for all SEL and attendance programs available for use in the U.S. (as we do in all sections of our website). If a program has no evidence that meets our inclusion standards, or if its evidence does meet our inclusion standards but no study found any effects that meet ESSA standards for “Strong,” “Moderate,” or “Promising” evidence of effectiveness for even a single variable, we include the study, as “No studies met inclusion criteria” or “Qualifying studies demonstrated no significant positive outcomes.” To find studies that fell into one of these categories, type in the program name in the green box that appears in the upper right corner. If you find any SEL or attendance programs you know to exist but that do not appear when you type the program name in the green box, please tell us, and send us any studies or citations to studies. Our intention is to have every program currently in existence represented in our website.

What is ESSA?

What are the different standards or levels of evidence defined in ESSA?