Assessing the Evidence Base: Strategies That Support Employment for Low-Income Adults

Over the past 25 years, there has been a plethora of research on public interventions intended to improve employment conditions for low-income adults. However, the sheer volume of the evidence generated has made it difficult to identify the most reliable and relevant research and therefore to learn from it. This paper by Emily Sama-Miller, Alyssa Maccarone, Annalisa Mastri, and Kelley Borradaile of Mathematica Policy Research attempts to address this gap.

It describes the studies assessed in, and presents some results from, The Employment Strategies for Low-Income Adults Evidence Review (ESER) which was conducted by the Office of Planning, Research and Evaluation of the U.S. Health and Human Services Department. The ESER was designed “to provide … a transparent, [accessible,] and systematic assessment of the research evidence for effectiveness of programs designed to improve the employment-related outcomes of low-income adults.”

In total, the authors examined 314 studies of 108 interventions that came from 51 evaluations. A study was reviewed if it targeted an intervention designed to improve employment-related outcomes among low-income adults in the United States, as well as in the United Kingdom and Canada, and if it used randomized controlled trials or comparison group designs (i.e. comparing the outcomes of program and comparison groups) to evaluate the intervention. The research was assessed on a number of criteria, including research quality, type of intervention, outcome measure, direction of impact, and time horizon. Other papers in this series include one describing the standards and methods used to assess the research, and others providing detailed results for each type of intervention.

Interventions assessed typically comprised a few different strategies and services in combination, but one strategy often stood out in terms of treating the most clients. In about three-quarters of them, one of six primary strategies was tested:

  • Work-readiness activities (18%): assessing employment barriers, skills, and interests; helping to design a résumé and cover letter; setting up job clubs or job-readiness workshops; job shadowing; developing an individual employment plan;
  • Financial incentives (14%): providing rewards for engaging in a specific activity or achieving a certain goal or sanctions for failing to participate in mandated services;
  • Employment retention services (14%): helping employed workers stayand advance in their jobs;
  • Case management (12%): holding individual or small group meeting(s) with an employment specialist or counselor who helps to assess clients’ needs and address barriers, including coaching and developing an individualized plan;
  • Training (10%); and
  • Education (8%): providing services to support educational attainment, such as GED (General Educational Development) support, adult basic education, or post-secondary education.

The main findings of the review are the following

  • Research quality: ESER found the quality of the research to be high. Of the 314 studies reviewed, 246 (78%) received a high rating, and one a moderate rating. The remaining 67 studies received a low rating. In this brief, the focus is on the 247 highly- or moderately-rated studies.
  • Outcome measure: A large majority had at least one measure of earnings (95%), employment (94%), or benefit receipt (87%). Education and training outcomes were seldom measured (only 15% of the studies).
  • Impact: Less than 3 in 10 of the statistically significant impacts were favourable. Of the 2,251 impacts measured, just 28% were positive; 67% were null and 5% were unfavourable. Favourable outcomes were relatively more common in the long-term than in the short-term. This suggests that employment and training interventions may require a longer time horizon (more than 18 months) to produce positive impacts for low-income adults. This was true for the outcomes of employment, earnings, and benefits receipt, but particularly so for education and training outcomes.

This summary paper, as well as the individual studies in the The Employment Strategies for Low-Income Adults Evidence Review, would be useful to practitioners, policymakers, and researchers who are interested in learning about what works in public programming directed at improving low income adults’ employment and income outcomes.