A growing need for a skilled workforce in the face of rapid changes in technology and the economy is increasing the pressure for effective and efficient employment and training services. Two of the US’s largest employment and training programs are the Adult and Dislocated Worker programs, which including intensive job search assistance and training and are delivered by the Employment and Training Administration (ETA) within the U.S. Department of Labor (DOL).
The DOL commissioned two research firms, Mathematica Policy Research and MDRC, to conduct a major evaluation of these services to assess the extent to which they are being efficiently and effectively delivered and are having the desired effects. This report (linked below, at which there is an embedded link to the full report) presents estimated impacts of the services based on clients’ experiences over the first 15 months after enrollment (a later report will present the longer-term findings).
The evaluation is based on a “random controlled trial (RCT)” methodology. This is considered to be the “gold standard” approach to program evaluation, as it enables researchers to isolate the true impact of a program by controlling for non-program factors that may have contributed to the outcomes. This is done by comparing the outcomes of those who have received the services (the “treatment” group) to the outcomes of those have not (the “control” group), which represent the “counterfactual,” i.e., what program participants would have done if they had not participated.
In this evaluation, potential participants were randomly assigned to one of three study groups: two treatment groups, one receiving both training and intensive services plus the core services, and another receiving only the intensive services plus the core services; and a control group who were offered core services consisting mainly of information and online tools to help them plan their careers and find a job. The incremental impact of the training services and the combined training and intensive services was measured by separately comparing the outcomes of the two treatment groups to the outcomes of the control group. The incremental impact of the combined intensive and training services over just the training was measured by comparing the outcomes of the two treatment groups to each other.
The interim findings are as follows:
- Intensive services, when offered as a standalone service without training, were effective in increasing earnings and employment. The services increased quarterly earnings by about 17%, or $600 (USD), in the fifth quarter after jobseekers were enrolled in the study. The services also led to participants getting jobs with more fringe benefits such as health insurance and retirement benefits.
- It was too soon to judge the effectiveness of the training. As expected, the researchers found that training reduced jobseekers’ employment and earnings in the first few quarters after enrollment, as those involved in training were less likely to have time for employment. By the fifth quarter, 13 to 15 months after enrollment, these jobseekers had yet to see their training result in higher employment rates or earnings. However, about 15% of all job seekers in the study were still in training at the time of the interim reading and may well realize gains by the time of the final reading.
- Similarly, it was too early to measure the impact of training in combination with intensive services. Together, intensive and training services increased the likelihood of finding work, but this did not translate into higher earnings at the 15-month mark.
This long-term study will be of value to practitioners as well as policy makers interested in understanding the relative effectiveness of different approaches in supporting unemployed job seekers find employment.