About the Study
In 2017, the Chief Evaluation Office (CEO) partnered with the Wage and Hour Division (WHD) to fund contractor Mathematica to conduct the Wage and Hour Division’s Compliance Strategies Evaluation. The impact design and outcomes evaluation aim to build evidence on how to assess effective compliance strategies and compliance assistance within WHD, including exploring ways WHD may use external data for ongoing monitoring and evaluation. The study also included design options for an impact evaluation of compliance assistance strategies informed by behavioral science. The design options will provide a framework and logic model for WHD's compliance assistance program and will support WHD in using rapid evaluation to support continuous learning and improvement.
The Department of Labor-funded study was a result of the annual learning agenda process. It contributes to the labor evidence-base to inform worker protection programs and policies and addresses Department strategic goals and priorities
- Compliance Strategies Evaluation Literature and Database Review (Final Report, October 2020)
A summary of what is currently known about effective compliance assistance strategies and suggestions for further research that can address gaps observed in the literature. - Exploring External Data to Enhance Monitoring and Evaluation of WHD’s Compliance Strategies (Final Report, October 2020)
Opportunities and limitations of WHD’s internal case management system, the Wage and Hour Investigative Support and Reporting Database (WHISARD), and a selection of external data sources for evaluation. - WHD Compliance Strategies: Directions for Future Research (Final Report, October 2020)
Highlights of effective practices for building and using evidence. - Behavioral Interventions for Compliance Assistance (Design Report, October 2020)
Intervention and evaluation designs for two behavioral intervention (BI) trials that, if implemented, would test whether webinar registrations increase when behavioral strategies are applied to emails targeting a given industry.
- What is the current evidence on effective approaches to delivering compliance assistance?
- What key outcomes are associated with WHD compliance strategies?
- Are there opportunities to implement and test behaviorally informed compliance assistance strategies?
- The Compliance Strategies Evaluation Literature and Database Review findings suggest that there are five gaps in evidence on the effectiveness of compliance strategies that should be addressed by further research, one of which is research on the effectiveness of behavioral interventions in increasing compliance among employers.
- The Exploring External Data to Enhance Monitoring and Evaluation of WHD’s Compliance Strategies found that low match rates limit the integration of WHISARD data and external datasets. The population of establishments covered in the WHISARD differ from those covered in external datasets, limiting the comparability of samples drawn from different sources.
- The WHD Compliance Strategies: Directions for Future Research report poses a monitoring and evaluation framework defined by five factors: documented and supported core activities; measurable outcomes; available, appropriate data; implementation maturity; internal validity. Considerations within each factor are described that are essential to the development of a strong monitoring and evaluation process. These considerations are presented as potential opportunities and potential challenges.
- The Behavioral Interventions for Compliance Assistance: Design Report findings suggest behavioral roadblocks may be preventing employers from ultimately attending WHD webinars. Engagement drop-off ranges from the employer not noticing or opening emails about the webinar to the employer forgetting to attend the webinar after they have registered.
Report
Dolfin, S., Maxwell, N., Gould-Werth, A., Yañez, A., Deutsch, J., Hendrix, L. (2020). Mathematica. Compliance Strategies Evaluation Literature and Database Review. Chief Evaluation Office, U.S. Department of Labor.
Report
Patnaik, A. (2020). Mathematica. Exploring External Data to Enhance Monitoring and Evaluation of WHD’s Compliance Strategies. Chief Evaluation Office, U.S. Department of Labor.
Report
Dolfin, S., Maxwell, N., Patnaik, A. (2020). WHD Compliance Strategies: Directions for Future Research. Chief Evaluation Office, U.S. Department of Labor.
Design Report
Deutsch, J., Gonzalez, N., Maxwell, N. (2020). Behavioral Interventions for Compliance Assistance: Design Report. Chief Evaluation Office, U.S. Department of Labor.
The Department of Labor’s (DOL) Chief Evaluation Office (CEO) sponsors independent evaluations and research, primarily conducted by external, third-party contractors in accordance with the Department of Labor Evaluation Policy. CEO’s research development process includes extensive technical review at the design, data collection and analysis stage, including: external contractor review and OMB review and approval of data collection methods and instruments per the Paperwork Reduction Act (PRA), Institutional Review Board (IRB) review to ensure studies adhere to the highest ethical standards, review by academic peers (e.g., Technical Working Groups), and inputs from relevant DOL agency and program officials and CEO technical staff. Final reports undergo an additional independent expert technical review and a review for Section 508 compliance prior to publication. The resulting reports represent findings from this independent research and do not represent DOL positions or policies.