Since 2006, the Career Pathways Initiative (CPI) administered by the Arkansas Department of Higher Education and the state’s 25 two-year colleges and technical centers has provided education and training to more than 30,000+ low-income Arkansans, helping them to acquire credentials and degrees leading to jobs in selected high-demand and high-wage industries. In 2014, ACC issued a Request for Proposals (RFP) seeking an evaluator for the state’s CPI. We knew from prior work that we conducted in Arkansas in 2009-10 assessing CPI performance that the program had been hitting and exceeding enrollment, educational attainment, and employment targets for many years. Yet we also knew that state financial constraints had resulted in significant decreases in CPI’s budget in recent years. The ACC RFP captured our interest because it emphasized complementing a rigorous outcomes analysis with a Return-on-Investment (ROI) analysis. In addition to seeking insights into the extent to which CPI students outperformed their non-CPI counterparts with respect to educational attainment and earnings, ACC sought to show that the CPI was benefitting all state residents, not just the participating students. The project seemed like an ideal opportunity to provide a partner with evidence they could use to demonstrate impact and attract investments to offset the budget reductions its program had been facing.

We were delighted when ACC selected us to conduct the CPI research. Last year ACC released our Phase-1 College Counts! findings http://www.collegecounts.us/results/, which showed, among other findings, that 52 percent of the students who participated in the CPI program from 2006 to 2013 completed at least an associate degree or technical certificate, compared to 24 percent of similarly-situated students who did not participate in CPI. In addition to impressive academic achievements, CPI participants in 2009 experienced an earnings boost during the year after program completion that was $2,600 higher than earnings for the matched comparison group of the TANF population who did not participate in CPI.

We are pleased that ACC has recently released the Phase-2 ROI analysis results. Our model compared program costs with the total of state tax revenues associated with CPI participants’ earnings gains, and decreases in public assistance spending by the state and federal government associated with CPI participation. The analysis demonstrated that there was a 179 percent return on investment of funds to taxpayers. In other words, every dollar invested in CPI programming resulted in a return of $1.79 to taxpayers over five years. ROI studies are often overly reliant upon average cost data and/or assumptions applying multiplier effects that seem too optimistic or far-fetched. With this in mind, the method that we used for the College Count$ ROI study aimed to minimize the need for such assumptions through maximum use of actual case-level data derived from state data files and the study’s various CPI treatment groups and their respective matched comparison group counterparts.

We are hopeful that the strong ROI findings can attract new sources of funding for CPI. We believe that more programs should consider building ROI into their impact study designs so that successful programs receive the investments needed to sustain effective (i.e., evidence-based) programming.

For more information about the ACC CPI evaluation, please contact Michael Scuello, Senior Associate, Metis Associates: mscuello@metisassoc.com.