Governance and Performance: The Influence of Program Structure and Management on Job Training Partnership Act (JTPA) Program Outcomes

Carolyn J. Heinrich and Laurence E. Lynn, Jr.

In their Harris School working paper titled, Governance Matters: The Influence of Program Structure and Management on Job Training Partnership Act (JTPA) Performance, University of Chicago researchers Carolyn J. Heinrich and Laurence E. Lynn, Jr. investigate the effects of JTPA program governance and management on the employment and earnings of JTPA program participants. Heinrich and Lynn undertake theory-based empirical research, taking advantage of hierarchical, time-varying data available from multiple sites in the national JTPA study, to identify the separate contributions of administrative structures, management strategies, and client characteristics to program outcomes. The findings presented in this paper bear significant policy implications for public management and governance, shedding light on the different outcomes resulting from different administrative entities. This research also emphasizes the benefits of utilizing hierarchical linear modeling statistical techniques in governance research.

Key Findings

Based on descriptive analyses of local administrative structures and management policies, Heinrich and Lynn suggest that when private sector representatives assume more formal authority (through the role of Private Industry Councils, PICs) both as administrative entities and recipients of federal job-training funds, they appear to place more emphasis on measured performance and to adopt administrative practices that demand accountability for results. In their multilevel analyses, Heinrich and Lynn find a large, positive, and highly significant effect of the PIC serving as the administrative entity on JTPA participants' earnings in the first post-program year. For example, when the PIC is the administrative entity, participants earn, on average, $1,737 more in the first post-program year than when a local elected official (LEO)/public entity or a Chief Executive Officer (CEO) of a for-profit firm is the administrative entity. The average performance of sites in which the administrative structure was shared by the PIC and CEO or LEO was the poorest, with participants earning $1,934 less than their counterparts during the same period.

Download a PDF file of the data tables associated with this Research Summary

Results for three of the tested performance incentive variables confirm that policies that strengthen performance incentives and requirements contribute positively to client outcomes. In addition, Heinrich and Lynn found that other management policies, such as the minimum number of standards SDAs must meet to qualify for performance bonuses and the requirement that performance bonuses be targeted to "hard-to-serve" groups, had much weaker effects on participants' program outcomes. These results are consistent with the researchers' argument that the effects of management strategies on client outcomes would be larger for more straightforward (less politicized), easily monitored incentive policies that contribute to a more coherent focus on program goals among management and staff. The results for variables measuring the influence of contracting policies, on the other hand, suggest possible countervailing influences. While earnings outcomes are enhanced when a greater percentage of services are contracted out rather than provided directly, the percentage of performance-based contracts is negatively and statistically significantly related to participants' post-program earnings. Researchers surmise that this may result because the measure of performance-based contracting might be insufficient to adequately characterize the various types of incentives included in the contracts (e.g., how stringent the performance requirements are, performance monitoring activities, and the nature of incentives/sanctions for enforcement.)

Heinrich and Lynn's multilevel models explain about 13 percent of the variation in individuals' earnings outcomes (consistent with other research) and nearly all (up to 97 percent) of the variation in earnings outcomes between the sites studied. Statistical tests confirm that with the structural and management variables included in these models, they were able to almost fully explain the link between governance and performance in JTPA programs (as measured by client employment and earnings outcomes). Reorganizing and restructuring SDA governance toward granting Private Industry Councils full authority over JTPA program operations would tend, ceteris paribus, to improve participant earnings levels significantly; alternatively, requiring PICs to share power equally with political executives in local political coalitions would have a similarly large but negative effect on program outcomes.

Methodology

Heinrich and Lynn analyzed data from two levels of program operations: (1) individual-level data that include measures of JTPA client characteristics (demographics and employment histories) and indicators for the types of training services they received (n=9,621 total observations), and (2) site-level data that includes variables describing the administrative structures in the 16 service delivery areas, performance incentive policies they face, their service delivery/contracting strategies, and the unemployment rate and regional indicators.

Researchers state that in the past, linear regression techniques have been limited in their ability to recognize and maximize the information contained in data with nested or hierarchical structures. Thus, they employ hierarchical linear modeling (HLM), which they find improves statistical estimation in comparison to the ordinary least squares (OLS) approach.

One of the most important governance findings of the researchers' client earnings outcome models is the positive relationship between the role of the PIC as the administrative entity and program performance. However, it is important to know whether this observed relationship occurs mainly because of (1) the advantages of more centralized control or authority over program administration that are realized when the PIC is the administrative entity (a structural component of their governance model) or (2) that it is primarily because PICs are more likely to demand accountability for results through more rigorous performance incentive policies (a management component of the governance model).

To separate these effects, researchers use the results of their most precisely estimated earnings outcome model -- participants' earnings levels in the first post-program year -- to evaluate the evidence for their hypotheses. To compute the effects of these policies on earnings outcomes, Heinrich and Lynn used the estimated coefficients on the performance incentive and contracting policy variables in the hierarchical linear model of participants' earnings in the first post-program year. These coefficient values were multiplied by the average values of the performance incentive/contracting policy variables for each of three SDA administrative structures to obtain their estimated effects on participants' earnings.

Background

The Job Training Partnership Act (JTPA), enacted in 1982 as a major initiative of the Reagan Administration, created what became a $5 billion federally-funded employment and training program for disadvantaged workers. The main features of JTPA governance were inspired by a belief that quasi-markets rather than traditional bureaucracy might be a more reliable way to insure that the program's goals were met. These features included a broad role for the private sector, an emphasis on performance standards and goals rather than traditional command-and-control compliance with procedural mandates, and the decentralization of authority over program implementation to state and local agencies so that administrative regimes could reflect local preferences.

By law, all JTPA service delivery areas are supervised by a Private Industry Council, which must include a majority of private sector representatives. These councils, in conjunction with state and local government representatives, make two main decisions that determine the governance structure of service delivery areas and the degree of power-sharing within them: (1) the choice of administrative entity, or the organization responsible for overseeing program operations, and (2) the grant recipient, or the organization which receives the grant of JTPA program funds from the state and manages the programs on a daily basis. Administrative entities and grant recipients can be a PIC or either a local elected official/public entity or a Chief Executive Officer of a for-profit firm.

JTPA has been the subject of a significant and still growing body of research. The evolution of JTPA research suggests growing interest in getting inside the "black box" of state- and local-level program implementation in order to understand how administrative discretion -- in the selection of program participants, the structuring of administrative and service delivery arrangements, and the use of performance incentives -- influences JTPA program outcomes and impacts. Prior to this work, an aggressive attempt had not been made to assess the potential contributions of governance and management factors in explaining the variance in JTPA program outcomes across sites.

Research Summaries are designed to help broaden the dissemination of current policy-relevant research. These Summaries are funded by the Irving B. Harris Graduate School of Public Policy Studies at the University of Chicago.

For more information, contact Jamie Rosman at HarrisSchool@uchicago.edu or (773) 702.2287.