Methodology & Findings
To calculate grades for each of the report’s six main categories—student access and success, efficiency and cost-effectiveness, meeting labor market demand, transparency and accountability, policy environment, and innovation—we analyzed a wide range of data both from existing sources and from our own research. The rationale for examining each category, the basic methodology used for each subcategory, and the core findings can be found in the six sections that follow.
Some major methodological decisions were common to all six categories. Our state-by-state analysis focused solely on public colleges and universities at the four- and two- year level. We chose to focus on these institutions for two reasons. First, they educate the vast majority of students enrolled in postsecondary education. Second, they are the institutions over which states have the most control, and they receive the bulk of state tax dollars dedicated to higher education. Private colleges, while important, represent a smaller portion of the higher education universe and are not subject to the same state regulatory and financial constraints as public institutions.
We also limited our scope to include four- and two-year degree and non-degree granting colleges. Less than two-year institutions, many of which include technical high schools and adult basic education centers, were not included in the analysis. While these schools are an important piece of some states’ workforce development portfolio, they are few in number and enroll a small fraction of postsecondary students. We also excluded tribal colleges, service academies, and special focus medical schools and medical centers from the analysis.
Wherever possible, we disaggregated our metrics across four-year and two-year colleges. We did this for two reasons. The first was to adequately reflect different institutional missions. The second was to ensure that we were providing a holistic picture of the various pieces of the higher education portfolio in each state. We categorized institutions based on the primary credential they award (i.e., institutions that are labeled four-year colleges but award primarily associate’s degrees or below are categorized as two-year colleges).
For the data-driven metrics in the first and second sections, data were collected at the institutional level. State-level averages, therefore, reflect sum totals of credentials produced, students retained, or dollars spent across all of the public institutions in the state in a given category. In other words, these are weighted averages, where the completion rate at any single institution is weighted by its size relative to the other institutions in the category and in the state. Wherever possible and unless otherwise noted, three years of data were used in order to smooth out any irregularities.
For each of the six sections, when we aggregated the individual measures into overall grades we weighted each measure equally. For sections one through three, which are based entirely on education data, we graded states on a five-point scale, with five points signifying the best possible score, according to how far above or below the national mean a state fell on each individual metric. We then summed these ratings and assigned letter grades on the basis of each state’s overall score. For the policy metrics in sections four through six, we combined the individual measures to create an overall score out of 100, then assigned grades by quintile.
For further details on how the grades were calculated, see the Technical Appendix.