Skip Navigation
Edvisors Star Logo
Edvisors Logo
  • Compare Lenders
      • Compare Lenders
      • Best Private Student Loans in March 2023
      • Student Loan Comparison: Federal Loans vs Private Loans
      • Parent PLUS Loans vs Private Student Loans
      • How Does Interest on Student Loans Work?
      • Private Student Loan Cosigner: Everything you Need to Know
      • View All Articles >
      College ave student loans apply now

      This is an advertisement.

      • Compare Lenders
      • Best Student Loan Refinance Companies for March 2023
      • Federal Student Loan Repayment Plans
      • Student Loan Forgiveness: How to Get Student Loans Forgiven
      • Repayment Plans for Private Student Loans
      • Parent PLUS Loan Forgiveness
      • View All Articles >
      refinance student loans with college ave get started

      This is an advertisement.

    • Private Student Loans
    • Refinance Student Loans
  • Plan for College
      • Best Job Search Sites
      • Resumes That Get Noticed
      • High Paying Careers
      • STEM Careers
      • View All Articles >
      Win Scholarships - Join ScholarshipPoints.com

      This is an advertisement.

      • Measuring Academic Fit: Reach, Match and Safety Schools
      • Need-Blind Admissions
      • How Long is a College Essay
      • When are College Applications Due
      • How to Write a Good College Essay
      • View All Articles >
      Win Scholarships - Join ScholarshipPoints.com

      This is an advertisement.

      • Filing the FAFSA 2023-24: A Simple Guide for Students
      • FAFSA Deadlines for 2023-2024
      • FAFSA FAQ - Your FAFSA Questions Answered
      • View All Articles >
      Win Scholarships - Join ScholarshipPoints.com

      This is an advertisement.

      • Most Popular Scholarships for College Students in 2023
      • Apply for Free Scholarships for College
      • College Scholarships for Kids
      • Most Prestigious Undergraduate Scholarships
      • Las mejores becas para estudiantes hispanos y latinos
      • View All Articles >
      Win Scholarships - Join ScholarshipPoints.com

      This is an advertisement.

      • How to Pay for College
      • Tuition Payment Plans
      • Pros and Cons: Borrowing From My 401K to Pay for College
      • Paying College Tuition with a Credit Card
      • Best Ways to Pay for College Without Financial Aid
      • View All Articles >
      Win Scholarships - Join ScholarshipPoints.com

      This is an advertisement.

      • Financial Aid Disbursement: When and How You Will Get It
      • Should I Pay Off My Student Loans Right Now
      • How to Minimize Student Loan Debt
      • View All Articles >
      Win Scholarships - Join ScholarshipPoints.com

      This is an advertisement.

    • Careers
    • College Admissions
    • FAFSA
    • Scholarships
    • Paying for College
    • Student LIFE Blog
  • Student Loans
      • Filing the FAFSA 2023-24: A Simple Guide for Students
      • FAFSA FAQ - Your FAFSA Questions Answered
      • FAFSA Deadlines for 2023-2024
      • FAFSA School Codes
      • FAFSA 2023-2024 Guide & Resources
      • View All Articles >
      College ave student loans apply now

      This is an advertisement.

      • Best Private Student Loans in March 2023
      • Student Loan Comparison: Federal Loans vs Private Loans
      • Parent PLUS Loans vs Private Student Loans
      • How Does Interest on Student Loans Work?
      • Private Student Loan Cosigner: Everything you Need to Know
      • View All Articles >
      College ave student loans apply now

      This is an advertisement.

      • Filing the FAFSA 2023-24: A Simple Guide for Students
      • FAFSA FAQ - Your FAFSA Questions Answered
      • The complete guide to federal undergraduate student loans
      • What are Federal Parent PLUS Loans and How to Apply?
      • Financial Aid for Graduate School
      • Compare Subsidized and Unsubsidized Student Loans
      • View All Articles >
      College ave student loans apply now

      This is an advertisement.

      • Best Student Loan Refinance Companies for March 2023
      • Federal Student Loan Repayment Plans
      • Student Loan Forgiveness: How to Get Student Loans Forgiven
      • Repayment Plans for Private Student Loans
      • Should You Refinance Federal Student Loans?
      • Parent PLUS Loan Forgiveness
      • View All Articles >
      • Sallie Mae Student Loan Review
      • College Ave Student Loan Review
      • SoFi Student Loan Review
      • Ascent Student Loan Review
      • View All Articles >
      College ave student loans apply now

      This is an advertisement.

    • FAFSA
    • Private Student Loans
    • Federal Student Loans
    • Repay Student Loans
    • Lender Reviews
  • Credit Cards
      • Petal® 2 Visa® Credit Card Review
      • SoFi Credit Card Review
      Win Scholarships - Join ScholarshipPoints.com

      This is an advertisement.

      • Best Credit Cards for 2023
      • Best Cash Back Credit Cards
      • Best 0% Introductory APR Credit Cards
      • Best Secured Credit Cards for 2023
      • Best Balance Transfer Credit Cards
      Win Scholarships - Join ScholarshipPoints.com

      This is an advertisement.

      • How to Build Credit?
      • What Age Can You Get a Credit Card?
      • What is a Secured Credit Card?
      • What is an Unsecured Credit Card?
      Win Scholarships - Join ScholarshipPoints.com

      This is an advertisement.

    • Card Reviews
    • Best Credit Cards
    • Credit Card FAQs
  • Money Management
      • Best Budgeting Apps
      • Budgeting Tips
      • Budgeting for Beginners
      Win Scholarships - Join ScholarshipPoints.com

      This is an advertisement.

      • Financial Goals
      • How to Save Money as a College Student
      • What is an Emergency Fund
      Win Scholarships - Join ScholarshipPoints.com

      This is an advertisement.

      • How to Pay Off Debt
      • Debt Relief
      • What is a Debt Management Plan
      Win Scholarships - Join ScholarshipPoints.com

      This is an advertisement.

      • How to Check Your Credit Score
      • Best Credit Building Apps
      • What is Credit?
      Self Credit Builder: Build Credit. Build Savings. Build Dreams.

      Sponsored Ad

      • Investing for Beginners
      • Best Investment Apps
      • Understanding the Stock Market
      Win Scholarships - Join ScholarshipPoints.com

      This is an advertisement.

      • What is Risk Management
      • How to Prevent Identity Theft
      • What is Insurance
      Info compromised in a breach? Don't compromise on identity theft protection.

      Sponsored Ad

    • Budgeting
    • Saving
    • Debt Management
    • Credit
    • Investing
    • Risk Management
Search Box Form
School Search Form
Advertisement Disclosure
Advertisement Disclosure
×

Edvisors (“Edvisors Network, Inc.”) provides independent advertising-supported platforms for consumers to search compare and apply for private student loans. Loan offers from participating lenders that appear on our websites are not affiliated with any college and/or universities, and there are no colleges and/or universities which endorse Edvisors’ products or services. Lender search results do not constitute an official college preferred lender list. Edvisors receives compensation from lenders that appear on this site. This compensation may impact the placement of where lenders appear on this site, for example, the order in which the lenders appear when included in a list. Not all lenders participate in our sites and lenders that do participate may not offer loans to every school.

Edvisors is not a lender and makes no representations or warranties about your eligibility for a particular loan or financial aid. Lenders are solely responsible for any and all credit decisions, loan approval and rates, terms and other costs of the loan offered and may vary based upon the lender you select. Please check with your school or lender directly for information related to your personal eligibility.

Edvisors has endeavored to provide accurate information. However, the results provided by lenders are for illustrative purposes only and accuracy is not guaranteed, as such, Edvisors assumes no responsibility for errors or omission in the information provided.

×
Get Your Free Guide to Filing the FAFSA Today!

Thank you for your interest in Edvisors' 2020-2021 Guide to Filing the FAFSA.

Please complete the form below to access your free copy.

Thank you! Click here to download your free FAFSA Guide.

I am a:

×
College knowledge starts here!

Enter your email below to receive your Student Loan Handbook from Edvisors.

Please check your email for the Student Loan Handbook.

After Navigation
Home Blog Government's College Rating System Lacks Details
  • Contents
  • Three Main Groups of Metrics
  • How will Metrics be Combined?
  • Accuracy Matters
  • Slow Progress
  • The Challenge of College Ratings and Rankings
  • Conclusion

Government's College Rating System Lacks Details

Date Published : December 29, 2014
Authored By : Edvisors Network
Share
Email This Article
Fill out the form below to send a copy of this article to your email.
By clicking "Submit" I have read and agree to the Edvisors' Terms of Use and Privacy Policy. We strongly encourage you to read our Privacy Policy to understand how we use and share your information.
A copy of this article has been sent to your email.

After more than a year in development, the U.S. Department of Education released a draft framework for its much-anticipated Postsecondary Institution Rating System (PIRS) on Friday, December 19, 2014.

The draft framework is incomplete and unfinished. In any kind of a rating or ranking system, the details matter, and in this case, the most important details are missing. By now, the U.S. Department of Education should have finalized the list of data sources, the metrics that will be used to rate the colleges and, most importantly, how the various metrics will be combined. But, the draft framework is tentative even about the nearly a dozen variables that will be the basis for the metrics.

Three Main Groups of Metrics

The metrics will focus on access, affordability and outcomes, but not the quality of education.

The access metrics will be based on

  • Percent Pell – the percentage of the student body that have received a Federal Pell Grant
  • EFC Gap – the average difference between each student’s Expected Family Contribution (EFC) and an unspecified baseline EFC
  • Family Income Quintiles
  • First-Generation College Status

As previously reported by the Chronicle of Higher Education on August 15, 2014, a key problem with family income quintiles as currently used by the U.S. Department of Education’s College Navigator is that the colleges can currently choose which students are in each quintile. Some colleges use adjusted gross income (AGI) while others use their own method.

The affordability metrics will be based on

  • Average Net Price
  • Net Price by Quintile

It is unclear, however, whether the net price figures will be averages for just the first year or for the full length of the education program. About half of all colleges practice front-loading of grants, where first-year students get a better mix of grants than during subsequent years. This yields a higher net price for upperclassmen.

The proposal omits consideration of average loan debt as an affordability metric. Colleges often talk about loans as making college more affordable, but don't consider whether students are graduating with unaffordable debt. Affordable debt can be measured using debt-to-income, debt-service-to-income and debt-service-to-discretionary-income ratios.

The outcomes metrics will be based on

  • Completion Rates – the percentage of the incoming class who graduate within a particular time-frame
  • Transfer Rates – the percentage of the incoming class who transfer to another college
  • Labor Market Success – the percentage of a college's graduates whose income at an unspecified time after graduation exceeds a baseline income level, such as 200 percent of the poverty line or the Federal minimum wage
  • Graduate School Attendance – the percentage of the graduating students who subsequently enroll in graduate or professional school
  • Loan Performance Outcomes – one or more measures of the performance of a college's loan portfolio, such as the percentage of borrowers whose loans are negatively amortized, the percentage of initial loan balances that are repaid or a loan repayment rate based on adding deferment and forbearance rates to default rates

The discussion of transfer rates provides a good example of how the U.S. Department of Education’s proposal lacks decisive detail concerning the metrics. The framework does not specify how it will handle transfer students, both students who transfer out of an institution and student who transfer into an institution. Many current data sources calculate outcomes based solely on first-time, full-time students, omitting students who transfer into a college and students who enroll part-time. Will transfer students count if they do not eventually graduate? The framework also does not specify whether it counts only students who transfer from 2-year institutions to 4-year institutions, or also lateral transfers (from 2-year to 2-year and from 4-year to 4-year) and reverse transfers (from 4-year to 2-year). There are more questions here than answers.

The loan performance outcomes metric does not use a direct measurement of the percentage of a college's borrowers who graduate with excessive debt, such as debt-to-income ratios, debt-service-to-income ratios or debt-service-to-discretionary-income ratios that exceed a particular threshold.

It is unclear why the U.S. Department of Education chose to benchmark labor market success against 200 percent of the poverty line. Eligibility for means-tested federal benefit programs is often based on comparing income with 100 percent, 125 percent, 130 percent, 150 percent or 185 percent of the poverty line. Although eligibility for the Federal Pell Grant is not currently based on a comparison of income with the poverty line, 84.4% of recipients have income below 200 percent of the poverty line, 93.0% have income below 250 percent of the poverty line and 97.4% have income below 300 percent of the poverty line, based on the 2011-12 National Postsecondary Student Aid Study (NPSAS).

The discussion of the impact of demographics is likewise minimal. The U.S. Department of Education proposes using a regression model based on family income, Expected Family Contribution (EFC), dependency status, parent educational attainment, age, gender, marital status, has children, veteran status, zip code, state of residence, transfer status and enrollment intensity, but not academic preparation (e.g., high school GPA and SAT/ACT scores), academic program or race. This is despite research that demonstrates a strong correlation between college completion and academic preparation. Without consideration of high school GPA and admissions test scores, it is not possible to distinguish between the selectivity of a college and the value added by the college. More than half of the better performance of elite colleges is due to their aggregating the most talented students. Accordingly, the ratings system may be meaningless in evaluating college performance, other than determining whether each college satisfies a set of minimum standards.

The current framework proposes to cluster predominantly 4-year institutions together and predominantly 2-year institutions together. This means that colleges that offer a mix of 4-year and 2-year degree programs will be compared with dissimilar institutions. The ratings need to be disaggregated by academic degree level, so that institutions that offer both Bachelor's degrees and Associate's degrees will have two ratings, one for each credential. The ratings also need to be disaggregated by or adjusted for other important institutional characteristics, such as selectivity, average endowment per student and state support per student, among other variables. Moreover, the outcome measures should perhaps be disaggregated by or normalized according to some of the access variables, such as Federal Pell Grant recipient status.

How will Metrics be Combined?

The method used to integrate the metrics is one of the most important unanswered questions. The proposal calls for a small number of unspecified performance categories, with three performance levels within each category: high, middle and low.

To the extent that the rating system is an attempt to influence consumer behavior – to shift enrollment to colleges that offer more “value” – it is unclear if having just three labels, as opposed to finer gradations, provides sufficient information to be useful to students and their families. There is a conflict between simplicity, accuracy and utility. In effect, the rating system may represent little more than a new set of minimum standards that colleges must satisfy for continued receipt of federal student aid funding. The limited information contained within the rating system may also make it difficult for colleges to determine what changes they need to make to improve their ratings.

Another risk is that the weights used to combine the metrics can be shaped to achieve a particular outcome. The choice of model will affect how the colleges are rated. One person’s definition of “value” may differ from another person’s definition. The design of the ratings system may be driven more by ideology than by policy considerations. The final ratings scheme may be based on subjective priorities as opposed to objective analysis.

There may also be significant unanticipated outcomes. For example, if the metrics seek to maximize graduation rates, as opposed to the number of college graduates, colleges may become more selective in their admissions policies. The easiest way to increase graduation rates is by filtering out high-risk students, such as low-income students, minority students, first-generation students and students who are single parents. That is a lot less expensive than removing the obstacles to student success. Helping high-risk students reach the finish line is more expensive than denying them access to a college education. There is a tension between access and completion, mediated by money.

Similarly, if the U.S. Department of Education were to try to improve graduation rates by setting minimum graduation rates for institutional eligibility for Title IV federal student aid funds, it would yield a decrease in the number of students graduating. (It would also lead to graduation inflation, as some schools would relax their graduation requirements.) Currently, 58% of students in Bachelor’s degree programs graduate within six years. If the U.S. Department of Education were to get rid of all colleges where less than half the students graduate in six years, 6.7 million students would lose eligibility for student financial aid, a total of $24 billion a year. While this change would increase the graduation rate by 14 percentage points, to 72%, it would decrease the number of students graduating by 26%. It would also decrease the number of colleges by 55% and the number of college students by 41%.

Consider a student who is a single parent. If the student’s babysitter is sick, the student has to stay home from school, missing class. The student’s academic performance deteriorates. If the student misses enough classes, he or she may be forced to drop out of college and may eventually default on his or her student loans. If faced with pressure to increase graduation rates, some colleges will change their admissions standards in ways that directly or indirectly prevent these students from obtaining a college education. Other colleges will respond by providing these students with access to reliable childcare resources, enabling them to graduate at the same rates as students who aren’t single parents.

Accuracy Matters

The proposed rating system will suffer from the GIGO problem. GIGO is an acronym for “garbage in, garbage out.” If one provides a computer program, such as a rating system, with inaccurate input data, it will produce inaccurate output results.

Available data is limited, suffers from accuracy problems and may be prone to manipulation. The main data sources include the Institutional Postsecondary Education Data System (IPEDS), the Common Data Set (CDS), and the National Student Loan Data System (NSLDS). IPEDS and CDS both contain unverified school-reported aggregate data. Several colleges have admitted to submitting inaccurate data to CDS, which is used by many college ranking systems. For example, some colleges based the debt-at-graduation figures reported to CDS on just need-based loans instead of all loans. Others calculated a mean debt-at-graduation figure instead of an average figure, by dividing the total debt at graduation by the total number of students who graduated instead of just the students who graduated with debt. Even NSLDS, which contains individual student-level data, suffers from data quality issues.

The data will also be prone to manipulation. For example, colleges will become more selective by admitting the least risky of the high-risk students. They might continue admitting Federal Pell Grant recipients, but limit these students to those who have the highest high school GPA or admissions test scores, or limit the students to the highest-income subset of Federal Pell Grant recipients.

Slow Progress

One could argue that the rating system has been in development since the White House introduced the FY2010 budget proposal on February 26, 2009. Page 378 of the U.S. Education of Education appendix to the FY2010 budget included a proposal to reallocate Perkins Loan funding according to a yet-to-be-determined formula:

“Loan volume would be allocated among degree-granting institutions using a method to be determined in consultation with Congress. The Administration intends for this new formula to encourage colleges to control costs and offer need-based aid to prevent excessive indebtedness. It may also reward schools that enroll and graduate students from low- and moderate-income families.”


A variation on this proposal was introduced in the U.S. House of Representatives on July 15, 2009, as part of the Student Aid and Fiscal Responsibility Act of 2009 (111th Congress, H.R. 3221), also known as SAFRA. The legislation proposed to base the allocation formula for Perkins Loans in part on incentives for colleges to maintain below-average tuition and to graduate Federal Pell Grant recipients. The legislation passed the House on September 17, 2009 by a vote of 253 to 171, but the proposals for reengineering the Perkins Loan were dropped in the Senate in what ultimately became the Health Care and Education Reconciliation Act of 2010 (P.L. 111-152).

The current rating system proposal might be little more than the latest reincarnation of SAFRA.

The U.S. Department of Education is soliciting comments from the public to [email protected] through February 17, 2015. These public comments will be informal, not through the regulatory process. It is unclear how the U.S. Department of Education will be able to finalize the details of the proposal and hold another public comment period on the final proposal in time to have the rating system implemented before the 2015-2016 academic year.

The Challenge of College Ratings and Rankings

Commercial organizations that attempt to evaluate college performance and create college rankings and ratings are subjected to significant criticism every year. So, why does the U.S. Department of Education believe that it can do better?

The U.S. Department of Education has a weak track record so far. For example, the College Scorecard reports the median debt of college graduates and dropouts combined. This yields little value to prospective students, who want to understand the likely debt burden if they graduate and the risks associated with failure. Blending the two figures into a single metric is not informative. Even if the U.S. Department of Education were to report the median debt of college graduates and dropouts separately, use of a single metric for each metric is still largely uninformative. Colleges where less than half the students borrow would report a median debt of zero. More information is necessary, such as the percentage of students graduating with debt and the mean/median debt among students who graduate with debt. Even then, the debt figures may be prone to the Yule-Simpson effect, where trends can disappear when groups of data are combined.

Similarly, the U.S. Department of Education’s attempts to develop a gainful employment metric were flawed by an arbitrary choice of thresholds for debt-service-to-income ratios. These thresholds were based on a small subset of old mortgage lender underwriting standards for overall and mortgage debt, as reported in a single paper, P. Scherschel, Student Indebtedness: Are Borrowers Pushing the Limits?, USA Group Foundation, 1998. (Most of the other papers were merely repeating the same set of flawed figures.) This derived threshold was not based on any kind of universal standard. For example, the U.S. Department of Education used a different threshold internally at the time. Moreover, the U.S. Department of Education’s gainful employment regulation effectively required all college graduates to be able to qualify for a mortgage as its criterion for gainful employment.

Conclusion

The U.S. Department of Education seems eager to move forward with a rating systems despite acknowledging numerous limitations and flaws.

The first principle of a rating system should be to do no harm.

The U.S. Department of Education needs to have the courage to abandon the rating system if, as seems apparent, current data sources are inadequate to do a credible job.

To paraphrase an idea from Stephen Jay Gould’s book, The Mismeasure of Man, just because you can calculate a number doesn’t mean it measures anything real.

Share

About Edvisors

For 20 years we have been known as one of the largest and most trusted resources to help you find your path to success. Everyone needs to find their own path, and we know that first-hand. Helping you reach your educational and financial goals without stress, is our mission. Let's face it, it may be unavoidable to borrow money, but there are ways to do it wisely, including making the right choices up front, as well as during repayment. If we can help you make an informed and careful decision, we consider that a good day in the office.

Find information on this site to help you plan and pay for college as well as manage your money with confidence.

America's College Promise Previous Article Blog Home Final Gainful Employment Regulations Next Article
Could You Be Saving More?
Subscribe to Edvisors' Student LIFE Newsletter to receive up-to-date financial aid information and advice, explore scholarship opportunities, loan repayment options, and learn expert strategies on how to plan and pay for college.
Newsletter Form
By clicking "Subscribe" I have read and agree to the Edvisors' Terms of Use and Privacy Policy. We strongly encourage you to read our Privacy Policy to understand how we use and share your information.
Thank you for subscribing to our newsletter.
We care about your privacy

This website uses cookies to offer you the most relevant marketing and targeted advertisements.  These cookies can collect information to personalize your experience and in some states are considered a sale of your personal information. You may exercise your right to opt out by clicking on deny cookies.  If you opt out, we or our partners will not be able to provide you with personalized ads and we will not provide your personal information to any third parties.   Please accept cookies for the optimal experience.  You can withdraw or modify your consent for non-essential cookies at any time by visiting Cookie Preference page.

Ask the Edvisor
Blog
Contact Us
Edvisors in the News
Link to Us
Newsletter
Partners
Press
Student Aid FAQ
Testimonials
Tools and Calculators
About Edvisors
Video Resources
Financial Terms Glossary
Accessibility Statement
Sitemap
Disclaimer
Notice of Collection
Interest-Based Ads
Cookie Settings
Privacy
Terms of Use
Your Opt-out Rights
Your Privacy Choices Privacy Options

Better Business Bureau logo

Edvisors provides expert advice on planning and paying for college. On Edvisors.com easily compare student loan lenders, learn how to apply for financial aid, and discover scholarships. Learn about federal and private student loans for students and parents, how and when to apply to college, and more!

Edvisors (“Edvisors Network, Inc.”) provides independent advertising-supported platforms for consumers to search compare and apply for private student loans. Loan offers from participating lenders that appear on our websites are not affiliated with any college and/or universities, and there are no colleges and/or universities which endorse Edvisors’ products or services. Lender search results do not constitute an official college preferred lender list. Edvisors receives compensation from lenders that appear on this site. This compensation may impact the placement of where lenders appear on this site, for example, the order in which the lenders appear when included in a list. Not all lenders participate in our sites and lenders that do participate may not offer loans to every school.

Edvisors is not a lender and makes no representations or warranties about your eligibility for a particular loan or financial aid. Lenders are solely responsible for any and all credit decisions, loan approval and rates, terms and other costs of the loan offered and may vary based upon the lender you select. Please check with your school or lender directly for information related to your personal eligibility.

Edvisors has endeavored to provide accurate information. However, the results provided by lenders are for illustrative purposes only and accuracy is not guaranteed, as such, Edvisors assumes no responsibility for errors or omission in the information provided.

Copyright © 1998-2023 by Edvisors Network, Inc. All rights reserved.

All other trademarks and service marks displayed on Edvisors Network, Inc. websites are the property of their respective owners.

Edvisors Network, Inc. 350 S. Rampart Blvd, Suite 200, Las Vegas, NV 89145

Copyright © 2023 by Edvisors.com. All rights reserved.

×
You Got In - Now How Do You Pay for It?

Get the money you need from leading lenders

1 Pick Your College
2 View Your Options
3 Click to Apply
Exit Modal

PrivateStudentLoans.com recommends you consider all financial aid alternatives including grants, scholarships and federal loans (Federal Stafford, Federal Parent PLUS, Federal Grad PLUS) prior to applying for private student loans.