Spanking The Rankings (Cont.)

I wrote a few weeks ago about the revolt of certain law schools–including my alma mater, the Georgetown University Law Center–against the U.S. News and World Report rankings. The revolt got some results. Earlier this month, U.S. News and World Report announced that it would be modifying its approach to rankings, and wrote an open letter to law school deans announcing the changes. You can see the letter here.

U.S. News says the changes are in response to the issues raised by the law schools, but it is also standing its ground in its position that its rankings are appropriate and a helpful data point for students deciding on where to go for their legal education. The letter to the law school deans says that the ranking algorithm will be modified, stating that “there will be some changes in how we weight certain data points, including a reduced emphasis on the peer assessment surveys of academics, lawyers and judges, and an increased weight on outcome measures.” At the same time, U.S. News let the law schools know that it is going to rank them, whether they participate or not, and offered an incentive of sorts: “We will rank law schools in the upcoming rankings using publicly available data that law schools annually make available as required by the American Bar Association whether or not schools respond to our annual survey. For schools that do respond, we will publish more detailed profiles, enabling students to create a more comprehensive picture of their various choices.”

Law schools don’t seem to be blown away by the U.S. News announcement. Yale’s dean was quoted as saying: “having a window into the operations and decision-making process at U.S. News in recent weeks has only cemented our decision to stop participating in the rankings.” Michigan says its decision to not participate in the rankings won’t change, either.

I’m not surprised that U.S. News is trying to put out the fire; its ranking publications are no doubt a big money-maker in an era where fewer and fewer people are buying magazines. But the tweaks to its approach also shows that rankings are really kind of silly, and the metrics in its formula, and their weighting, aren’t some immutable, undeniable way of objectively evaluating law schools. If the metrics and the weighting can change in the blink of an eye because law schools have said enough is enough, why should anyone trust that the new formula is the right mix?

I think law students would be better served by ignoring rankings and thinking about what they want out of law school, using their personal interests and concerns to narrow the field of law school candidates by looking at rational considerations like cost, and then talking to recent graduates. Law school is not a one-size-fits-all proposition, and the revolt against the U.S. News rankings illustrates that fact. That’s a useful lesson.

Spanking The Rankings

An interesting movement is underway among some of America’s most prominent law schools. One by one, they have begun deciding not to participate in the U.S. News and World Report law school rankings. Yale and Harvard led the way earlier this month; my law school alma mater, the Georgetown University Law Center, followed two days later. You can see a list of the law schools that have eschewed the U.S. News rankings here.

Each law school that has withdrawn from the rankings process has released a statement explaining its reasons. You can read the statement of Georgetown Law Dean William Treanor here. In a nutshell, he contends that the rankings reflect priorities that are at variance with the public service orientation of the law school and do not adequately account for efforts to help students attend law school. He writes:

“Rankings formulas that over-emphasize GPA/LSAT scores, that refuse to credit public interest lawyers who are subsidized by school-sponsored fellowships as fully employed, that treat need-based financial aid as a disfavored use of resources, and that penalize schools that admit students who have to borrow to fund their legal educations are not rewarding quality education and are not advancing our profession’s high ideals.”

U.S. News and World Report is pretty transparent about its methodology in compiling the rankings. You can see a description of the methodology here. The publication summarizes its approach as “evaluat[ing] institutions on their successful placement of graduates, faculty resources, academic achievements of entering students, and opinions by law schools, lawyers and judges on overall program quality.” Among the data it analyzes are LSAT and GRE scores, acceptance rates, median undergraduate grade point average, bar exam passage rates, and post-graduation employment rates. The decisions of Georgetown and other law schools to not participate in the rankings presumably means that U.S. News won’t be able to collect at least some of the data it would use to evaluate the withdrawing schools against the same standards applied to participating schools.

So, are the decisions of law schools to withdraw from the rankings a good thing? Of course, academic institutions are free to choose whether to participate or not, and if they feel that the ranking algorithm is fundamentally contrary to their values, as Georgetown does, then withdrawal makes sense. It will leave incoming students with less information–but some have questioned the credibility of the rankings and whether schools have goosed the data they provide to inflate their ultimate ranking. A whistleblower recently raised issues about the accuracy of data supplied by Columbia University, and last year a dean at Temple University was convicted for fraud offenses for falsifying data.

I’ve always had a basic objection to the U.S. News rankings. I think they are both absurd, because an education simply can’t be reduced to a mathematical equation, and distortive, because institutions inevitably end up making changes in hopes of increasing their ranking–even if those changes alter the character of the school and its goals. If the law school withdrawals, coupled with the issues about the validity of the data underlying the rankings, cause the rankings to go the way of the dodo, I think that would be a good thing. Schools should get back to focusing on delivering what they consider to be the best approach to education, and stop the mindless competition to be “top ranked” on some list or another.

The Ceaseless Quest For Rankings

If you want some tangible evidence of how rankings have affected the activities of colleges, universities, and other institutions of higher education, you need look no farther than Philadelphia, Pennsylvania– where a federal court jury recently convicted Moshe Porat, the long-time dean of the Temple University Richard J. Fox School of Business and Management, of mail and wire fraud in connection with a scheme to boost that school’s U.S. News and World Report ranking.

According to a statement released by the U.S. Attorney’s office for Eastern Pennsylvania, Porat, who served as the dean of the business management school from 1996 to 2018, was convicted after the jury found that he had “conspired and schemed to deceive the school’s applicants, students, and donors into believing that the school offered top-ranked business degree programs, so that they would pay tuition and make donations to Temple.” The statement explains that Porat and two other conspirators “agreed to provide false information to U.S. News about the number of Fox’s [on-line MBA (“OMBA”) and part-time MBA (“PMBA”)] students who had taken the Graduate Management Admission Test (“GMAT”); the average work experience of Fox’s PMBA students; and the percentage of Fox students who were enrolled part-time, all because it was believed that better numbers for these metrics would result in better rankings for the programs.”

The scheme to goose the school’s rankings evidently worked, too. The U.S. Attorney statement explains: “Relying on the false information it had received from Fox, U.S. News ranked Fox’s OMBA program Number One in the country four years in a row (2015 – 2018). U.S. News also moved Fox’s PMBA program up its rankings from No. 53 in 2014 to No. 20 in 2015, to No. 16 in 2016, and to No. 7 in 2017.” Porat then touted the rankings in “marketing materials directed at potential Fox students and donors,” and “[e]nrollment in Fox’s OMBA and PMBA programs grew dramatically in a few short years, which led to millions of dollars a year in increased tuition revenues.”

The “rankings” established by publications like U.S. News and World Report have had a profound–and in my view, negative–impact on the world of higher education. Parents and students use them to help in making application decisions, and schools reorient their admissions standards and processes and make other important decisions in an endless quest to better their rankings. The notion that you can boil down the whole college experience, or a law school education, to a ranking based on metrics is absurd on its face, but the rankings give schools something to boast about, or goals to achieve. Never mind the distorting and pernicious effect the zeal for higher rankings might have on a school’s educational mission–or the fact that the rankings have become such a dominant force that they caused one school administrator to apparently engaged in fraudulent conduct.

We’re past the point where our kids are making school decisions, but this incident really makes you wonder how meaningful those rankings really are.

The Latest Government Rating System

The federal government has announced that it plans to “rate” America’s colleges and universities.  The New York Times story linked above describes the ratings push as “a radical new effort by the federal government to hold America’s 7,000 colleges and universities accountable by injecting the executive branch into the business of helping prospective students weigh collegiate pros and cons.”

The underlying concept is that colleges and universities receive $150 billion in federal loans and grants, so the federal government should determine whether the schools are “worth it.”  The proposed rating system would apparently be based on how many students graduate, how much debt they accumulate during their college years, and how much money they make after they graduate, among other factors. The federal rating would compete with the college guides, like that produced by U.S. News and World Report, that are all too familiar to the parents of a college-bound high school student.

College administrators are reacting with horror to the idea.  In some respects, it’s hard not to feel a certain schadenfreude when you read their outright dismissal of the idea.  For years our institutions of higher learning have been relentlessly raising their tuition and fees and administrator salaries, blithely rejecting thousands of applicants, and happily operating in their own, comfortable sphere of almost complete autonomy.  Now they’re the ones who will be judged, and they don’t like it.

But the college presidents have a point.  One Department of Education official said the rating system would be a cinch, like “rating a blender.”  Sounds like the same unfounded  bureaucratic arrogance that led to the disastrous roll-out of the healthcare.gov website, doesn’t it?  And speaking as someone who went to a land-grant school for college and a private school for a law degree, and was the parent of children who have gone to small and medium-sized private schools and a state school for college and master’s programs, I have zero confidence in the judgment of anyone who thinks that rating schools is even remotely comparable to rating an appliance.  There are far too many variables and differences, and focusing on financial issues — like how much graduates are paid — inevitably gives short shrift to the idea of getting a well-rounded liberal arts education.

More fundamentally, I am royally tired of the federal government injecting itself into every facet of American life.  The process is always the same — first the government provides money, then it says it needs to establish oversight to ensure that the money is being spent wisely. (Of course, there’s never any reconsideration of the idea of the federal government spending the money in the first place.)  We know from years of experience that if the Bureau of Federal Higher Education Rating is created, it will immediately become another calcified government program that can never be cut or terminated.

We don’t need President Obama or the federal bureaucracy dreaming up new ways to regulate and new administrative positions that need to be filled, we need them to focus on doing a better job of running the programs that already exist and figuring out how to run them more efficiently — and determining whether they are truly needed at all.  I’d give the notion of establishing a federal college rating system an “F.”

When Law Schools Lie

Recently the University of Illinois College of Law announced that Law School Aptitude Test (LSAT) and grade point average (GPA) statistics provided on the school’s website were inflated.  The school posted the correct numbers, said it was conducting a thorough review of how the error happened, and placed the dean of admissions on leave as part of the investigation process.

The incident is part of a broader trend of concern about the credibility of law school admissions statistics at the beginning of the law school experience (the LSAT scores and GPAs of the incoming classes) and the placement statistics at the end of the process (how many graduates get law-related jobs).  The LSAT and GPA statistics are significant to the law schools’ ranking by the U.S. News and World Report, which uses those numbers to measure “selectivity.”  Every college and graduate school student, parent, professor, and administrator knows that the U.S. News and World Report ranking carries a huge amount of weight.  With law school admissions dropping — and they fell by 10 percent in 2011 — schools competing for the reduced pool of applicants may be sorely tempted to cook their  admissions and job placement figures.  Interestingly, plaintiffs’ lawyers have noted the issue and are pondering whether inaccurate reporting should be met by a class action lawsuit on behalf of students.

Institutions of higher learning used to presume to occupy the moral high ground.  More and more, however, those institutions behave like businesses and are facing the same kinds of scandals we see in the business world.  What do such scandals mean for a school’s ability to achieve its educational mission?  How is a law school that admits to falsifying data supposed to enforce an honor code, or credibly instruct students about legal ethics?