Lies, Damn Lies and Statistic(al ranking)s (how much, school rankings)

[GoPadge]

I was following up on the source of the data for this thread //www.city-data.com/forum/dalla...vers-were.html and was lead to Prince Market Research's website (www.pmresearch.com). I couldn't find the source data I was looking for, but I did find this blogpost that made me pause and reflect on my love of / affinity for / addiction to statistical data. While I think I do a fair job of trying to look beyond just the numbers and finding the underlying context, I know it's a area where I could be more diligent. My purpose here is to provide positive and accurate information on the Metroplex and not to get into petty territorial arguments. (Sometimes I backslide... ) My point being, I think it's important for us to not just quote statistics, but to look beyond them for the context and see where we have been and where we are potentially going. One thing is for certain, we cannot change where we will go, if we cannot see the direction we are heading.

Hot or Not, the Higher Ed Edition
08.22.2006

Is your school hot or not? The US News & World Report ratings are out, and no doubt this edition - as in previous years - will be a big seller. But how much to rankings really matter?

As market researchers, many of [us] have been encouraged by our clients to develop a single number that tells them how well they are doing in satisfying their customers - a Customer Satisfaction Index. While this can be useful, it's important to have context. Sometimes with clients we can get them to understand context, to do drill downs to better understand what that index number really means and why it increased or decreased compared to last year. Sometimes we can't, and the index number is left to stand on it's own.

US News explains for their rankings that "the rankings condense a great deal of information about the quality of the education at each school, making it easier to compare institutions and select the best one for an individual." But does making it easier necessarily result in a better decision? Certainly it's an easy shortcut, but here's the inherent problem/conundrum in this: If a piece of information or a statistic is important enough to be included in the US News' (or any other mag's) methodology, then why "condense" it? If you believe that the items/data included in any mag's school rankings are truly important, then why substitute your judgment on weighting of those factors for theirs?

All this to say that some decisions, like where to go college/law school/business school, should be complicated. To those that "condense" that complicated decision making model and use the rankings as gospel, I say caveat emptor.

From http://www.pmresearch.com/blog.php?id=4