Tamler Sommers and Michael Twa
“When a measure becomes a target, it ceases to be a good measure.” Goodhart’s Law
Recently, a scandal rocked higher education, as multiple universities' efforts to increase their rejection rates in order to move up in the US News college rankings board came to light. Whose fault was this? Was it US News for using rejection rates as a metric for “good” education? Was it the fault of college administrators trying to game the system? Over the last two decades, there has been an increasing obsession with metrics in domains as diverse as education (e.g. standardized testing), criminal justice, social science research, medicine, and management. What are the cost, benefits, and hidden assumptions behind this new emphasis? Do metrics offer more accurate, less biased information or merely a veneer of objectivity that crowds out human judgment and discretion?
Have you ever looked at a dataset and thought, “how did they come up with these numbers?” Have you ever wondered how a business can claim to be ranked #1 in customer satisfaction? How is satisfaction measured, anyhow? At this table we will discuss all of these problems and many more.