Sociology Professor Discusses Measuring Higher Ed Outcomes at Lecture

Assistant Professor of Sociology and Social Studies Christina Ciocca Eller presented research on how colleges and universities can find better performance measures in a lecture on inequality in higher education Thursday evening.

In the study she presented, Ciocca Eller used year-long interviews with students from urban, public college systems and university administrative data to compare graduation rates and overall success of students from different racial groups across schools. Using this data, Ciocca Eller said she hopes to provide more clear measurements of college effectiveness than the current ranking system.

Ciocca Eller identified two major problems with colleges and universities’ current performance metrics. First, she said student outcomes reported by universities are collected by organizations like the National Center for Education Statistics and U.S. News and World Report using aggregate data.

“I like to call these ice cream sundae measures,” Ciocca Eller said. “We really don't know if it's students pre-college characteristics, the selection process between the student and the college, or the college itself that is exerting this aggregate effect that we see most likely to be reported.”

She added that administrators’ awareness of potential issues in student outcome reports creates another problem. As administrators realize the potentially negative impact of reports on potential applicants, they may cater organizational changes toward improving reports rather than actual outcomes. She calls this phenomenon “superficial recoupling.”

“What you get here is a similar kind of pressure being put on colleges and universities to give over outcome data — to look very closely at it and say, ‘what can I send to the world to try to make myself look a certain way on paper?’” Ciocca Eller said.

Ciocca Eller said she focused on urban college systems because they frequently have a large population of underrepresented minority students and are often overlooked in college rankings.

“One, they're just huge. On an annual basis, if we were to combine the public college populations of New York City, Chicago, Los Angeles, and Miami, it's over 600,000 students served a year,” Ciocca Eller said. “Second of all, it serves a very high proportion of students who are considered underrepresented in higher ed, both by their income and by their racial background.”

Ciocca Eller said she thinks more researchers should focus on how these universities can improve.

“We really have an opportunity to think about what is not going as well as it could be in better you in order to produce even greater utilization,” she said.

In order to improve student success, Ciocca Eller said universities should implement methods of measuring student outcome that keep administrators accountable.

Ciocca Eller also proposed a new model which she said allows her to statistically match students from different universities and track academic success across schools in order to isolate how university policies affect student outcomes across racial and social groups.

“I’m using this kind of statistical matching or equalization in order to try to remove some of the bias that we would get from the selection process,” she said.

Graduate School of Education Professor Julie A. Reuben, who organized the talk, said she considers Ciocca Eller’s research vital to providing a more precise understanding of how well universities perform.

“I think what this research does is it begins to find ways of identifying whether institutions are serving their students — well, helping them achieve this goal of graduating — and breaking that down so we could see it by different groups of students and also by the different preparation levels that the students come in with,” Reuben said.

Correction: Nov. 23, 2019

A previous version of this article misspelled Assistant Professor of Sociology and Social Studies Christina Ciocca Eller's name.