A Northwestern University research project to analyze Evanston school data to identify indictors for student success reported its progress this week at a joint meeting of the District 65 and ETHS school boards.
The schools partnered with David Figlio, dean of the school of education and social policy at Northwestern, to collect and analyze school datato identify the factors that contribute to post-secondary success and develop a model that identifies on-track indicators from middle school through high school.
The partnership, called Northwestern Evanston Education Research Alliance, or NEERA, was established last fall, with a goal to “bring teaching, research and service goals of Northwestern University in a mission to be of service to our home towns,” said Figlio.
Both superintendents – Paul Goren of District 65 and Eric Witherspoon of ETHS – cautioned that the project is a work in progress. “It will take time to build the infrastructure,” said Goren.
Figlio said that because of funding from the Lewis-Sebring Family Foundation and Spencer Foundation, “we’ve been able to do in one year what it would have taken five years” to do without the grants, including hiring staff, transferring data securely, and aligning data across the districts,” as outlined in the project report.
While initially working in collaboration with the Evanston schools, Figlio added, “We aspire to work with other leading organizations such as the McGaw YMCA.”
“This project is the first of its kind in the country,” Figlio said.
The research goals were determined by the administrations of the two school systems.
Because ETHS has a relatively high graduation rate, Figlio said, at least compared to the Chicago public schools, NU’s other “home town” and to the statewide average, the research team identified success as “college persistence,” defined as completion of five consecutive semesters of college or completion of a two-year or certificate program.
“We will look at key points in a student’s progress through the school system,” explained Lila Goldstein, NEERA research data analyst lead, to determine if there are “factors related to their experience in school that can predict post-secondary success.”
The project started with eighth and ninth graders to see what indicators predict post-secondary outcomes and will then fill in other grades, Goldstein said. The next question is what middle school indicators can predict high school success.
Chart from NEERA analysis.
The first analysis looked at ninth grade data for students in the ETHS graduating classes of 2012-14, with the goal of identifying actionable, trackable factors that correlate with college persistence.
The data shows that 85 percent of ninth graders for the class years 2012-14 graduated and 85 percent of those enrolled in college. Only 64 percent of those who enrolled in college completed five consecutive semesters or a 2-year certificate or associates degree.
Chart from NEERA report.
The data also documented racial disparities in graduation, college enrollment and persistence.
The project analyzes factors that are considered both trackable and actionable to predict outcomes and address students’ needs early.
- grades
- progression through school
- course-taking pattern
- standardized test scores
- attendance
- disciplinary incidents
In addition, the project measures other factors, including race, sex, family, birth country, and home language to ensure that the predictors work for all students.
Goldstein noted that different measures have different levels of predictive power, such as getting an F or not getting an F in 9th grade, or the interaction of different factors, such as absences and grades in English classes.
However, she emphasized, the formula does not tell us how to address a student’s needs and is not deterministic. In addition, she said, “we cannot measure everything that impacts student outcomes.”
Moving forward, Goldstein said, they’ll add grades 10-12, begin to predict high school readiness from grades 6-8, predict other college outcomes, and identify thresholds for flagging students for support.
The research team anticipates that the data analysis will be able to address students’ needs early, inform instruction and practice, provide feedback for program improvement and monitor high level trends.
“I’m grateful for what you’re doing,” said Jonathan Baum, ETHS board member, “but also frustrated,” with the seeming lack of progress on goals, such as the joint literacy goal that was established five years ago. “You said we don’t have all the data yet. What’s missing?”
FIglio responded that the data collection infrastructure has only been in place since February and they needed to harmonize the data between the two districts and sign data sharing agreements.
Based on the work done so far, “Northwestern, the donors and Evanston taxpayers are getting a great deal,” he said.
“Some of the data has never been cleaned before,” he added, explaining that they’ve tightened up some variables and added years of national clearinghouse data, which provides post-secondary education data. “It may take longer to develop formulas for some predictors.”
“What can be done faster,” asked Baum. “Are there combinations of variables that are predictive of what Evanston cares about? Are some years more important than others? Are you looking at that?”
Figlio responded that Northwestern researchers wanted to look at everything to find breakthroughs.
Jude Laude, another ETHS board member noted that there are other factors that affect college persistence.
“What does a successful college student look like? What are the non-academic factors when a student gets on campus, such as availability of academic support, culture shock, resiliency and financial aid? What are the attributes of colleges and university where students succeed?”
Figlio responded that they start from what is administratively available. “Although there may be other factors besides the on-track indicator, we need to collect the data systematically.”
Joseph Hailpern, D65 board member, expressed his frustration that they don’t seem to know the effectiveness of various interventions in early grades, such as what approaches work to help kids who aren’t reading at grade level.
“There seem to be lots of metrics to measure,” he said. “Can we or will you be able to measure success at the early childhood level?”
Figlio responded that there will be a cascading system of on-track indicators that will be synchronized from early childhood through high school.
Sergio Hernandez, District 65 board member, noted that several factors, such as attendance and disciplinary incidents, can be backtracked to early childhood.
“That’s way past reading and math scores.” he said. “If attendance matters starting at pre-school, we can provide supports to get them in school.”
Hailpern asked how this data can be used. “Our faculty K through 5 needs to actively use these factors.”
“We need to know who requires support,” he added, but we also need to know what supports to provide and their effectiveness. “There’s data in the system, but we need to actively identify the experiences our kids are having — who is getting what support — and track that as part of the student profile.”
“It’ll take ten years” to see an effect,” he said.
“You’re way too pessimistic,” Figlio said. “It won’t take ten years. If the data is in the system, it’s just elbow grease to clean it up.”
“We can’t stop doing what we know to do,” said Monique Parsons, ETHS board member who is also chief operating officer at McGaw YMCA. “We’re not waiting for Northwestern. Whatever we get we will use at the agencies we’re responsible for.”
High School teachers of other times/countries
A frequent discussion is on schools, students and sometimes teachers.
I think about the biographies of mathematicians from ‘Remarkable Mathematicians: Euler too Von Neumann’ and ‘The Honors Class: Hilbert’s Problems and Their Solvers.’
While in the U.S. those who teach K-12 mathematics [it seems] are also required or expected to coach sports. And with rare exception are not recognized. These two books describe how some of the greatest mathematician of the last 300 years [for at least a time] taught in the equivalent of the U.S. high schools. Many of them had a PhD and a second research doctorate in mathematics [and no ‘College of Education’ degrees or classes]. Many of these were German, French and Russian ‘high schools.’ Given that the education system in these countries were different than the U.S. and [probably] fewer students attended the high school level classes.
Still it demonstrates the qualification of teachers that taught ‘high school.’
In the U.S. students, teachers and social activists complain about the in-equity and how minority or poor students don’t have a fighting chance. In German, French and Russian schools, being a woman, Jewish or even from the wrong town or poor would doom you—yet there are obstacles but these books show how people could REALLY try.
Somewhat related is how, especially German, scientists and mathematicians [some of the greats] who came to the U.S. before or during WWII had to, at least start, in what we would call second level colleges or even liberal arts schools—granted there were [almost always unsupported] suspicions of ties to the Nazi requiems.
K-12 in other countries
This link is to the German K-12 system, which may interest some. https://www.howtogermany.com/pages/germanschools.html
I know a number of people who have came from Russian and their children started Middle School in the U.S. and some in Evanston. Generally the parents [many PhD or Dr. Sci] say their children are bored in the American schools, at least with Middle School [including Evanston]. Fortunately many of them find the high school more interesting.
Years ago a Cambridge professor move to Evanston. His son was so bored with New Trier that he spent most of his time working at a gas station until he could go to college. Since then I’ve not been able to find a well educated parent who moved to the U.S. and children started Middle or high school here to see how the parents or children find the comparison. In particular where it seems the post-high school specialization [esp. Cambridge and Oxford] seems to assume they got most of the ‘survey’ and broad education U.S. colleges have, in their English high school.
Schools get early report on student success
Are four years of college the only measure of success. No plumbers or electricians, no carpenters? Truck drivers make a respectable living as do auto mechanics. Not every high school graduate belongs in a four year program. Is the purpose of this exercise simply to encourage post secondary enrolement? Who will fix my air conditioner? Some guy or gal trying to pay off two hundred thousand in student loans.