What Data Can — and Can’t Yet — Tell Us

When my twin brother, a commercial pilot, flies a plane, he knows sensors informed by tens of thousands of prior flights are equipped to predict engine failures. But that kind of sensor data and machine learning doesn’t exist for human areas, such as student success. Data and predictive analytics can only foretell so much about which students are likeliest to stumble and why, because human behavior is complex and unpredictable.

Posing the question of how colleges and universities can best help students succeed comes at a crossroads for higher education. When I went to college in the 1990s, my classmates and I were expected to succeed. If something extracurricular held us back, professors and administrators expected us to resolve it before arriving, punctually, to class ready to learn. Many thought it was part of the experience to weed out unsuccessful students. Those who made it were thought to do so for their sheer will, grit and skill.

Even before COVID-19, higher education institutions had shifted the pendulum, so students and universities now share the onus of success. We aim to help rather than consider a struggling student’s falloff to be natural selection. To that end, many institutions are investing in new approaches, including professionalized advising, expanded career services and innovations in teaching and learning. They’re modernizing systems, adopting new technologies and looking to data to provide answers about how to more effectively and efficiently support student success.

This work is challenging during the best of times, let alone in a post-pandemic environment. But today's complicated climate has institutions turning to student success analytics more than ever. An Educause Quick Poll conducted in late May found that since the transition to emergency remote teaching, demand for student success analytics has increased at 66 percent of surveyed institutions.

Unfortunately, this is not an easy, on-demand endeavor. When thinking about what data can tell us about student success, it helps to first understand the data landscape and what data exist and do not exist … yet.

Already captured and easily available.

Colleges and universities already collect and can access a variety of data relevant to student success. That includes data related to prior academic work, current academic performance and financial need. While the data have limitations, they are the place to start.

Already captured, but not easily available.

Relevant data live in systems across campus but are more difficult to surface, share and use for student success purposes. Student affairs and student engagement data are good examples. While they are collected, they are harder to access. But by making them available, institutions can add a layer of sophistication to understanding student success. For example, by integrating learning management system and learning data, institutions can understand how students are responding to remote and online learning. Or by merging facilities and course registration data, they can better understand how to schedule classes and be compliant with social distancing guidelines.

Not yet captured and not easily available.

This is where institutions really have a greenfield opportunity to meaningfully impact student success, but the degree of difficulty is high, because institutions do not yet have systems or procedures in place to collect and access this data. Most often, such data are surfaced through intentional interactions, for example, with academic advisers and residential life staff.

This gets at mind-set, concerns, career aspirations and the like. Are students having trouble with housing or their roommates? Are family finances causing additional stress and distracting them from their studies? Are they struggling to adjust to the new realities of life and learning in the era of COVID-19? Was their internship canceled or will they be unable to complete their practicum? All could have a significant impact on student success, but few institutions have mechanisms to know this very human information, let alone act on it. These are blind spots.

By taking this view on data, an action plan emerges. Colleges can begin by focusing on the data that are easily available while understanding their limitations. They can gain greater insights by expanding data sets to include data that can be extracted from all relevant systems. They can start to eliminate blind spots by establishing new systems and procedures to capture more individualized data that the various people involved in student success across the campus can provide.

Once colleges and universities have collected all available data, they can create models and assign weights to understand which are primary versus secondary variables. For example, students’ financial needs for paying for college might be the primary concern when it comes to student success, making their academic performance secondary. Students may have good grades, but if they can’t afford to attend the institution, they will not persist. Along the way, institutions need to include as many data points as possible to mitigate blind spots, and that data can help with a range of actionable analyses, such as financial aid optimization and course sequencing.

Remember the Human Element

The key to all data-driven student success efforts is to remember that students are people first. In my former role as vice president of analytics at the University of Maryland Global Campus, we found that the most effective intervention sent to students was a simple, empathetic message asking them, “How are you?” That open-ended intervention, rather than a prescriptive one, like “We noticed you didn’t read the syllabus before class,” elicited the most positive student responses. It was primarily a human question, which is why it was so well received.

Not only must colleges and universities remember the human element when applying data to support student success, but they must also do so when they capture and assess that data. It is vital to respect student rights and privacy. There’s good cause to be on high alert about improper data collection and analysis these days. Institutions should strive to be the gold standard for responsible and sensitive data use and articulate clearly the purpose of the data being collected and how they are being used to support students.

This is especially important today as institutions grapple with COVID reporting and the need to balance student rights and privacy with public health concerns. If you have to collect data for contact tracking and tracing, be very explicit about the limited use cases for when you will use it, as well as shared responsibilities. Some institutions have begun to ask students to sign pledges that describe guidelines and obligations to fellow students and the university if they get sick. This kind of transparency and joint accountability is key.

We also need to be humble about what data can and cannot do. When wielded correctly, it can help us do just about everything better, but it is not an easy button to press that magically fixes everything.

The final caveat here relates to scale. Even if your initiative is successful, you should expect incremental results and that getting results will take time. You’re not going to make one tweak here and see a 10-percentage-point increase in retention next year. That outcome would be unheard-of, and even the best predictive models don’t perform miracles. As much as we wish for one, there are no quick fixes.

Ultimately, the goal should be getting to where your institution is able to use data from across campus -- including more-difficult-to-capture data -- to better understand students and how to best support their success. But even as colleges and universities strive to build out these capabilities, they can still make an impact right now with data that they can already access. If we wait, we don't just lose time until we see the benefits of data, we lose students. The key is to get started and use whatever data are available to help the students we can today while expanding efforts to help even more tomorrow.

To view the original article, please refer to the following link: https://www.insidehighered.com/views/2020/08/06/due-covid-demand-analytics-has-risen-significantly-information-doesnt-mean-action