kirill_makarov - stock.adobe.com

Feature

Student engagement data helps educators with remote learning

Distance learning is necessary for most institutions during the COVID-19 pandemic. See how institutions are using analytics to measure their success in the shift.

Kara E. Joyce
By
Published: 04 Jun 2020

In March, COVID-19 caused educational institutions around the world to rapidly switch to online learning. For many, this was a new frontier.

Even with the rise of online higher education options, most post-secondary education is still taught in a classroom, especially at the undergraduate level, forcing many colleges and universities to make major changes to how they structure classes. Primary and secondary educational institutions had to make the change as well, most without the experience that many colleges and universities have from online and hybrid programs.

Education providers had to rapidly adapt to online learning initiatives, leaving many to wonder how to engage students when they are not in a classroom setting. But how do you measure success and engagement with distance learning? The answer lies in analytics of student engagement data.

Analytics in higher education

Miami University is a public university located in Oxford, Ohio, founded in 1809, and it functions as a residential college, meaning most programs -- with the exception of some graduate degrees -- are taught completely on campus.

Within about three hours, according to Jeffrey Toaddy, the senior business process analyst at Miami University, the university had moved to 100% online offerings. The university used Tableau to report collected data to the provost and coordinators for each division.

One of the first challenges they had to address was accreditation to be sure their online offerings met educational standards.

"The real dig into the data that we did here was to make sure that we maintain these accreditation standards," Toaddy said. "And we are trying to get an experience that is the same, class to class, for the students."

To monitor how students handled the changes, the university launched a data collecting initiative. A number of faculty and staff volunteered to call all of the university's 20,000 students to collect information on how the situation was affecting them individually.

"What we're really doing is looking at the [learning management system], but right now it's really just looking at -- is the student needing something right now," said Sarah Matthews, data analyst at Miami University.

The main focus at the start was individual students, Toaddy said, with a focus later on how to address the overall student body.

According to Mollie Miller, associate director for institutional research and effectiveness at Miami University, sharing the data from students led to faculty working toward a benchmark.

Miller said Miami University had adapted previous analytics measures just by sharing current reports with university deans and chairs. "This is something that they've never really had a view of at this level," she said.

The university  pulled student engagement data from Canvas, a learning management system (LMS) where educators can post assessments, assignments, syllabi and other course material for students to access and complete. However, measuring student engagement data from the learning platform was difficult at times when it came to certain majors, especially students with creative-focused majors, according to Miller, as many students were already pursuing independent studies or required less structured assessment.

When it comes to moving courses back on campus, it's all about maximizing the student engagement data collected in the past few months, according to Miller. The vast data collected by the university was shared with data analysts for each division to seek improvement. This included looking at how well faculty used the LMS when it came to assigning material or how quickly they were able to turn around grades.

Once campus life has returned to the new normal, though, what happens to these student engagement data analytics? According to Toaddy, that could include using push notifications in Canvas to make sure faculty match university baselines and give them a chance to see data on how strong engagement was throughout the course.

The university also compiled reports and visualizations for benchmarks created by the university senate that they shared with faculty so they could see how they compare to those standards created for distance learning.

"I can see this being used to see if students are being engaged in enrollment and student success," Matthews said.

While being sure to maintain student privacy in the data, Matthews said, use of these educational analytics once back on campus could measure student engagement in a way that would benefit the university's student success measures, which help students stay on track to graduate.

Analytics measures for K-12

When it comes to primary and secondary education, analytics in education started with a similar student-first focus for Loudoun County Public Schools in Virginia.

Early in the process, Loudoun County Public Schools used a Qlik dashboard to compile data from families in the school district and supplied Wi-Fi hotspots to families who didn't have access to internet so those students could still benefit from the necessary distance learning.

"We had a fairly good grasp of using analytics to drive our operations [before COVID-19], but there were pockets," said Rachel Johnson, the director of enterprise solutions at Loudoun Public Schools. "This event really exploded into other operations and other dimensions of using analytics."

Before the pandemic analytics in education for Loudoun County focused internally on teacher licensure, student achievement, student assessment and enrollment data. The county was also compiling data regarding student equity to identify students with advancement opportunities.

"Now we're heavily focused on emergency meals served and [internet] hotspots," Johnson said.

One main area for analytics in education is student grades, but how do you assess students' grades when there is no school? According to Johnson, the county school board gave parents and students three options to evaluate final grades. Using a Qlik dashboard, they developed a platform that would divide the final grades based on courses students completed before schools closed due to lockdowns.

"We've had to be pretty creative in how we're leveraging these analytics tools so that we can give families and school counselors and teachers the opportunity to see the data in a different way," Johnson said.

The county's data was pulled in from several sources, from LMS providers to school-centered social media platforms, in an effort to understand student engagement in the current situation. According to Johnson, correlating that engagement has been difficult.

"We don't know yet what successful engagement for distance learning will look like," she said. "How do you determine engagement when there's no assessment to determine growth?"

One specific issue especially at the K-12 education level is ensuring compliance with Individual Education Programs (IEP), which are necessary for students in special education programs. According to Johnson, the question ends up being whether the student's level of engagement is enough to meet IEP requirements.

"We are using the data dashboards and the analytics tools to identify those students because it's very challenging to just generate a report out of the student information system and make that actionable," Johnson said.

Virtual learning is a new frontier for much of K-12 education. Online primary and secondary education exists, but enrollment is still not as strong as in-person enrollment for K-12 students. But with the pandemic, schools have had to rethink what success looks like with online teaching programs on a large scale and continue to assess how it may work if distance learning is still necessary come the fall.

"Quite frankly, one of the interesting things that's come out of this is that we've been able to demonstrate success in delivery of instruction in this virtual world," Johnson said.

She said there hasn't been much time to reflect on all the student engagement data Loudoun County has managed to collect regarding its distance learning program so far due to the situation's continuously evolving nature. But data analytics in education looks to the future and allows for adjustments to cement the process.

"You can't walk back from [distance learning], but what you can do when this is over is assess it and figure out what worked, what didn't work, what needs to be tweaked," Johnson said. "As we continue to grow in this area, [we'll identify] what are some more considerations that we need to prepare for."

Dig Deeper on Business intelligence management