Research faculty invested in economics of students success

Chanit’a Holmes really cares about her students. She cares so much that her research focuses on how to best help them succeed in college.

“The U.S. has a graduation problem, meaning 40 percent of students don’t earn a degree after six years of being in a four-year institution,” said Holmes, assistant professor of agricultural and applied economics in the College of Agriculture and Life Sciences. “The disparity is even worse when we go by racial groups – only 60 percent of Hispanic students and 50 percent of Black students graduate compared to 75 to 80 percent of white students.” 

Holmes recently completed two studies exploring the success weekly electronic feedback and support might have on students’ academic success. One study utilized personalized emails, while the other looked at one and two-way texts. Both produced similar results - an initial uptick in performance followed by a steady decline.

“What I have now recognized going forward is this feedback might be important, but I have to consider the frequency of how to send them, as well as the timing of them,” Holmes said.

According to the National Center for Education Statistics, the graduation rate at public universities or colleges was 63 percent within six years in 2020. Also in 2020, the retention rate, the number of students who returned to the same institution after their first year, was 82 percent for public four-year colleges or universities. 

In contrast, 92 percent of Hokies return to Virginia Tech after their first year and the university has an undergraduate time-to-degree average of just under four years. And last fall, the university announced the Virginia Tech Advantage, which, among other aspects, evaluates time to graduation, student retention, and graduation rates as a part of increasing accessibility and affordability to higher education.

Holmes’ concern with graduation and retention rates is related to economics.

“Basically my research looks at the economics of discrimination,” Holmes said. “What are the differences between race, gender, underrepresented minorities, and first generation students in terms of educational attainment?”

Statistics show that more non-white students take more than four years to attain degrees, if they attain them at all. The longer a student takes to graduate, the more in debt that student becomes and the harder it is to pay off that larger debt accumulation, according to Holmes. 

In the pilot study Holmes conducted with her intermediate micro economics class in fall 2022, she gave personalized feedback in emails to a portion of her students. She then gauged their performance compared to the students who did not receive the feedback. 

The messages included letting them know how they performed on specific assignments as well as how they performed compared to others in the class and words of encouragement such as “You are doing a good job,” or “I am proud of you.” If the student was doing poorly, Holmes would add comments such as “I understand you may be concerned, stop by during my office hours,” or “These are tips that you can use to help you on your next assignment.”

Initially, the results were as Holmes hoped. 

“After the first feedback the students got, they did very well in the following assignment compared to the students who did not receive an email with any feedback at all,” Holmes said. “That positive effect started to decline as we proceeded throughout the semester.”

Post-course interviews with the students revealed the text messages did not affect the students as intended: It led them to put in less effort in the class rather than more. However, when Holmes reviewed the students’ grades for all classes during the semester, she noticed that the students receiving the feedback outperformed those who did not.

“I learned that they were taking the feedback they were receiving about their performance in the class in relation to the other students and reallocating their efforts to classes in which they were performing poorly,” Holmes said.

Similar results occurred during the second study, which Holmes is currently finishing up alongside Susan Chen and Catherine Larochelle, both associate professors in the agriculture and applied economics department, and Nicole Pitterson, an assistant professor in the engineering education department in the College of Engineering.

During the fall semester 2022, the research team studied the academic performance of approximately 1,500 students in an entry-level course for economics and an entry-level course for agriculture and applied economics. The students were divided into three groups: one group received no text message feedback at all, one group received one-way texts giving feedback on the students’ academic work, and the third group received messages from peer coaches that allowed for a two-way interaction through texting only. The team received a 2021-22 Institute for Society, Culture, and Environment Scholars award for the project. 

About 35 student peer coaches from diverse backgrounds were trained by Christina Fabrey, director of the Student Success Center, who provides training through the university on coaching in peer education programs. Each coach had also previously taken the course and was responsible for about five students. 

“The idea was to try to get the conversation started,” Holmes said. “Each week, the coaches would send a reminder of the next assignment to the students or just check in with them.”

The preliminary results were similar to the pilot study. Initially, the coached students did better than all the other groups, but then their performance declined. 

In exit interviews, the coached students said they felt the texts were more automated than personalized, and they were overwhelmed with the pressure to do well from the texts. Some coached students even stopped replying to their coaches, but only about 5 percent actually dropped out of the study.

“The students were free to unsubscribe at any time during the course, despite not being engaged with the coaches, but they did not drop out, and they stayed in the class as well,” Holmes said. “They liked having the information from the text messages, but they did not want the engagement from the coaches.”

Overall both groups of students who received text feedback, either bilaterally or laterally,  performed better than the group that received no feedback texts, but there was no difference between the two text receiving groups. 

Holmes said the team is still working to unpack the latter study’s full findings. 

“We are now in the process of fully analyzing the data, and we had to get additional information from the registrar’s office,” Holmes said. “Are these underrepresented students? Are these first-generation students? Further results will look at if there is a difference between genders,  first generation, or under-represented students.”

She hopes that will provide more clues as to what best helps students’ succeed.

“Going forward, I am going to try to figure out what is the best intervention to help the students,” Holmes said, because she really does care.

Note: A previous version of this article contain a misspelling of Christina Fabrey's name.