Andrew Katz receives CAREER award to explore the impact of generative AI on educators’ instructional decisions
ChatGPT and Claude are two examples of generative artificial intelligence (AI) tools that are increasingly being incorporated into the classroom.
With a recent survey showing more than 86 percent of college students using generative AI in their studies, these tools have started to raise questions about how educators teach and assess students.
Andrew Katz, assistant professor in the Department of Engineering Education, has received a National Science Foundation Faculty Early Career Development (CAREER) award to better understand how generative AI impacts instructional decisions. His research focuses on assessment, which ranges from exams to lab reports to homework.
Generative AI models "are becoming more capable of doing a lot of things students are typically asked to do,” said Katz. “That has many ramifications for the way we educate engineers. I'm focusing on assessment because it has such a big impact on education and how we evaluate our students’ understanding of key concepts.”
The CAREER program serves to support early career faculty who have the potential to serve as academic role models in research and education and lead advances in the missions of their department or organization. This five-year grant will support Katz’s research on engineering education faculty members’ mental models of generative AI and instructional decisions. He is the 12th CAREER awardee in the department since its creation in 2004.
Opportunities for generative AI
Evolving digital tools bring change to assessment guidelines and how educators approach assessments. This change is creating new ways for students to learn and educators to teach. How faculty members understand generative AI and incorporate it into educational practices such as assessments is still unknown. That's what Katz is seeking to understand.
Katz describes generative AI as “a computer model that can generate something as an output. This can be text, audio, images and video, for now. The output is based in part on the input of what you tell it to create.”
Generative AI is a tool that can be helpful for a lot of students. These tools can act as a tutor, assist with coding, draft written materials, and summarize articles. Properly integrating these features into education is what poses questions for researchers.
Katz’s research will use interviews and annual surveys to understand how faculty integrate generative AI into education. Data will be collected nationally from various engineering disciplines. The goal is to see how faculty are reacting to generative AI in the classroom.
Katz plans to use his research findings to collaborate with the National Effective Teaching Institute to hold educational faculty workshops centered on improving assessment strategies when generative AI is used. Katz assisted in holding a faculty workshop with Virginia Tech’s office of Technology-enhanced Learning and Online Strategies to provide information on generative AI and ways that faculty can implement it in the classroom. By “trying to form communities for practice” the goal is for faculty to share ongoing ideas and experiences with each other.
Diving deeper
ChatGPT, Claude, and other generative AI models use natural language processing to understand what is being asked and then create a response to the request. Natural language processing is a model that deals with how computers understand, process, and manipulate human languages. Katz will use his experience in natural language processing research to continue answering questions about generative AI in engineering education.
One way natural language processing can be used to support students without resources is by acting as a translation device. Natural language processing can translate audio into text using audio files, or during a live lecture. This can save time and make information available in multiple formats, creating a more broad range of resources for students. Natural language processing can also translate text or audio from one language into another. This translation feature can be useful for multilingual students, or students who want to enhance their skills in a new language.
Looking into the future, Katz sees the long-term contribution of his research as gaining fundamental knowledge about learning and technology use.
“We’re aiming to determine how people think and learn about technologies and update their ways of thinking about those technologies,” Katz said. “This fundamental research can be useful when the next technology comes along, so we can anticipate the types of issues that might emerge and prepare for the next technological innovation.”
Related content
Katz has featured this research on WDBJ7 as well as the Curious Conversations podcast.
