What happens when AI writes software code? Researcher will use CAREER award to find out
Muhammad Ali Gulzar, assistant professor of computer science, is studying how to keep artificial intelligence-generated software understandable, secure, and safe.
What happens when people no longer fully understand the software artificial intelligence (AI) creates?
That question is at the center of new research led by Muhammad Ali Gulzar, assistant professor in the Department of Computer Science, who received a National Science Foundation (NSF) Faculty Early Career Development Program (CAREER) award to pursue it.
“AI has made software creation dramatically easier,” Gulzar said. “But the long-term challenge is that humans may no longer fully understand the code being produced, which creates serious issues when something breaks or needs to be updated.”
His lab is working to make AI-assisted software development more reliable, maintainable, and sustainable.
“The NSF CAREER award is one of the most prestigious investments our field makes in early career researchers, and Gulzar's focus on a question that matters enormously has earned him that support,” said Christine Julien, head of computer science.
“As AI makes writing code easier, the harder problem becomes maintaining it,” Julien said. “By studying and strengthening AI-based code understanding, Gulzar’s work is addressing a gap that will only grow more consequential as our reliance on AI-generated code deepens.”
Bringing human work back into AI
Modern AI coding systems can generate thousands of lines of code in seconds, accelerating productivity for businesses, researchers, and everyday users. But Gulzar said the rapid rise of AI-generated software also introduces new risks when developers must later diagnose errors, patch vulnerabilities, or adapt systems to new needs.
Traditionally, programmers build software line by line, developing a detailed understanding of how systems work. With AI-generated code, that understanding can disappear.
“The maintenance will get more and more difficult as we generate billions of lines of code through AI,” Gulzar said.
His research focuses on identifying hidden weaknesses that emerge when large language models are used to create or repair software. Rather than removing people from the process entirely, Gulzar’s work aims to involve human expertise strategically when AI encounters gaps in reasoning or understanding of software.
The project proposes new evaluation frameworks that can identify where AI systems struggle during software maintenance tasks and determine what targeted human insight is needed to resolve those problems efficiently.
Gulzar said the work is especially important as society grows increasingly dependent on AI-generated systems in high-stakes settings such as healthcare, transportation, infrastructure, and cybersecurity.
“We’re interested in the rare cases where a small mistake can create a huge impact,” he said.
AI needs human insight
The CAREER award also supports delving into broader questions about the future of artificial intelligence itself. Gulzar said overreliance on AI-generated content could eventually limit innovation if models increasingly learn only from material created by other AI systems.
“So much code is now generated by AI,” Gulzar said. “Eventually, there may not be enough genuinely new human knowledge for AI systems to learn from.”
In addition to advancing foundational AI research, the award will support graduate student training and mentorship at Virginia Tech.
The CAREER Program is among the foundation’s most competitive honors for early career faculty members and recognizes researchers with the potential to lead advances in both research and education.
“This is a valuable resource and recognition that can advance my research group’s scientific ambitions,” Gulzar said. “I’m excited about taking that research forward, funding students, and mentoring.”
