Amazon-Virginia Tech Initiative announces support for two Amazon Fellows and five faculty-led projects for 2023-24 academic year

The Amazon–Virginia Tech Initiative for Efficient and Robust Machine Learning will support two Amazon Fellows and five innovative research projects led by Virginia Tech faculty in the 2023-24 academic year that further the initiative’s mission of advancing innovation in machine learning. 

The initiative, launched in 2022, is funded by Amazon, housed in the College of Engineering, and directed by researchers at the Sanghani Center for Artificial Intelligence and Data Analytics on Virginia Tech’s Blacksburg campus and at the Virginia Tech Innovation Campus in Alexandria. 

An open call for fellowship nominations and faculty projects went out across the Virginia Tech campuses. An advisory committee of Virginia Tech faculty and Amazon researchers selected two Amazon Fellows from 27 nominations — more than double what was received last year — and five faculty projects from 17 submitted proposals.

Amazon Fellows

• Minsu Kim, Ph.D. student in the Bradley Department of Electrical and Computer Engineering. His research interests are in resource-efficiency, data privacy/integrity, and machine learning in wireless distributed systems. His current focus is on building green, sustainable, and robust federated learning solutions with tangible benefits for all artificial intelligence (AI)-embedded products that use federated learning and wireless communications. This work requires a more holistic view of the life cycle of federated learning algorithms including data acquisition, algorithm and model design, training, and inference/retraining. Kim is advised by Walid Saad.
• Ying Shen, Ph.D. student in the Department of Computer Science. Her research interests lie in natural language processing and multi-modal, the vibrant multidisciplinary research field that focuses on integrating and modeling multiple communicative modalities, including linguistic, acoustic, and visual messages. She is particularly enthusiastic about building more human-like interactive agents to better understand, interpret, and reason about the world around us. Shen is co-advised by Lifu Huang and Ismini Lourentzou.

Faculty and their projects

• Lifu Huang, assistant professor in the Department of Computer Science and core faculty at the Sanghani Center, “Semi-Parametric Open Domain Conversation Generation and Evaluation with Multi-dimensional Judgements from Instruction Tuning.” The goal of this project is two-fold. First, it will develop an innovative, semi-parametric conversational framework that augments a large parametric conversation generation model with a large collection of information sources so desired knowledge is dynamically retrieved and integrated to the generative model, improving the adaptivity and scalability of the conversational agent toward open domain topics. Secondly, it will simulate fine-grained human judgments on machine-generated responses in multidimensions by leveraging instruction tuning on large-scale pre-trained models. The pseudo human judgments can be used to train a lightweight multidimensional conversation evaluator or provide feedback to conversation generation.
• Ruoxi Jia,  assistant professor in the Bradley Department of Electrical and Computer Engineering and core faculty at the Sanghani Center, “Cutting to the Chase: Strategic Data Acquisition and Pruning for Efficient and Robust Machine Learning.” This project focuses on developing strategic data acquisition and pruning techniques that enhance training efficiency while addressing robustness against suboptimal data quality by creating targeted data acquisition strategies that optimize the collection of the most valuable and informative data for a specific task, designing data pruning methods to eliminate redundant and irrelevant data points, and assessing the impact of these approaches on computational costs, model performance, and robustness. When successfully completed it will optimize the data-for-AI pipeline by accelerating the development of accurate and responsible machine learning models across various applications. 
• Ming Jin, assistant professor in the Bradley Department of Electrical and Computer Engineering and core faculty at the Sanghani Center, “Safe Reinforcement Learning for Interactive Systems with Stakeholder Alignment.” Through the integration of reinforcement learning and game theory, the project aims to develop a fresh framework to ensure the safe and effective operation of interactive systems, such as conversational robots like ChatGPT. Importantly, the framework is designed to align with the needs and preferences of all relevant stakeholders, including users and service providers, each holding a vested interest in the system's performance. 
• Ismini Lourentzou, assistant professor in the Department of Computer Science and core faculty at the Sanghani Center, “Diffusion-based Scene-Graph Enabled Embodied AI Agents.” The objective of this research is to design embodied agents capable of tracking long-term changes in the environment, modeling object transformations in response to the agent's actions, and adapting to human preferences and feedback. The outcome of the proposed work will be more intuitive and attuned embodied task assistants, enhancing their ability to interact with the world in a natural and responsive manner.
• Xuan Wang, assistant professor in the Department of Computer Science and core faculty at the Sanghani Center, “Fact-Checking in Open-Domain Dialogue Generation through Self-Talk.” There is a growing concern about accuracy and truthfulness of information provided by open-domain dialogue generation systems such as chatbots and virtual assistants, particularly in health care and finance where incorrect information can have serious consequences. This project proposes a fact-checking approach for open-domain dialogue generation using language-model-based self-talk, which automatically validates the generated responses and further provides supporting evidence.  

Continued collaboration

“We are very pleased to continue our partnership with Amazon to encourage and support our faculty and student researchers aimed at finding solutions to important and worldwide industry-focused problems across a range of machine learning applications,” said Naren Ramakrishnan, the Thomas L. Phillips Professor of Engineering and director of the Amazon-Virginia Tech Initiative.

“As we move into our second year, we are expanding into additional areas of machine learning such as robust large language model deployment, combining large language models with reasoning capabilities and multimodal interfaces,” Ramakrishnan said. 

“Our sincere appreciation to the Virginia Tech team for their unwavering dedication to excellence in both research and education as reflected in the impactful research and significant progress made during the first year of our partnership as well as the high-quality proposals and fellowship applications we have received this year,” said Reza Ghanadan, senior principal research scientist in Alexa AI and the lead for the initiative at Amazon. 

“I look forward to continuing our collaborations with the esteemed faculty and students at Virginia Tech to advance our shared goal of ensuring the robustness of machine learning systems while creating impactful AI applications across diverse domains enriching our society,” Ghanadan said.

Last spring, Virginia Tech and Amazon gathered for a Machine Learning Day at the Virginia Tech Research Center — Arlington to celebrate and further solidify their collaboration. 

The program included presentations by the initiative’s inaugural cohort; panel discussions; a poster session with Virginia Tech graduate students from Blacksburg and the greater Washington, D.C., metro area; and networking opportunities.

Ramakrishnan said that a similar event will be planned for 2023-24 awardees in the spring.