Children’s National, Virginia Tech collaborate to improve pediatric ER outcomes

In the fall of 2021, the worst of the COVID-19 pandemic appeared — at least to many outside the medical community — to be behind us. As schools returned to fully in-person instruction, though, a whole new crisis hit pediatric emergency departments.

After having been spared the worst surges of the pandemic that had overwhelmed general hospitals, suddenly pediatric ERs were inundated with the tripledemic of COVID, flu, and RSV, all hitting a medical system that had been thinned out in prior years from more retirements and fewer new doctors.

Just as the national attention turned away, following a year and a half of medical stories dominating the headlines, pediatric emergency room doctors like Kenny McKinley at Children’s National Hospital in Washington, D.C., were underwater.

“The shift in the national mentality, from ‘we’re in a crisis’ to ‘we’re no longer in a crisis’ created a crisis for pediatric emergency medicine,” said McKinley.

McKinley saw the urgency and decided to focus on one aspect he could control. Now, Virginia Tech is helping bolster his and Children’s National’s initial efforts through artificial intelligence (AI) modeling to improve patient outcomes here today and hopefully all over in the near future.

When patients arrive at the emergency department, they are evaluated for the severity — or acuity — of their conditions. The fall 2021 surge created so much emergency room crowding that it led specifically to a spike in high acuity patients leaving the ER without being seen — a real safety risk for patients, but also an emotional burden for those whose job it is to treat them.

“It was hard work in the emergency department for most of the months of the year for several years in a row, starting really with the end of quarantine-related restrictions in 2021 and just a crash of patients needing care in the emergency department,” said McKinley. “We worked really hard and despite that, on a lot of days, it was challenging to find the emergency resources we needed to treat the needs of the community.”

These conditions conspired to form a downward spiral felt at children’s hospitals nationwide: Overcrowding led to worse outcomes, demoralizing the doctors and nurses that led to more burnout and staffing losses,  which led to more staffing shortages, more emergency department (ED) overcrowding, and more patients leaving without being seen.

“Left without being seen and ED crowding became just this overwhelming problem for me, working clinically in the fall of 2021, where it was not satisfying for me to go to work in the same way as it had been previously,” said McKinley.

As a self-described amateur programmer, he worked with the NERDS at Children’s — the Nexus for Emergency Research and Data Science — to build a crude, manually updated model with 48 predictive variables to examine past data in an attempt to predict days when there might be a large amount of patients leaving without being seen. Deployed in June 2023, the model delivered a prediction that there would be more than three such patients one day. McKinley deployed a surge team, consisting of himself and an additional nurse, and worked feverishly through the caseload. Only one patient left without being seen that day.

“It felt like a wild success,” he said.

It also came within a few weeks of the first AI for Pediatric Health and Rare Diseases symposium, a collaboration between Children’s National and Virginia Tech.

“I was flying off of that enthusiasm when I spoke with the Virginia Tech crowd about this opportunity to use a real-time forecast and do something about it, for an outcome that makes a difference, that matters,” said McKinley.

He found a partner in Patrick Butler, senior research associate at the Sanghani Center for Artificial Intelligence and Data Analytics at Virginia Tech’s new academic building in Alexandria, which brings together core faculty from the Institute for Advanced Computing, computer science, and other academic units across the university. Suddenly a project with much greater potential began to take shape.

Partnering to solve a real-world challenge

Butler has been at Virginia Tech since starting his undergraduate program in 2001, continuing on through a master’s degree, and earning his Ph.D. in computer science in 2014. Sanghani Center Director Naren Ramakrishnan asked him to come back just a few months later, and he’s been working on forecasting population level events and participating in health care-related initiatives such as the Centers for Disease Control and Prevention's FluSight challenge in the intervening years. 

“A lot of times in the private sector, you’re working toward one singular goal. In academia, if you’re lucky, you get to solve lots of different, interesting problems,” said Butler.

McKinley’s challenge was right up his alley, both personally and professionally.

“Kenny had a very interesting, very meaningful problem,” said Butler. “It’s a real-world problem that helps people and saves lives.”

The XGBoost model was a “really good first pass,” sort of a version 1.0 of what might be possible. But it suffered from some year-to-year and month-to-month drift, which makes sense considering the seasonal nature of contagious disease. Virginia Tech’s depth and breadth of knowledge immediately offered the chance to try lots of different, more time-intensive modeling to help refine and better identify trends in the historical data, which had now settled into a multiyear, “new normal” since the initial surge.

“All of that requires a level of rigor, including the need to consider the most advanced algorithms, that Virginia Tech is able to bring and make all of our work more meaningful,” said McKinley.

The two sides agreed to start a pilot program, working to refine McKinley’s first model into something that they hoped would provide accurate predictions of days when two or more patients would end up leaving without being seen, based on a combination of historical factors. That accuracy is paramount from both sides of the equation — false negatives mean patients go unseen, but false positives mean wasted resources and additional burnout for physicians being needlessly called in. 

McKinley credits his team and his superiors with having the faith in his process to start following the model’s predictions, deploying surge teams on one set day a week, if the conditions are met.

“With forward-thinking colleagues and a big enough problem, it was enough to start doing stuff with it,” he said.

Building a better model

Version 2.0 has been promising. The area under its receiver operating characteristic curve — a measure for reducing false negatives and positives, which ranges from 0 to 1 — was 0.85 from May 2024-April 2025. Of the 24 times the model predicted four or more patients who left without being seen in the last year, all 24 saw that outcome. The pilot program deployed surge teams on 20 of those days, which has certainly helped, though it’s hard to tell by exactly how much under the model’s current iteration.

That’s why what comes next will define the scope of what’s possible with this collaboration. The current model is built to anticipate patients leaving without being seen but not the many external variables that go into the conditions that create that outcome. What’s the weather that day? Was there a recent school holiday? Was there a major sporting event convening lots of people in the same place?

“We know that there’s an opportunity to improve model performance if we can capture some of the drivers for those individual decisions,” said McKinley.

That’s where a forward-looking, predictive model could be a game changer. The team has multisite data to analyze. This can help create scalable data pipelines and site-specific models, providing insight into performance before deployment. Yiqi Su, a Virginia Tech computer science Ph.D. student, is helping on the coding side while undergraduate student Gillian Doherty is helping clean data sources.

“There’s different levels of expertise and enthusiasm along the whole collaboration with Virginia Tech,” said McKinley. “At every level, there’s this little piece that VT is contributing that ultimately will get us to what I hope will be generalized models with better performance.”

Both sides hope the automated, much more refined version 3.0 will be available by the end of 2025, ideally leading to more meaningful predictions and a framework that can be adopted by other hospitals using their own data. But even if that’s the goal, it won’t be a finish line.

“I don’t know that I’d ever call it finished,” said Butler. “We’re scientists — we’re always making predictions, and making changes to those predictions, and learning from the present and the past.”

Butler and McKinley will be presenting their work at Tech on Tap at Virginia Tech’s Academic Building One in Alexandria on Tuesday, June 24, at 6 p.m. Learn more and get your tickets today.