Ten years of five-star ratings

Stefan Duma has told the story of Lester Karlin’s phone call before. One day in 2009, Karlin — then the equipment manager for Virginia Tech Athletics — called him up “and asked what helmets I would recommend that he buy for the football team.”

Two years later, Duma’s lab published the first independent safety ratings for varsity football helmets. Ten years after that, what became the Virginia Tech Helmet Lab has rated protective headgear for six sports, and more are in the works. The ratings initiated a sea change in how sports equipment is designed and marketed, driving the development of better equipment by shifting the balance of information from companies to consumers. 

Duma, the Harry Wyatt Professor of Engineering and a faculty member in the Department of Biomedical Engineering and Mechanics, had already earned national recognition for his work in injury biomechanics. But he was the one Karlin called for helmet advice that day because, since 2003, the researcher had been working with football team physician Gunnar Brolinson, associate athletics director Mike Goforth, and then-head coach Frank Beamer to study head impacts in college football using a new kind of sensor that could be embedded directly in the players’ helmets. 

At a time when awareness of concussion was just surfacing as a serious problem in contact sports, these sensors gave researchers the first direct access to quantitative data about the forces players’ heads were subjected to on the field. Duma had never evaluated the helmets themselves, though, and wasn’t sure what to tell Karlin. 

But he was curious. So he went out and bought every model of varsity football helmet on the market and carted them back to the basement lab, then mostly devoted to Duma’s auto safety research. Much of the floor space was occupied by a sled designed to simulate car crashes, but Duma and his team carved out space along the back wall for a drop tower that could reproduce football-style impacts with a dummy headform that plummeted like the blade of a guillotine down cables onto an anvil. Sensors hidden inside the head’s rubbery blue casing recorded the hit.  

Using the thousands of impacts they’d recorded on the field, Duma and a team of graduate students — led by Steve Rowson, who now directs the lab — developed a system of impact tests at different forces and angles that reflected how players actually hit their heads. The system, which they called Summation of Tests for the Analysis of Risk — STAR — generated a numerical value that described how effectively each helmet reduced the force on the head, compared to wearing no helmet at all. The scores were plotted on a scale from zero to five stars. 

“What we were doing was totally different than anything else that was available at the time,” Duma said. 

Since the 1970s, all football helmets sold in the U.S. have had to clear a pass-fail standard established by the National Operating Committee for Standards on Athletic Equipment (NOCSAE) certifying they could keep forces on a player’s head below a threshold associated with catastrophic injuries like skull fractures. But those test results weren’t public, and Duma and Rowson’s data showed that that simple threshold was obscuring huge variation in performance. 

“We were really surprised,” said Rowson, an associate professor of biomedical engineering and mechanics. “And we realized that no one was putting any of this information out there.” 

NOCSAE’s standard also didn’t address forces below that catastrophic threshold. An impact doesn’t have to crack the skull to be dangerous: The abrupt deceleration the brain experiences when a player’s head crashes into a hard surface strains the delicate tissue, causing damage that can lead to injuries like concussion that can be sublethal but still devastating.

Duma and Rowson’s rating system was the first to account for these concussion-level forces. When they released their results, Duma said, “It was like a light-switch change in the sport.” Colleges collectively ditched their poorly performing helmets (which were a lot of them: about half of varsity squads were playing in a popular helmet that only managed one star) and shelled out for more protective ones. 

From the beginning, it was crucial to Duma that the ratings be easy for consumers to interpret, and the five-star scale — a callback to his auto safety work — felt intuitively accessible. “There’s a huge amount of data and science that goes into the ratings, but the five-star system is something people understand.”

“Once we started talking to parents and coaches, it was obvious that there was an extraordinary demand for this information,” Duma said. “Helmets are heavily marketed, and people are used to hearing everyone say, ‘our helmet’s the best.’ The independence and the science behind our research really resonated with people.”  

The lab made a point of not accepting funding from helmet companies, and the researchers, invited all over the country to give talks on the STAR system, donated their speaking fees to buy new helmets for local youth teams. 

By 2014, the ratings had been cited in an Institute of Medicine report on sports-related concussion — a signal of approval from the academic establishment — and the team had decided to expand the approach beyond football. Ratings for hockey helmets — a sport with a famously high head-injury rate — rolled out in 2015, followed by soccer, cycling, flag football, and youth football. Each sport gets a unique set of test methods, developed based on information about real-world head impacts. 

The ratings are all available publicly, on a website that has racked up more than a million unique visitors at a rate of about 1,000 a day. The lab’s work has been covered in The New York Times, The Washington Post, Sports Illustrated, WIRED, and on hundreds of broadcast channels and ESPN. Major manufacturers use the lab’s logo as a point of pride for high-performing products, and a flag football league in Texas mandates headgear that earns at least four stars in the lab’s ratings.

Much of the research has been funded by the National Institutes of Health, with increasing support over the last five years from the NCAA through a multi-university consortium dedicated to preventing concussions in college athletes. 

“The ratings exemplify the concept of translational research,” said Duma, who is also the director of the Institute for Critical Technology and Applied Science. “People are using this data to make more informed decisions, and it reduces the head impact burden on players in all these sports. We know we’re reducing concussion risk in millions of athletes.” 

They’ve also compelled helmet companies to improve their game. 

“Helmet designs have changed dramatically,” Rowson said. “If people are making choices based on these ratings, manufacturers have to respond. And the way they respond is they make better helmets.” 

Tracking the ratings over time is a window into how helmet technology has matured. There was only one five-star helmet in that first set of ratings Duma released in 2011. On the ratings website today, you can scroll through 18 five-star options. In the first hockey helmet ratings, no helmet managed more than three stars, but within two years companies had rolled out four- and five-star models. “They have thicker padding, more advanced padding, more elegant designs,” Duma explained. “A lot more engineering goes into them.”

The trend has repeated itself in sport after sport. “You can see all the companies making better products now,” Rowson said. “The idea is to get everybody making the best possible protection.” 

Over the next few years, Duma and Rowson hope to extend the ratings to all major helmeted sports; testing is underway for baseball, softball, whitewater, and equestrian helmets. The equestrian project, part of a collaboration with the US Equestrian Foundation and the U.S. Hunter Jumper Association, has already attracted more than $425,000 in private funding. 

Duma says that it’s been especially rewarding to be able to engage so many students. “Each STAR system is, for the most part, a Ph.D. dissertation,” he said. “But at any given time we also have half a dozen undergraduate students in the lab, getting experience in research, learning to run the helmet tests.”

And it’s not just Virginia Tech students. The public-facing nature of the helmet ratings, and the hunger for information about head-impact risk, meant that almost immediately calls started flooding in from school groups wanting to see the lab, parents wanting helmet advice, and students with ideas for science fair projects. The lab channeled that enthusiasm into what is, today, a thriving outreach program involving school tours, “Build-a-helmet” activities, and now an exhibit at the Science Museum of Western Virginia. 

When tour groups walk through the lab, they stop to examine a helmet sliced in half to reveal its layers of padding; the metal track where two blue headforms crash together to simulate a pair of soccer players going up for the same header; the drop tower where bike helmets tumble off an anvil; and all the other specialized apparatus Duma and Rowson’s team has designed over the past 10 years. 

Over their heads, a row of industrial-size metal track lights attached to a beam are one remaining artifact of the lab's history: During automotive safety tests, they used to illuminate the crash sled in a blaze of incandescence. 

“When we started we had no idea that that helmet research would become as big as it has,” Duma said. “We just thought we would help the football team figure out which helmets were better.”

Hundreds of thousands of impacts later, it’s safe to say they did.