The ratings are based on laboratory tests: While the drop tests performed by the Virginia Tech researchers are designed in theory to measure the ability of a helmet to absorb the energy from hits coming from various directions, and the STAR value is based on the theory that lowering head acceleration is likely to result in fewer injuries on the field, Virginia Tech did not test helmets under game conditions.
In its July 2013 statement, NOCSAE criticized the STAR rating system on that basis: "There are a near infinite number of ways to test helmets (varying temperatures, impact location, helmet size, drop height, etc.) and, therefore, generalizations were made so that the helmets could be tested in a practical manner. Helmets were not tested under game conditions. For example, air bladder fitting and protection systems were not inflated to achieve it, even though the NOCSAE standards require that manufacturer fitting instructions be followed."
At the time the 2013 STAR ratings were issued, there were no methodologically sound peer-reviewed studies finding that helmet design can reduce risk of concussion in practices and games, with the one that did find such a reduction, published in 2006, heavily criticized as methodologically flawed or marred by conflicts of interest. [1]
Since that time, however, a study published online in January 2014 in the Journal of Neurosurgery[9] has reported a 53.9% reduction of concussion risk associated with the Riddell Revolution helmet compared to another Riddell helmet, the VSR4. The difference in concussion risk reported between the two helmets matched almost exactly the estimated concussion risk between the two helmets predicted by the 2013 STAR ratings, which estimated that a Riddell Revolution, a 4-STAR helmet introduced in 2000, would result in a 54% reduction in risk of concussion compared to the Riddell VSR4, a 1-star helmet employing 20-year-old technology which, although no longer on the market, was still being worn by as many as half of all collegiate and professional players as recently as 2011.
"We would never have guessed the risk reduction would have matched so closely," said Duma. "We think that this is excellent validation" of the STAR system, and, in our opinion makes the overall STAR ratings very useful." [For the full interview between MomsTEAM's Lindsay Barton Straus and Duma, click here.]
Other helmet companies and NOCSAE were quick, however, to point out limitations in that new study.
Said Ashley Quintero, National Sales Manager for SG Helmets, "We believe the most informative research is comprehensive and unbiased. In our opinion, the recent Virginia Tech study was not comprehensive, as the study limited its exploration to the products of just one manufacturer." Quintero expressed concern that, unless those limitations were considered, "there exists the potential for coaches, parents, and athletes to assume these results reflect all football helmets."
She cautioned "consumers against expanding the findings of one study and generalizing that its outcome holds true for all products," and said SG Helmets "recommends that coaches, parents and athletes make a well-informed decision based on the advantages and disadvantages of every available product before making a purchase."
"It would be misleading to suggest that the data [in the new study] supports the proposition that one helmet reduces the incidence of concussion more than another. There is a need for an abundance of caution in this respect," said Schutt CEO, Rob Erb. "Stripped of the headline grabbing claims of double-digit concussion reduction, there is little here to get excited about. The best that can be said about the published Technical Note, based upon the limited data set, is that it suggests that players who wore large standoff shelled Riddell Revolution helmets were diagnosed with fewer concussions than their counterparts who wore the much smaller standoff VSR helmets. However, even this conclusion would have to be taken with a grain of salt. The statistical differences can be accounted for by a number of factors not discussed by the authors." [For the full text of Erb's lengthy email to Brooke de Lench on the Rowson study, click here.]
Given those comments, it was not the least bit surprising that Erb, despite the fact that two Schutt helmets topped the list of 5-star helmets in the Virginia Tech 2014 STAR rating (and continues to hold down the top two spots in the current ratings), was quoted in NOCSAE in its 2014 statement as being quick to acknowledge that the ratings do not support a conclusion that the helmets will limit or prevent concussions. "Shutt Sports would never represent to somebody that they're not going to get a concussion if they wear one of our helmets," he said. "As a manufacturer of a helmet considered by this rating system to be the best available, I believe that telling people that an athlete is less likely to get a concussion if they use a 5 STAR helmet is irresponsible. The best helmet is the one that carries NOCSAE certification, fits the position, is configured with the proper mask and the player is comfortable with it," Erb said.
For his part, Mike Oliver, Executive Director of NOCSAE told MomsTEAM's Brooke de Lench at the time the 2014 study was issued that, "We are still reviewing the study and its conclusions, and it is hard to tell at this point whether the conclusion is that the Riddell Revolution helmet model is better than the Riddell VSR 4 model, or whether that conclusion also applies to other helmet brands and models. Those two helmets are based on different design concepts, with the main difference being that the Revolution model has a larger shell with more padding on the inside. This type of design difference also exists in almost all other brand helmet models introduced since 2000 when the Revolution was first made available, and this new design type has replaced the older style helmets over time, but we see an increase in concussions, which is not consistent with the study findings. So there are lots of questions and information that the study does not address." [For Brooke de Lench's complete email interview with Oliver, click here]
