FORT CARSON, Colo.—Does the weather outside affect how ready troops are for battle? What about the cumulative stress of training, exercises, or overseas deployment?
Before it can answer those questions, the Army must first determine how to measure how troops are affected by their environment. One way to do that is through serious incident reports, or SIRs, which document harmful behavior such as suicide, threats, accidents, domestic violence, child abuse, vandalism, and theft of personal information.
“We’ve got all this data. We’re just trying to…measure the right things, figure out what’s connected, and then use it to generate insights and make better decisions,” said Lt. Col. Jonathan Bate, commander for the 2nd Battalion, 23rd Infantry Regiment in the 4th Infantry Division’s 1st Stryker Brigade Combat Team. “Things like weather trends and prior harmful behaviors, unit type, operational tempo—and figure out what’s the risk of a unit having a harmful behavior or a serious incident report…so we’re able to quantify the risk level.”
Using the weather conditions example, Bate said one theory is that a unit has a 50-percent chance of having a serious incident if they’re working quickly in 20-degree rainy weather. But incorporating that type of predictive data modeling is a work in progress that hasn’t yet resulted in anything “meaningful,” he said. But picking the right metrics is a start, and could make it easier to predict human behavior.
“We’re all humans, and…we follow the same patterns, but if we know what to measure based on common sense…then we can measure and put them into a data model,” he said. That model then “spits out a probability, and also helps us know where to look, and that’s where the real value in this is.”
In 2019, the Army began shifting its primary focus to soldiers’ quality of life and culture as a way to improve the branch’s readiness for war. Senior Army leaders announced an official posture change before Congress in 2021.
A data team nestled in the 4th Infantry Division has been working on data analytics projects for more than a year, and some of those efforts have started to show results. One project is called the unit risk forecasting tool, which aims to predict and reduce harmful soldier behaviors using a data regression model that looks at the relationship between patterns in prior serious incident reports and the risk of a unit experiencing another serious incident. By figuring that out, command teams can develop the best ways to intervene.
The updated 2.0 model went live in February; it tracks SIRs, which are automatically sent up the chain of command, and sends an email alert when units accumulate too many. This allows leaders to pinpoint where there might be a potential problem.
“If a company, troop, or battery has two or more serious incident reports in the past three weeks, they have, at least, about a 20 percent chance of having another SIR in the current week. We call that a moderate risk level,” Bate said. “We call it a momentum effect, that when a string of bad things happen, there’s a more likely [event] to have other things happen, and so we can get the right resources.”
The 1st Stryker Brigade Combat Team, also called the Raider Brigade, has about 4,400 soldiers, organized in 37 companies, troops, and batteries. When the model sends a SIR alert, leaders can zoom in on the company that is struggling, said Bate, who also gets the alerts.
“We call it a Raider care surge. We bring the chaplain, the military, family life consultant, alcohol counselors to try to get to the bottom of what’s causing these harmful behaviors,” he said.
The brigade has had about a dozen care surges in the past year.
Army units used to report details about these incidents in a Word document. Those details were then logged into an Excel spreadsheet. Now, that data automatically goes into the division’s “data lake,” since SIRs can be reported via an online portal or dashboard.
“Now all of the elements on Ft. Carson, they report in a standardized template, and it feeds a common database…So we can use that data a lot more easily.” Bate said.
Two projects, one database
The unit risk forecasting tool is just one way the division is using data to keep soldiers safe. The 4th Infantry Division has 195 dashboards that leaders from across Ft. Carson can access to better understand their organizations. About 100 of these dashboards allow users to drill down to see trends at the company or battalion levels.
Most of the dashboards pull information from the Army’s Vantage platform, which has been powered by AI-company Palantir since 2019. Others use bespoke databases with culled data that wasn’t collected previously.
Some dashboards allow commanders to see where troops are deployed, if they’re fit for deployment (e.g. if they have immunizations due), or check on equipment status and training.
One is dedicated to serious incident reports.
“Before this fiscal year, we couldn’t see the trends—any issues—unless we were hooked into DES, [the directorate] of emergency services, the provost marshal on base,” said Maj. Lam “Beau” Nguyen, operations research and systems analyst with the 4th Infantry Division.
The division has since standardized the way commanders can file serious incident reports, and made it possible via an online portal.
Army Forces Command, or FORSCOM, requires submitted SIRs to have 123 data points recorded in a Word document, Nguyen said.
“We standardized it [into] a scroll down menu, and then we’ve used the FORSCOM workspace to automate it. Now, as the commander or first sergeant, inputs this SIR, it goes into the system to where, if I click on 1st Brigade, it’ll populate which SIRs are live. Then, at the battalion level, at the brigade level, at the company level, they can click on it and move it along, instead of having an email chain that goes back and forth with a Word document.”
Once approved, those reports are automatically imported into a Word document—the format FORSCOM requires—and emailed to the right parties, cutting down the time it takes to create and submit a SIR.
Leaders can also access trends over time.
“For fiscal year 2024, we have the trends for all of division over the course of months, and we started implementing a trend predictor to try to forecast how many SIRs we’re going to have in the upcoming month, so we can see if we match that trend or a little bit lower,” Nguyen said.
This is separate from Bate’s predictive model—although they use the same data. The dashboard model uses previously collected data to forecast what might happen in the future with a 95 percent confidence level.
The division is also using the platform it built for SIRs to assess other wellness programs on the installation, to see if participation affects the number of reports issued.
“We’re also using this to try to build our framework for assessing different programs on post. Between the [Better Opportunities for Single Soldiers] program, the ACS [Army Community Service] program, behavioral health…all the programs that the chief of staff of the Army cares about, they want to know if the programs are effective,” Nguyen said, “So we’ve also standardized the utilization data so that if a program is servicing a brigade, we can see if that helps with the trends for either SIRs or retention.”
Troubleshooting solutions
Data management is a significant barrier to the Army’s analytics efforts—mainly because a lot of information hasn’t been collected yet, so a database needs to be created first. That’s what happened with the serious incident reports.
“Before [Bate’s] model, the data wasn’t collected…It wasn’t captured and stored for analysis. It was just reported and briefed,” said Lt. Col. Nate Platz, commander of the 704th Brigade Support Battalion in the 4th Infantry Division’s 2nd Stryker Brigade Combat Team. “Now his model captures it and captures the trends, learns as it goes, and then is able to report on those trends.”
The goal for the next version of the unit risk forecasting tool is to make it even better at predicting behavioral trends that can pinpoint individual units and soldiers who need an intervention, Bate said.
“We want to measure things across the PSERT framework—personnel, supply, equipment, readiness, and training. Things like operational tempo, how many days do you spend in the field, how many days until a major training event when they’re going to deploy away from home,” Bate said. “It would be great to measure family stress, financial stress. Just broadly, I would say individual unit and environmental factors is the framework we’re using. Turns out, it’s really hard to measure things that get into a structured data set that we can use.”
Those barriers go beyond rote data entry. It becomes a bit of a conundrum: as they’re trying to figure out which categories to measure, some of the data isn’t being collected. And the predictive models need data to function.
One example is how, at least anecdotally, changes in leadership cause stress among troops, which can lead to a spike in serious incident reports. Bate would like to measure that—but he has to create a new database first.
“The hypothesis is that this induces stress—new leadership. So I want to measure that and test it. But what does it take?” he said. “I’ve got a team member on this, a lieutenant, who’s going through the brigade commander’s calendar here to get all of the hundreds of changes of command and responsibilities over the past two years. Essentially going through a spreadsheet by date, by company, and putting a one or a zero.”
As for the tool itself, there’s positive potential. A preliminary analysis showed the predictive approach with alerts and intervening with resources reduced the number of serious incident reports, Bate said.
“We also did some statistical analysis with a difference-in-differences model and showed that units that were treated after [the unit risk forecasting tool] had slow to slight reduction, statistically significant,” in serious incident reports,” he said. “There’s initial evidence that this might work, so we’re gonna keep refining the model to help it work better.”
The team also found a link between soldiers’ feelings of wellness and unit retention.
They asked soldiers a simple question: “‘Are you being developed and you have a path to achieve your goals in the Army?’ And the companies that scored higher on average had higher retention,” Bate said. “So [it’s] just correlation, not causation, but it suggests factors that commanders might want to invest in—things like [general technical] score improvements, doing more development activities, like training for the soldiers so they feel like they are advancing, getting ready for the their next [promotion] board…tuition assistance.”
Soldiers feeling valued also translated into fewer serious incident reports in their unit, Bate said.
“We’re just just scratching the surface, but there’s these things we can measure that help us achieve the outcomes we want.”
