Intelligence analysts at the Homeland Security Department are experimenting with cloud-based AI tools on sensitive but unclassified data to help make their jobs easier—but there will always be a human in the loop, Avery Alpha, principal deputy undersecretary for intelligence and analysis, said Thursday.
“We’re in a bit of a hybrid information environment, my piece of the intel community specifically, because…DHS Intel was created after 9/11 to plug that gap that became very obvious between the state and local partners, and the more highly classified intel community,” Alpha said during a panel on AI and national security challenges at the Amazon Web Services Summit in Washington, D.C. The invite-only panel was hosted by the Atlantic Council.
“We’re really that bridge to push information in both directions, try to get the messaging, the sensitivity from the highest classified levels down to something that we can share with state and local and then infuse their insights back into the more closed-off operating environment of the federal [intelligence community].”
State and local law enforcement organizations use a lot of “for official use only” or sensitive data—which is not secret, but still has to be handled with care.
That data “still has plenty of restrictions and a lot of concerns when you’re looking at the data you’re putting into these models, because of course, [companies] own it, for lack of better word, once you put it into your prompts,” Alpha said. “So we are waiting for these things to go FedRAMP High, so we can fully implement them in the classified space, and then build in the guardrails necessary to ensure that bias is not introduced, ensure that we can see the underlying sourcing, because my analysts need to be able to speak to the actual originating source for every element of the argument that’s being built and then created with the output.”
Alpha stressed that intelligence analysts will also need to be able to use their judgment to assess the accuracy of information the AI tools spit out.
“We have to be able to describe what the logic was behind it. And this can be a very helpful tool as a starting point. But at the end of the day, I want a person to get in and say…’This piece of information, I will put more confidence in it. This one not so much. And at the end of the day, my overall confidence level is X.’”
DHS is testing the tools in unclassified environments first to help the workforce get familiar with the technology.
“We are piloting now with our workforce with the appropriate training and managerial approval, using the fully unclassified models in unclassified space with unclassified prompts to do those beginning drafts of training or briefing materials, and then to infuse that into the classified space and add manually the material,” Alpha said. “My intent with that is to give my team the ability to get familiar with the tools. And so when we do have models that can operate securely in our classified environment entirely, they’ll know how to play with it and manipulate it because it’s a skill to learn how to do a proper prompt and how to ask the questions the right way.”
Late last year, the White House pushed for all of the federal government to implement artificial intelligence where possible, through an executive order followed by a set of executive actions to manage risks announced in January.
DHS is also putting data scientists in its analytics centers to encourage “cross talk” with “technical experts, learning analysis and tradecraft and the needs and the key intelligence questions, and then how we can take those back and better manipulate the large data sets that we have, to extract insights that then enrich their analytic products,” Alpha said.
The department has used those methods to identify migration trends for border security and used AI tools to spot suspicious visa applications.
“We’ve also used AI tools to help us identify individuals from some of our foreign adversary nations who are trying to exploit our visa system, to their own benefit, and flag those visa applicants, refer them for appropriate action within our own government, and then also share that concern surrounding some of these applications … with our Five Eyes partners,” she said.
