Pentagon lacks tools, support to scale AI, says RAND report

The U.S. military’s plan to harness artificial intelligence for defense through the Joint AI Center is ambitious but it will need more resources and organizational support to scale the technology across the department, according to a new congressionally mandated report.

As required by the 2019 National Defense Authorization Act, the Department of Defense and the JAIC brought in RAND Corp. as an independent auditor to assess the department’s state of AI and its plans to scale it. The federally funded research and development center ultimately found in a 164-page report that DOD “lacks baselines and metrics in conjunction with its AI vision” and that the JAIC needs better “visibility and authorities to carry out its present role.”

“The DOD recognizes that AI could be a game-changer and has set up organizational structures focusing on AI,” said Danielle Tarraf, lead author of the report and a senior information scientist at RAND. “But currently the JAIC doesn’t have the authorities or resources it needs to carry out its mission.”

In particular, the report says, the lack of longer-term budgeting for AI development is hurting the progress of JAIC, which is led by Lt. Gen. Jack Shanahan.

In essence, RAND’s main findings suggest that DOD needs to get more “realistic” about its hopes of expanding AI across the military in the near-term. That’s possible, the report says, especially for enterprise, back-office applications — but operational and mission-support AI will take much longer to field.

But in any case, efficient and successful adoption of AI “would require fundamentally transforming DoD’s culture into a data-enabled one that values data and uses them to their full potential — a monumental endeavor but one that could enhance efficiencies across the board, in alignment with the defense objectives identified by DoD in the Summary of the 2018 [National Defense Strategy].” And DOD needs a plan to grow, track and cultivate personnel with the skills to develop and sustain the technology, the report says.

On top of all this, DOD’s verification, validation, testing and evaluation (VVT&E) isn’t yet up to snuff, making it so that the department can’t ensure the performance and safety of the applications it does field. This struggle, although common to all sectors that adopt AI, is especially worrisome in a defense context with safety-critical systems.

Many AI applications “are not ready yet for prime time in safety-critical systems,” Tarraf said.

RAND issued 11 recommendations in the report, to include adapting AI governance structure so that they align authorities and resources toward scaling the technology, developing a five-year AI roadmap for the JAIC and each military service, progressing the department’s VVT&E of AI, and others related to data and talent.

In September, JAIC’s Shanahan said the center is poised to have a huge year in fiscal 2020. “I am optimistic that 2020 will be a breakout year for the department when it comes to fielding AI-enabled capabilities,” he said, pointing to JAIC’s growth from $93 million in initial funding to a “healthy” budget — the department requested $268 million — for fiscal 2020.