DHS tests AI for making sense of contractor past-performance data
The Department of Homeland Security is almost done with the first phase of a project that will allow federal agencies to use artificial intelligence for a task that can be overwhelming for humans: extracting, analyzing and visualizing the underutilized data in the governmentwide system for contractor past-performance records.
Next week DHS will finish reviewing services from nine vendors who were given data sets from the Contractor Performance Assessment Reporting System (CPARS). Before awarding contracts worth more than $250,000, agencies review historical data in CPARS to see how offerers acted under previous agreements. It’s not an easy task.
With more than 1 million records covering 60,000 contractors, CPARS can bog down contracting officers, said Polly Hall, acquisition innovation advocate director of the Procurement Innovation Lab at DHS. They can sort records by contract value, Data Universal Numbering System (DUNS) number or performance period, but even once those filters are applied, the results frequently still number in the hundreds and can only be viewed one PDF at a time.
“There’s risk involved in that,” Hall said, while speaking at the ACT-IAC AI/IA Forum on Wednesday. “As a human, I might not read something in a given record that will be very relevant to my decision as a contracting officer in the source selection.” Agencies attempt to mitigate that risk by having offerers complete past-performance questionnaires, but vendors paint themselves in a good light and rarely distinguish themselves from their competition, Hall added.
Due to budget constraints, the Office of Procurement Operations (OPO) within DHS opted against buying a solution and instead issued a general solicitation in August under the relatively new Commercial Solutions Opening Pilot Program authority. The solicitation sought AI prototypes that could quickly find a vendor’s past performance records and possibly even predict if they would deliver on a contract.
“What we want is the output from the solution,” Hall said. “We want that output in the form of a CPARS AI report or maybe a subscription service with a dashboard and other visualization capabilities to help us extract, analyze and visualize the data.”
DHS envisions a commercial, multi-vendor marketplace where report or subscription services are sold — similar to the credit-report marketplace, she added.
Nine AI-enabled prototypes
The department committed in its solicitation to providing vendors with CPARS data and access to users — contracting officers, contracting officer representatives and CPARS focal points — for prototypes and received about 40 proposals.
DHS used peer review to select nine services in September for prototyping: Hangar Government Solutions, Accenture, CORMAC, IBM Global Business Services – US Federal, Strongbridge, World Wide Technology, TrueTandem, ElectrifAi, and Federal Government Experts.
When prototype contracts were awarded, DHS lacked the ability to go into CPARS with an application programming interface (API) or script and pull out the records in batch. Instead records were downloaded one PDF at a time and prepared and parsed into CSV files. Data had to be human-rated so vendors could use it to train their AI algorithms and then masked to protect contractor information being shared.
Vendors have participated in interactive demonstrations leading up to the final demonstrations of phase one next week. DHS requires vendors produce a CPARS AI report so the department can validate the system ingests the data and provides meaningful insights.
Explanations of AI methodology and recommendations for improving CPARS data quality are also expected from each vendor.
DHS can then award follow-on contracts, up to $50,000 in value, without further competition.
“Our intent is to maintain as much competition as makes sense as we move forward into the next phase — likely more prototyping with a more robust, larger data set,” Hall said. “All of the vendors have concurred that’s definitely a need.”
More data will ensure services aren’t overly sensitive and can support more data as it scales, she added.
Not just for DHS
CPARS is expected to merge with nine other legacy contract-award systems under beta.SAM.gov within two years.
DHS essentially offered to manage CPARS on behalf of the government and formed a coalition with the Office of Federal Procurement Policy that’s funded by the Chief Acquisition Officers Council, which also provides vendors access to users, Hall said.
The Navy owns CPARS through the Integrated Award Environment at the General Services Administration. So DHS also needed IAE’s involvement to eventually access live-production data through a third-party API.
OPO additionally partnered with the DHS chief information officer and the Center for Accelerating Operational Efficiency, the department’s center of excellence managed out of Arizona State University.
