The Department of Homeland Security says it has found a solution for emergency big data and cloud processing.
Called Terrapixel, the Web-based imagery hosting platform allows DHS agencies like the Federal Emergency Management Agency and Immigration and Customs Enforcement to streamline the distribution of massive data sets, cutting costs and accelerating computation.
“We need to make sure data is accessible quickly, so the folks on the ground responding to incidents have the same resources we have at headquarters,” Mike Donnelly, a DHS geospatial data architect, said at the Amazon Web Services Government, Education and Nonprofits Symposium Thursday. “We’ve come to a point now where we have a lot of data — collecting data is not the core issue for us at DHS. The issue is how to make the data easy to use.”
Cloud computing has cut computing costs and added efficiency. Infrastructure that once cost $2 million and took six months to construct can now be created instantly on the cloud and perform 40 years of computing in eight hours for $4,372, according to Novartis Global Head of Scientific Computing Steve Litster, whose research has benefited from cloud technology.
The lure of quick, cheap computing prompted the DHS Terrapixel program, which evolved out of the need to render aerial imaging of disaster zones to first responders at “near-instant” speeds.
“Responding to a natural disaster is done on a local level; our core constituents are not just feds, they’re firefighters and fusion centers all over the country,” Donnelly said.
After the Deepwater Horizon oil rig explosion, DHS collected 52 terabytes of data, including 46,363 separate aerial imagery files from the National Oceanic and Atmospheric Administration, the Environmental Protection Agency, NASA, the Civil Air Patrol, and others, not including some additional satellite data.
Superstorm Sandy posed a comparable dilemma with 31 TB of data from aerial and satellite sources.
Compiling these data and getting them to workers on the ground was an expensive and time-consuming process, hindering damage assessment and reparation efforts. Donnelly said this boiled down to a lack of raw computing power, combined with declining budgets.
“The budget is going down, and the data keep getting larger,” he said. “That’s where cloud computing becomes an appealing prospect.”
Terrapixel first rolled out in the wake of the Joplin, Missouri, tornado of 2011, a level F5 vortex that killed 158 people and caused almost $3 billion in damage.
FEMA workers reported issues in identifying which areas had already been assessed and searched for survivors, due to a lack of aerial data for the town. DHS planes were deployed to the scene, and access to the Terrapixel system was extended to agents on the ground.
With the new data, responders established a coherent grid, streamlining their efforts. Donnelly said the incident cemented Terrapixel’s status as a DHS asset, as well as the department’s commitment to investing in cloud technology going forward.
“We were able to get in there and make that data available immediately,” Donnelly said. “It was a major success story for the department.”