The White House wants federal officials and others to tread carefully when using big data to prevent unintended bias or even outright civil rights violations — some which have already happened.
In a report released this week, officials warn that sound analytical practices have failed to keep pace with technology that allows for much easier collection of and access to information. The resulting gap creates the possibility that poor people, racial minorities or other disadvantaged populations could be discriminated against — even by supposedly neutral algorithms.
Discrimination that could result from improper usage of algorithms ranges from selection bias to the unintentional perpetuation of historical disadvantage, say the authors in a blog post. Failure to properly use big data has caused numerous, albeit unintended, civil rights infringements.
One example the report cites is in the consumer credit industry, where possible borrowers can be automatically disqualified, for example if their credit history is too thin or non-existent.
“Big data is here to stay,” write U.S. CTO Megan Smith, Deputy CTO DJ Patil and Domestic Policy Council Director Cecilia Munoz in the blog post. “The question is how it will be used: to advance civil rights and opportunity, or to undermine them.”
“Big data techniques have the potential to enhance our ability to detect and prevent discriminatory harm,” the report states. “But, if these technologies are not implemented with care, they can also perpetuate, exacerbate, or mask harmful discrimination.”
One example cited in the report for how big data can be retooled to help resolve civil rights issues rather than perpetuate them is through the streamlining of law enforcement data. Big data allows law enforcement officials to identify patterns in criminal activity by linking offense data to patterns in temperature, time of day, proximity to other structures and facilities, and other variables in order to allocate scarce resources more efficiently.
The report recommends that law enforcement use those same techniques to shape model policies and best practices to increase community trust and to monitor how the modeling of criminal hotspots has affected target communities. In addition, the findings should be made public so that accountability and transparency will go hand in hand with big data uses.
The report is not meant to be the final word on proper big data uses, merely part of an ongoing process.
As part of that, the National Science Foundation has been directed to begin researching new techniques to perfect the fairest and most ethical uses of big data. In addition, the NSF will be researching a method to help teach those new techniques.
The process of adopting new, more ethical standards when using big data may not be quick, but according to the report, the potential benefits outweigh any temporary growing pains that will be experienced along the way.
“The use of Big Data can create great value for the American people, but as these technologies expand in reach throughout society, we must uphold our fundamental values so these systems are neither destructive nor opportunity limiting,” the report concludes.
“Moving forward, it is essential that the public and private sectors continue to have collaborative conversations about how to achieve the most out of Big Data technologies while deliberately applying these tools to avoid – and when appropriate, address – discrimination.”
The full report is available online here.