AI and tech ethics resources
I follow as much discourse around ethics in machine learning, data analysis, and artificial intelligence as I can. These are the resources I’ve used over the years to help me gather knowledge and perspectives and form my own opinions about these types of technology and implementations.
I’ve co-presented two talks on machine learning bias, and gave another on my own about the effects of missing data on data analysis. The slides for those talks can be found on About Sarah.
As an individual person, I have specific interests and biases that inform what content goes from “encountered in my news feed or RSS reader” to “opened in a new tab” to “actually read the thing”. This list is categorized according to those loose interests and resultant taxonomy. Some items might fall into multiple categories.
I’ve denoted things as follows:
- π for books
- π for research papers
- ποΈ for podcasts
- π° for articles in non-academic publications
- π» for blogs or blog posts
- π§ for email newsletters
I’ve also included some content that I haven’t yet made the time to consume, denoted with a βοΈ emoji.
Dataset creation and curation, including data labeling #
- π Living in Data by Jer Thorp
- π Ghost Work: How to Stop Silicon Valley from Building a New Global Underclass by Mary L Gray and Siddharth Suri
- I found this book to be pretty dry, and there were some editing issues in the Kindle version that I read, but still highly informative.
- π Data and its (dis)contents: A survey of dataset development and use in machine learning research
- π The Data-Production Dispositif
- π Large image datasets: A pyrrhic win for computer vision?
- π Datasheets for datasets
- π An Assessment of Intrinsic and Extrinsic Motivation on Task Performance in Crowdsourcing Markets
- π° Underpaid Workers Are Being Forced to Train Biased AI on Mechanical Turk (Vice)
- π° The Exploited Labor Behind Artificial Intelligence (Noema)
- ποΈ GPT-4 and the Politics of OpaqueAI (ft. Abeba Birhane) by This Machine Kills
- π Discussed in this episode The Values Encoded in Machine Learning Research
- ποΈ How Britain Killed its Computing Industry w/ Mar Hicks - Tech Wonβt Save Us
- I have a note from listening to this podcast with the phrase “do away with the notion of data as a mirror to society, it distorts society” so relevant if only for that line.
- ποΈ How AI Makes Living Labor Undead - This Machine Kills
- π Atlas of AI by Kate Crawford βοΈ
- π Data Feminism by Catherine D’Ignazio and Lauren F. Klein βοΈ
- π Wisdom for the Crowd: Discoursive Power in Annotation Instructions for Computer Vision βοΈ
- π Do Datasets Have Politics? Disciplinary Values in Computer Vision Dataset Development βοΈ
- π Documenting Computer Vision Datasets βοΈ
- π We Haven’t Gone Paperless Yet: Why the Printing Press Can Help Us Understand Data and AI βοΈ
- π Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data βοΈ
- π Towards Accountability for Machine Learning Datasets βοΈ
Representation in tech and machine learning, globally, linguistically, racially, or otherwise #
- π Invisible Women: Data Bias in a World Designed for Men by Caroline Criado Perez
- This book is the canonical “missing data = bias” book, but I found it to be problematic for two reasons:
- The writing is fairly academic, with citations upon citations and discourse about specific research findings that go on for so long that you can’t quite remember what point the research summary is supporting.
- The premise of the book is deeply gender essentialist. A book about missing data that never mentioned gender non-conforming or trans folks, and relied on a definition of gender that was rooted in biological sex. A great reminder to be mindful of the biases present even in books about bias!
- This book is the canonical “missing data = bias” book, but I found it to be problematic for two reasons:
- π° An MIT Technology Review Series: AI Colonialism (MIT Technology Review) βοΈ
- π [2111.15366] AI and the Everything in the Whole Wide World Benchmark βοΈ
- π More Than a Glitch: Confronting Race, Gender, and Ability Bias in Tech by Meredith Broussard βοΈ
- π Programmed Inequality by Mar Hicks βοΈ
- π Black Software by Charlton D. McIlwain βοΈ
- π Mismatch: How Inclusion Shapes Design by Kat Holmes βοΈ
- π The Coloniality of Data Work in Latin America βοΈ
- π Towards decolonising computational sciences βοΈ
- π The Limits of Global Inclusion in AI Development βοΈ
- π The Forgotten Margins of AI Ethics βοΈ
Machine learning bias, especially for decision-making #
- π The Alignment Problem by Brian Christian
- ποΈ Transcript: Ezra Klein Interviews Brian Christian
- π Weapons of Math Destruction by Cathy O’Neil
- Much more casually written and full of personal anecdotes than the Alignment Problem, but that probably explains why this book was such a mainstream hit.
- π Studying Up Machine Learning Data: Why Talk About Bias When We Mean Power?
- π° Machine Bias β ProPublica (ProPublica)
- Basically the canonical example of irresponsibly deployed machine learning models.
- ποΈ The Limitations of ChatGPT with Emily M. Bender and Casey Fiesler β The Radical AI Podcast
- π° Health Care Bias Is Dangerous. But So Are βFairnessβ Algorithms | Wired hopped by 12ft.io βοΈ
- π° The Fraud-Detection Business Has a Dirty Secret | WIRED (series) βοΈ
- π Algorithms of Oppression: How Search Engines Reinforce Racism by Safiya Umoja Noble βοΈ
- π Automating inequality: how high-tech tools profile, police, and punish the poor by Virginia Eubanks βοΈ
- π The Black Box Society:The Secret Algorithms That Control Money and Information by Frank Pasquale βοΈ
- π Predict and Surveil: Data, Discretion, and the Future of Policing by Sarah Brayne βοΈ
- π Artificial Unintelligence: How Computers Misunderstand the World by Meredith Broussard βοΈ
- π Technically Wrong: Sexist Apps, Biased Algorithms, and Other Threats of Toxic Tech by Sara Wachter-Boettcher βοΈ
- π Hello World by Hannah Fry βοΈ
- π Your Computer Is on Fire edited by Thomas S. Mullaney, Benjamin Peters, and Mar Hicks βοΈ
- π Captivating Technology: Race, Carceral Technoscience, and Liberatory Imagination in Everyday Life edited by Ruha Benjamin βοΈ
- π Sorting Things Out: Classification and Its Consequences by Geoffrey C. Bowker and Susan Leigh Star βοΈ
Machine learning development and implementation #
- π The Ethical Algorithm by Michael Kearns and Aaron Roth
- A practical discussion of ethical approaches to algorithm design and the involved tradeoffs.
- ποΈ Deploying Machine Learning Models Safely and Systematically β The Data Exchange
- π On the Dangers of Stochastic Parrots
- ποΈ Emily M. Bender β Language Models and Linguistics | gradient-dissent β Weights & Biases
- ποΈ The Power of Linguistics: Unpacking Natural Language Processing Ethics with Emily M. Bender
- π Understand, Manage, and Prevent Algorithmic Bias: A Guide for Business Users and Data Scientists by Tobias Baer βοΈ
- π The Oxford Handbook of Ethics of AI edited by Markus D. Dubber, Frank Pasquale, and Sunit Das βοΈ
- π You Look Like A Thing and I Love You by Janelle Shane βοΈ
- I’m halfway through this and it’s excellent so far. A very straightforward discussion of how ML works, how it doesn’t work, and why.
- π The Fallacy of AI Functionality βοΈ
Auditing, testing, and monitoring machine learning #
- ποΈ Auditing machine learning models for discrimination, bias, and other risks β The Data Exchange
- π Who Audits the Auditors? Recommendations from a field scan of the algorithmic auditing ecosystem βοΈ
- ποΈ Donβt Fall for the AI Hype w/ Timnit Gebru - Tech Won’t Save Us
- π Closing the AI accountability gap: defining an end-to-end framework for internal algorithmic auditing βοΈ
Alternate approaches to data-driven and ML-driven systems #
- π Design Justice: Community-Led Practices to Build the Worlds We Need by Sasha Costanza-Chock βοΈ
- π Viral Justice: How We Grow the World We Want by Ruha Benjamin βοΈ
- π Race After Technology: Abolitionist Tools for the New Jim Code by Ruha Benjamin βοΈ
- π Data Conscience: Algorithmic Siege on our Humanity by Brandeis Hill Marshall βοΈ
General resources #
- ποΈ The Radical AI Podcast
- A podcast at the intersection of ethics and artificial intelligence.
- ποΈ The Data Exchange Podcast
- A podcast from folks in the data industry.
- ποΈ This Machine Kills
- A podcast from researchers and critics of AI and the data industry.
- ποΈ Tech Won’t Save Us
- A podcast that critically examines the tech industry, data included.
- ποΈ Gradient Dissent from Weights and Biases
- A podcast from folks in the data industry.
- π§ The AI Ethics Brief
- π» β
β€β° Vicki Boykis β
β€β°
- An excellent blog from an ML practitioner and champion of data grunt work.
- π§ Import AI
- Not about ethics or bias specifically, but a worthwhile perspective from an AI insider.
- π§ AI Snake Oil
- π§ Counting Stuff
- π§ Benn Stancil
- π§ Rebel Tech
- π arXiv > Computer Science > Artificial Intelligence
- π arXiv > Computer Science > Computer Vision and Pattern Recognition
- π arXiv > Computer Science > Computers and Society
- π arXiv > Computer Science > Computation and Language
- π arXiv > Computer Science > Machine Learning
- π arXiv > Computer Science > Human-Computer Interaction
- I use a keyword filter on the Computer Science portion of arXiv with my RSS feed reader to pull up potentially relevant articles, but these are the areas in which they usually appear.
- π AAAI/ACM Conference on AI, Ethics, and Society (AIES)
- π ACM Conference on Fairness, Accountability, and Transparency (FAccT)
- Useful conference proceedings to dig into each year.