The Ethics of Autonomous Systems and Algorithmic Decision-Making

Autonomous and semi-autonomous decision systems — self-driving vehicles, automated hiring and credit-scoring tools, predictive policing software, and weapons systems capable of selecting and engaging targets with limited direct human control — pose a distinctive philosophical difficulty: assigning moral and legal responsibility becomes harder when a machine-learning system's internal reasoning is not fully specified in advance by any human designer and is often not fully interpretable even after the fact. Philosopher Andreas Matthias, in "The Responsibility Gap: Ascribing Responsibility for the Actions of Learning Automata" (Ethics and Information Technology, 2004), named this difficulty the "responsibility gap": self-learning, adaptive systems can produce harmful outcomes that neither the manufacturer, who did not specifically program the harmful behavior, nor the operator, who could not have reasonably foreseen it, nor the machine itself, which lacks the standing to be a moral or legal agent, can be straightforwardly held responsible for.