Many of AI’s risks in areas like AI evaluations, privacy, value alignment, copyright, concentration of power, loss of control, and hallucinations can be reduced to the lack of attribution-based control in AI systems, which itself can be reduced to the overuse of addition and copying within AI algorithms.
OpenMined’s primary technical goal is to synthesize a handful of recently proposed techniques in deep learning, cryptography, and distributed systems into a viable path to reduce addition and copying, provide attribution-based control, and address many of AI’s primary risks while maximizing its benefits — unlocking 6+ orders of magnitude more data, compute, and associated AI evaluations and capability in the process.
This article details the fundamental problem of attribution-based control, and the ways it underpins other problems like privacy, value alignment, copyright, concentration of power, loss of control, hallucinations, and AI evaluations.
The Problem of Attribution-Based Control (ABC)
Attribution-based control (ABC) describes a relationship between sources of AI-related data, compute, and talent and the AI users receiving predictions. These two groups of people have ABC when two things are true:
- Resource owners: dynamically control how much intelligence to give each AI prediction they support
- AI users: dynamically control which resource providers they want to rely upon for each AI prediction.
While AI systems learn from data, current AI architectures fail to offer resource owners and users attribution-based control—meaning they cannot verify or manage which data sources influence which AI predictions. This has cascading consequences across society at multiple levels: individual, institutional, societal, and geopolitical.
Individual Consequences: Authenticity and Trust
At an individual level, missing ABC leads to authenticity issues—hallucinations, deepfakes, and disinformation. Unlike traditional research methods, AI-generated content typically offers no bibliography or verification mechanism. For instance, a medical student researching rare diseases cannot verify if the AI-sourced information originates from peer-reviewed journals or less reliable sources.
AI “hallucinates” by combining unrelated data points to produce plausible yet incorrect outputs, a problem inherent to the lack of attribution. Similarly, disinformation occurs when AI merges unrelated concepts, like combining celebrity images with controversial acts to produce misleading deepfakes. Without ABC, users cannot identify or correct such misinformation.
Institutional Consequences: Data Sharing & Control
For institutions holding valuable data, the absence of ABC creates an incentive problem. Institutions must choose between contributing to AI development—risking loss of control and privacy—or withholding data, limiting AI’s capabilities. Major medical institutions, for instance, often decline AI partnerships due to privacy and control concerns, significantly hindering advancements in fields like cancer detection.
Privacy-enhancing technologies (PETs), such as federated learning and secure multi-party computation, address privacy but still fail to deliver precise control over data usage. Without attribution, institutions cannot selectively support or withdraw from specific AI predictions, perpetuating data siloing.
Societal Consequences: Governance & Representation
At a societal level, the absence of ABC creates governance crises—AI safety concerns, value misalignment, and bias. AI systems concentrate control in the hands of their creators or maintainers, leading to decisions misaligned with societal values. Without ABC, AI decisions about content filtering and model behavior are made by a small group, not reflecting the broader public’s interests.
ABC addresses this issue by reconnecting data contributors and AI users, ensuring continuous public consent and representation in AI-driven decisions.
Geopolitical Consequences: Democracy vs. Centralization
On the geopolitical stage, the lack of ABC creates a stark choice: centralize data and decision-making, compromising democratic values, or lose AI capabilities to authoritarian regimes that do centralize aggressively. This dilemma underpins current geopolitical AI competitions, notably between democratic nations and authoritarian states like China.
Proposals like an “AI Manhattan Project” to centralize resources in democratic nations risk undermining core democratic principles without guaranteeing competitive advantage. ABC provides a decentralized alternative, allowing democratic nations to leverage vast global data and compute resources without centralization.
Centralization Consequences: AI as Central Intelligence or Communication Tool
Ultimately, the lack of ABC positions AI as a tool of central control rather than a decentralized communication medium. Current AI systems centralize knowledge, guiding its use according to centralized interests, which disadvantages societies that resist centralization.
By shifting from operations that break ABC (addition and copying) to those that preserve it, AI can evolve into a true communication technology, directly connecting knowledge holders with seekers. This change is not merely technical; it aligns AI development with democratic principles and broad societal interests.
The Road Ahead: From Deep Learning to Broad Listening
Recent breakthroughs in deep learning, cryptography, and distributed systems have begun addressing these fundamental issues, paving the way toward practical ABC solutions. At OpenMined, we’re seeking to transform those solutions into free, open-source software across a live network: the public network for non-public information.
For further reading, check out the What is Broad Listening? blog post.
Sign up for updates or join our community using the form below.