Safe and Cooperative AI

Brief description of the context of the research including potential impact
Further research into techniques for analysis and guarantees of alignment and cooperativeness
in advanced AI systems. Methods for scalable interpretability and oversight of learning systems.
This is in the context of other attempts to develop alignment and oversight techniques, as well
as the cooperative AI agenda, and potential impacts include improved safety guarantees,
better-informed decisions about training and deployment, and greater foundational
understanding of large-scale learning systems.
Aims and Objectives
- Scalable oversight of advanced AI technologies: increase ability of operators to delegate
complex tasks to AI systems and achieve desired, well-understood outcomes.
- Analysis and guarantees of alignment: empower informed decisions about training and
deployment of advanced AI systems, mitigating risk of unintended outcomes.
- Analysis and guarantees of cooperativeness: avoid negative externalities, undesirable
network effects, and conflict resulting from deployment of AI systems.
Novelty of the research methodology
We have already developed a novel analytical framework for multi-principal-multi-agent
delegation in Delegation Games (Sourbut et al forthcoming). Research methodology is expected
to comprise a mixture of theoretical analysis, algorithmic innovation, and computational
experiments (simulations, learning algorithms).
Alignment to EPSRC's strategies and research areas (which EPSRC research area the
project relates to)
Artificial Intelligence Technologies, Building a Secure and Resilient World, Theoretical Computer
Science, Verification and Correctness
Any companies or collaborators involved
Cooperative AI Foundation (CAIF) (probable)