• How do groups of humans make a decision?

• How do parallel predictions in the mind decide what to do?

• How do multiple biological cells act as one organism?

• How do parts of AI systems work together?

All of these systems have similar structure:

• Decentralized information integrating.

• Parts forming a whole.

This is what I think alignment is: the cooperation of smaller parts to build bigger agents.

This unifies my two primary interests in the last 2y: AI alignment and Human alignment. Because Human alignment ⇔ AI alignment ⇔ Alignment.

The goal? Unblock alignment on all scales.

A recursive process of parts forming ever larger wholes:

How does it work?

Thanks to Alex Zhu, Adam Goldstein, Ivan Vendrov, Xiq, and Stag Lynn for conversations.

Terms discussed in this post: superagent, subagents, collective intelligence, hivemind, feeling like part of a greater whole, community feeling; boundaries, autonomy, causal distance; scale-free alignment, multi-scale alignment.

Further Reading

Hierarchical Agency: A Missing Piece in AI Alignment by Jan_Kulvet

Towards a scale-free theory of intelligent agency by Richard Ngo

Michael Levin, e.g.: Technological Approach to Mind Everywhere (TAME) framework:

Romeo Stevens:

See all posts tagged “AI” on this blog, especially: 

Complete autonomy is impossible, of course. This is the lim→∞.

For example, imagine a bacterium.

It has insides: its guts. 

It has outsides: its environment.

And in between, it has a membrane. 

The membrane is the thing that attempts to control what comes in and what goes out. For example, pores can guide nutrients in and waste out.

The membrane is what maintains the bacterium’s separation from the environment.

Boundaries can grant autonomy.

For a bacterium to stay autonomous, its membrane must not dissolve.

In general, for an agent to stay autonomous, its boundaries must not dissolve. 

(A bacterium is just one kind of agent, and “boundaries” are just another term for “membrane”.)

Suitable boundaries grant autonomy.

Boundaries can provide safety without physical distance.

Let’s say an agent (blue) is endangered by a threat (red).

What could make blue safe?

One way is physical distance:

Another way is suitable boundaries:

Boundaries can provide “causal distance”.

In this way, suitable boundaries provide causal distance without physical distance. 

For details, see Agent membranes and causal distance.

Boundaries can allow cooperation.

When agents can rely on boundaries for safety (as opposed to relying on physical distance), they can interact more closely with each other. 

And when agents interact more closely, they can cooperate more!

For details, see «Boundaries», Part 1: a key missing concept from utility theory. 

To preserve autonomy, preserve boundaries.

Boundaries can prevent your insides from being controlled by your outsides.

Further reading

How this relates to multi-scale alignment:

For discussion about emotional boundaries:

All of my posts about boundaries/membranes on LessWrong

Formalizing «Boundaries» with Markov blankets

• Pol.is: "Input Crowd, Output Meaning"

  • Inspired Twitter/X community notes

  • People: Colin Megill, et al.

• Collective Intelligence Project

  • vibe: democratic AI, “How AI and Democracy Can Fix Each Other”

  • People: Divya Siddharth, Saffron Huang, et al.

• AI Objectives Institute

  • Talk to the City: "an open-source LLM interface for improving collective deliberation and decision-making by analyzing detailed, qualitative data. It aggregates responses and arranges similar arguments into clusters."

  • AI Objectives Institute works closely with the Taiwanese government.

  • Other projects in development.

  • People: Colleen McKenzie, Değer Turan, et al.

• Meaning Alignment Institute

  • vibe: democratic AI, kinda. 

  • I think they think that if you can help individuals make wiser decisions, at scale, then this converges to be equivalent with solving outer alignment.

• Remesh

  • Similar to pol.is AFAIK? I haven't played with it.

  • People: Andrew Konya, et al.

• Loomio: "a flexible decision-making tool that helps you create a more engaged and collaborative culture, build trust and coordinate action"

• Deliberative Technology for Alignment paper

  • They also discuss other tools for this use like Discord, Snapshot, Dembrane

  • People: Andrew Konya, Deger Turan, Aviv Ovadya, Lina Qui, Daanish Masood, Flynn Devine, Lisa Schirch, Isabella Roberts, and Deliberative Alignment Forum

  • Someone in the know told me to only read sections 4 and 5 of this paper

• Plurality Institute

  • People: David Bloomin, Rose Bloomin, et al.

  • Also working on some de-escalator bots for essentially Reddit comment wars

• Lots of crypto projects

  • Quadratic voting

  • Gitcoin

  • Metagov: “a laboratory for digital governance”

  • Soulbound tokens

  • Various voting and aggregation systems, liquid democracy

• Decidem

• Decide Madrid

• Consider.it

• Stanford Online Deliberation Platform

• Lightcone Chord (in development)

  • Brief description

  • People: Jacob Lagerros (LessWrong)

• All of the prediction markets

  • Manifold, Kalshi, Metaculus, PredictIt, etc.

• Midjourney has a Collective Intelligence Team now according to Ivan Vendrov’s website. I couldn’t find any other information online.

Any tools for small groups?

Most examples above focus on large group collective intelligence (say, ~300+ people). But what about smaller groups? Are there tools that will help me coordinate with 30 friends? Or just one friend? I haven’t found many tools like these, but I’ve listed some below. If you know of any more, please email or leave a comment on LW.

Small groups:

• Nexae (in development)

  • “Nexae Systems builds sociotechnical infrastructure to enable the creation of new types of businesses and organizations.” 

• double crux bot

• Liberating Structures looks list of exercises for small groups to try to enhance Collective Intelligence (no modern technology required)

• I’m surprised I haven’t heard of many other LLM-facilitated communication tools

Medium group (~30-300 people) projects: 

• Jason Benn’s unconference tools, eg Idea Ranker. Notes.

Other lists

• New 09-20: Plurality Institute’s Mapping Project

• @exgenesis's short tweet thread. Couple things I haven’t listed here.

• Nathan Helm-Burger's google doc list

• Know of any I should add? Please email me or leave comments on LW.

Opportunities 

• Metagov is running https://metagov.org/projects/ai-palace which seems similar

Updates and comments

See updates on LessWrong

Hypothesis: Understanding human alignment is just like understanding AI alignment because the underlying principles (alignment) are the same.

Example: Boundaries

Boundaries provide autonomy. When individuals’ boundaries are preserved, unnecessary conflict between those individuals is minimized. Boundaries could help specify the safety in AI safety. [Claim #4 here, Claim #9 here.]

My work in this area has mostly been writing distillations and convening researchers.

• Conceptual Boundaries Workshop (3-days)

• Mathematical Boundaries Workshop (5-days)

I became interested in this after I started thinking about the human boundaries, and this required understanding the causal distance between agents in general. And as it turned out, other researchers were already also thinking about this.1

How might boundaries be concretely applied to AI safety? One way is as a formal spec for provably safe AI in davidad’s £59m ARIA programme.

Conjecture: “Goodness”?

One way I like to think about what we want from ‘full alignment’ is two somewhat-independent properties:

(Also, notice that I haven’t smooshed Goodness and Safety into one axis. Usually when people do this they call it “Utility”.) Goodness and Safety are have different causes!)

And while boundaries/safety is nice, they don’t actively provide Goodness. So the question is: How can Goodness be specified?

Similarly, for human alignment: learning to do boundaries minimizes unnecessary social conflict, but… What causes joy? Connection? Collaboration? What causes Goodness?

I suspect the answer is whatever causes collective intelligence / synchronous social interaction.

Why might AI alignment be like human alignment?

As Michael Levin says, “all intelligence is collective intelligence”. Every intelligence worth hoping for or worrying about is made of smaller parts.

In which case, alignment itself can be defined as “How do smaller parts build bigger agents and avoid internal cancers?” Human alignment deals with the same questions:

• How do groups of humans make a decision?

• How do parallel predictions in the mind decide what’s right?

More here:

Thanks to Alex Zhu, Adam Goldstein, Ivan Vendrov, and David Spivak.