Collective Constitutional AI: Aligning a Language Model with Public Input

This post details the process and results of a public input experiment to create a constitution for an AI system. It describes the use of the Polis platform with ~1,000 Americans to gather statements, the processing of these statements into a constitution, and the subsequent training of two Claude Instant-sized models: one with the public constitution and one with an Anthropic-written constitution. The post highlights key differences in the public constitution (focus on objectivity, impartiality, accessibility, promotion of desired behavior) and presents evaluation results showing the public-trained model to be less biased than the standard model, with equivalent helpfulness and harmlessness. It also transparently shares subjective judgment calls made during participant selection, screening, and platform choice.