Anthropic thinks Apple’s business terms could be good for humanity



summary
Summary

AI startup Anthropic gives a first look at the “constitution” of its Claude language model, which it hopes will set it apart from other models.

Anthropic uses a “constitution” to keep its Claude language model in check. As long as the model obeys the laws, it can respond freely. The startup doesn’t consider this approach to be perfect, but it thinks it’s more understandable and easier to adapt than training with human feedback, which is used by ChatGPT, for example.

For Anthropic, there are several reasons not to use Reinforcement Learning from Human Feedback (RLHF):

  • People who rate content may be exposed to annoying content.
  • It’s inefficient.
  • It complicates research.

AI feedback replaces human feedback

The constitution is used at two points in the training process. In the first phase, the model critiques its answers using the principles and some examples.

ad

In the second phase, the model is trained using reinforcement learning, but with AI-generated feedback instead of human feedback. The model’s task is to choose the “safer” output based on the constitution.

Training proceeds in two phases, during which the AI ​​improves using AI-generated feedback. | Picture: Anthropic

Interestingly, the model is not provided with any human-generated data on safety or harmlessness. All results in this regard come exclusively from AI training.

According to Anthropic, training with its constitutional method can lead to a win-win situation where constitutional RL is more useful and safer than RLHF. It is also more scalable and transparent.

A constitution inspired by Apple’s business terms

anthropogenic detailed Claude’s constitution in a blog post. For the constitution, the startup took inspiration from

  • the Universal Declaration of Human Rights,
  • the rules of Deepmind’s Sparrow, and
  • Apple’s terms of service.

Anthropic wants to show that a constitution should consist of best practices that don’t need to be reinvented. Respect for “non-Western” outlook is also addressed in the constitution.

Recommendation

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top