Metas Llama 3 is set to release in July and could be a bit looser

[ad_1]


summary
Summary

Meta’s next open-source language model, Llama 3, is scheduled for release in July and is intended to be on par with GPT-4.

According to The Information’s sources, the new version of the model will be more responsive to the user, providing context for difficult topics rather than blocking tricky questions.

For words that can have a dangerous or harmless meaning depending on the context (e.g. “kill”), Llama 3 is said to be better at differentiating. With Llama 2, Meta took a more conservative approach to security, wanting to avoid a PR disaster at all costs.

Meta wants to put someone in charge of the model’s tone and safety training to make the model’s responses more nuanced. However, three key people in the field of AI safety have left Meta this month alone, The Information reports.

Ad

Ad

There are currently two camps when it comes to AI safety: One is the highly regulated approach, such as OpenAI’s ChatGPT, GPT-4, and Google Advanced, where the AI model has clear guidelines on how to respond in a polite and politically correct manner.

Then there is the more unconstrained open-source movement and the contrasting program of X owner Elon Musk, who deliberately offers a chatbot that better represents conservative world views. Unfortunately for the target audience, Grok is still too “woke” and respects transsexuals, for example.

Llama 3 could be twice as big as Llama 2

Whether Llama 3 will again be a pure language model or multimodal, i.e., able to understand and generate images as well as text, has not yet been decided. With more than 140 billion parameters, the largest version of Llama 3 could be twice as large as Llama 2 with a maximum of 70 billion parameters.

However, this would still be only a fraction of the original GPT-4 Mixture-of-Experts model with 1.76 trillion parameters. It is not known how many parameters the GPT-4 models currently running ChatGPT have, but it is probably less. Parameters are no longer the only and most important measure of the expected output quality of AI models.

Fine-tuning of Llama 3 has not yet begun, according to The Information’s sources. The final model is said to reach GPT-4 quality. Llama 2 was released in July 2023.

Recommendation

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top