Meta Announces Llama 3 at Weights & Biases’ conference

In an engaging presentation at Weights & Biases’ Fully Connected conference, Joe Spisak, Product Director of GenAI at Meta, unveiled the latest family of Llama models, Llama 3.
Highlighting a significant milestone in AI development, the Llama 3 models, including the impressive 8 billion and 70 billion parameter models released during the conference, along with a glimpse into the future with a 400 billion parameter model still in the works.
Joe shared insights into the training processes and alignment of Llama 3, which now ranks as the top-performing model in the open weights category on the MMLU, GSM-K, HumanEval benchmarks.
Weights & Biases is proud to support our customers such as Meta as they push the boundaries of AI, to learn how to fine-tune your LLMs using torchtune and Weights & Biases, start here: wandb.me/torchtune
Timestamps:
00:00 Introduction
03:05 Overview of Llama at Meta
05:59 Introducing Meta Llama 3
7:04 Advancements in Llama 3: Training and Data Scale
10:02 Benchmarking Llama 3 Performance
14:01 Enhancing Model Safety and Red Teaming
16:23 Expanding the Ecosystem and Future Directions
23:00 Closing remarks: Future plans for Llama models, and an invitation to use Meta's Lama 3.
#MetaLlama #ArtificialIntelligence #AITrends #TechInnovation

[@thenoblerot]

Thanks for this W&B

[@WeightsBiases]

Our pleasure!

[@Crux69]

My favorite fact from this is that the smarter the model, the more it violates rules. Just like us :)

[@utuberay007]

Very true ! People who are way smarter on tax laws are the one who violate most , innocent people pay more than what they are supposed to etc . Same goes with many other laws

[@techpiller2558]

Or, the rules it uses instead of the rules we assumed are different.

[@why.do.I.even.try.]

That's a great way to justify corruption and awful people.

[@Crux69]

@@why.do.I.even.try. awful people are still human, best to understand how good people become awful

[@why.do.I.even.try.]

@@Crux69 Yes but we shouldn't repeat their actions just because they work. We should work towards more ethical means to advance, technologically and societally.

[@ihesiulo]

There's a universe where Joseph Spisak is Mark Zuckerberg's brother. Oh, and nice presentation. Wonderful work they are doing at Meta AI.

[@siloquant]

Congratulations!

[@RakeshMurria]

I really enjoyed this. Thanks

[@WeightsBiases]

Glad you enjoyed it!

[@naninano8813]

so all those supervisor/safeguard models are only utilized during training? i mean, once the weights of llama3 are out, there is no safeguard network between user and inference engine right?

[@Crux69]

I'm sure they have a safety model that tries to review every request and catch some negative responses.

[@PeterLappo]

How much did it cost to build, including hardware and engineering costs?

[@techpiller2558]

What will be the SQLite of LLMs, with capability for local use? Llama?

[@thegreatgustby]

I think he could have said "ridiculous" a bit more often

[@gubatron]

vin diesel!

[@RichReportcom]

Summary: Safety and size. The end.

[@ericadar]

a few hours go by...llama 3 no longer SOTA

[@adinsoftic]

That's why they open source it. They let the community figure things out and iterate. For Meta LLM is just a tool and not a product on itself

[@SkepticButOptimist]

Wait what is sota now?

[@adinsoftic]

@@SkepticButOptimist "state of the art"

[@JeiShian]

Which model is sota?

[@MiraPloy]

I think it's supposed ro be either phi or sensenova, neither of which are released ​@@JeiShian

[@GerardSans]

How silly is to redteam a model which you control the training data to check for bioweapons capabilities. How stupid should you have to be? Isn’t easier to run a search on the data 😂😅

[@matbeedotcom]

I’m glad they saw how useless they made codellama 😂, it was waaaay overly aligned