Amazon Olympus (large language model)

👋 Hi, I'm Alan. I advise government and enterprise on post-2020 AI like OpenAI GPT-n and Google DeepMind Gemini. You definitely want to keep up with the AI revolution this year. My paid subscribers (RAND, NASA, Microsoft...) receive bleeding-edge and exclusive insights on AI as it happens.
Get The Memo.



Waiting for Amazon Olympus? Check out Google DeepMind Gemini…

Google DeepMind Gemini: A general specialist

An independent report by
Alan D. Thompson
LifeArchitect.ai
September 2023
18 pages incl title page, references, appendix.

Read more…


 

Summary

Organization Amazon
Model name Olympus
Internal/project name
Model type LLM
Parameter count 2T
Dataset size (tokens) Alan expects: 40T (around 80TB). See also: Gemini
Training data end date Alan expects: Oct/2023
Training start date Alan expects: Nov/2023
Training end/convergence date Alan expects: May/2024
Training time (total) ~25,000 H100s/A100s for ~120 days (compare with GPT-4 @ ~25,000 A100s for ~90 days, and GPT-3 @ ~1,024 A100s for 34 days)
Release date (public) Alan expects: Aug/2024
Paper
Playground

Mythology

2nd gen: Titan: the Titans were second-generation deities who lived on Mount Othrys. The Titans were gigantic compared to the Olympians.

3rd gen: Olympus: the Olympians were third-generation gods who occupied Mount Olympus. The Olympians outnumbered the Titans which resulted in their victory. (source)

2023-2024 optimal language model size highlights

Download source (PDF)
Permissions: Yes, you can use these visualizations anywhere, please leave the citation intact.

Models table

Summary of current models: View the full data (Google sheets)
Download PDF version

Olympus Updates

Dataset

Timeline to Olympus

Date Milestone
28/Sep/2023 Titan embeddings released.
8/Nov/2023 Olympus plans leaked.
Next… Olympus…

AI Race

Download source (PDF)
Permissions: Yes, you can use these visualizations anywhere, please leave the citation intact.


AGI

Read more about Alan’s conservative countdown to AGI


Get The Memo

by Dr Alan D. Thompson · Be inside the lightning-fast AI revolution.
Thousands of paid subscribers. Readers from Microsoft, Tesla, Google AI...
Artificial intelligence that matters, as it happens, in plain English.
Get The Memo.

Dr Alan D. Thompson is an AI expert and consultant, advising Fortune 500s and governments on post-2020 large language models. His work on artificial intelligence has been featured at NYU, with Microsoft AI and Google AI teams, at the University of Oxford’s 2021 debate on AI Ethics, and in the Leta AI (GPT-3) experiments viewed more than 4.5 million times. A contributor to the fields of human intelligence and peak performance, he has held positions as chairman for Mensa International, consultant to GE and Warner Bros, and memberships with the IEEE and IET. Technical highlights.

This page last updated: 26/Nov/2023. https://lifearchitect.ai/olympus/