Get The Memo.
Waiting for Amazon Olympus? Check out Google DeepMind Gemini…
Google DeepMind Gemini: A general specialist
An independent report by
Alan D. Thompson
LifeArchitect.ai
September 2023
18 pages incl title page, references, appendix.
Summary
Organization | Amazon |
Model name | Olympus |
Internal/project name | – |
Model type | LLM |
Parameter count | 2T |
Dataset size (tokens) | Alan expects: 40T (around 80TB). See also: Gemini |
Training data end date | Alan expects: Oct/2023 |
Training start date | Alan expects: Nov/2023 |
Training end/convergence date | Alan expects: May/2024 |
Training time (total) | ~25,000 H100s/A100s for ~120 days (compare with GPT-4 @ ~25,000 A100s for ~90 days, and GPT-3 @ ~1,024 A100s for 34 days) |
Release date (public) | Alan expects: Aug/2024 |
Paper | – |
Playground | – |
Mythology
2nd gen: Titan: the Titans were second-generation deities who lived on Mount Othrys. The Titans were gigantic compared to the Olympians.
3rd gen: Olympus: the Olympians were third-generation gods who occupied Mount Olympus. The Olympians outnumbered the Titans which resulted in their victory. (source)
2023-2024 optimal language model size highlights

Permissions: Yes, you can use these visualizations anywhere, please leave the citation intact.
Models table
Summary of current models: View the full data (Google sheets)Download PDF version
Olympus Updates
Dataset
Timeline to Olympus
Date | Milestone |
28/Sep/2023 | Titan embeddings released. |
8/Nov/2023 | Olympus plans leaked. |
Next… | Olympus… |
AI Race
Download source (PDF)
Permissions: Yes, you can use these visualizations anywhere, please leave the citation intact.
AGI
Read more about Alan’s conservative countdown to AGI…
Get The Memo
by Dr Alan D. Thompson · Be inside the lightning-fast AI revolution.Thousands of paid subscribers. Readers from Microsoft, Tesla, Google AI...
Artificial intelligence that matters, as it happens, in plain English.
Get The Memo.

This page last updated: 26/Nov/2023. https://lifearchitect.ai/olympus/↑