The Memo.
Alan D. Thompson
July 2023
Details about Amazon Titan are scarce.
Here’s what we know about Amazon Titan (not very much):
Model names:
- Text generation: amazon.titan-tg1-large
- Embeddings: amazon.titan-e1t-medium
Summary
| Organization | Amazon |
| Model name | Titan |
| Internal/project name | amazon.titan-tg1-large |
| Model type | Dense |
| Parameter count | 200B |
| Dataset size (tokens) | 4T (4,000B) |
| Training data end date | Undisclosed |
| Convergence date | Estimate Feb/2023 |
| Release date (public) | 13/Apr/2023 (large clients only) |
| Annotated paper | – |
| Playground | Convoluted application process: https://aws.amazon.com/bedrock/titan/ |
Titan updates
7/May/2024: Amazon Titan Text Premier benchmarks. MMLU=70.5. (Amazon)
18/Mar/2024: Amazon Titan = 200B trained on 4T tokens.
Amazon can see $1 billion training runs on the horizon:
…Technical talk from a longtime AWS person sheds light on frontier AI training…
James Hamilton, a distinguished engineer at Amazon, said at a talk this year that within the last year Amazon carried out a $65m training run. Specifically, they trained a 200B dense model on 4T tokens of data across 13,760 NVIDIA A100 chips (using 1,720 P4d nodes). It took 48 days to train. Hamilton described this training run as “1 gen old” so we can assume Amazon has moved on to larger runs since then. Looking ahead, Hamilton said “training runs soon to cross $1b”. (18/Mar/2024, via Jack Clark)
29/Aug/2023: Amazon Titan news on Amazon Science.
24/Jul/2023: Amazon Titan reference (‘amazon.titan-tg1-large’) spotted in the wild on GitHub.
22/Jun/2023: AWS Launches New $100M Generative AI Innovation Center (link).
13/Apr/2023: Amazon announces ‘Bedrock’ AI platform to take on OpenAI (BI).
2/Aug/2022: Amazon AlexaTM 20B models (arXiv paper).
Get The Memo
by Dr Alan D. Thompson · Be inside the lightning-fast AI revolution.Informs research at Apple, Google, Microsoft · Bestseller in 147 countries.
Artificial intelligence that matters, as it happens, in plain English.
Get The Memo.
Alan D. Thompson is a world expert in artificial intelligence, advising everyone from Apple to the US Government on integrated AI. Throughout Mensa International’s history, both Isaac Asimov and Alan held leadership roles, each exploring the frontier between human and artificial minds. His landmark analysis of post-2020 AI—from his widely-cited Models Table to his regular intelligence briefing The Memo—has shaped how governments and Fortune 500s approach artificial intelligence. With popular tools like the Declaration on AI Consciousness, and the ASI checklist, Alan continues to illuminate humanity’s AI evolution. Technical highlights.This page last updated: 26/May/2024. https://lifearchitect.ai/titan/↑
