A general specialist
An independent report by
Alan D. Thompson
LifeArchitect.ai
September 2023
18 pages incl title page, references, appendix.
The report
Download report (PDF).
(Available exclusively to full subscribers of The Memo.)
Notice: This is a pre-release report made available before the release of Google DeepMind Gemini. While this independent report is based on rigorous analysis of public and private information, some parts of the report rely on determined estimates. For this reason, the pre-release versions of this report should not be used for decision making. A final version of this independent report will be clearly marked, and published after Gemini’s official announcement.
Abstract
Since Google’s discovery of the Transformer architecture in 2017, and subsequent release of their pre-trained transformer language model BERT in October 2018, training large language models (LLMs) has become a new space race, bringing humanity towards its largest evolutionary change yet: ‘superintelligence.’
Between 2020 and 2023, LLMs continued to be trained on increasingly larger datasets, by ever larger teams of data scientists, with compute now measured in the hundreds of millions of dollars.
The information synthesized here covers the progress made by Google and DeepMind, presenting as one company under the Alphabet umbrella in 2023, with a focus on the massive Gemini multimodal model due for release in the US fall (in 2023, this means between 23 September and 21 December).
Contents
1. Background
1.1. Etymology
1.2. Google DeepMind: Two archers with one target
1.3. Gemini personnel and resources
1.4. Large language models
1.5. Text-to-image and visual language models
1.6. The Alpha series of AI systems
1.7. Putting it together: LLM + VLM + Text-to-image + Expertise
2. Datasets
2.1. Datasets: Text: MassiveText multilingual
2.2. Datasets: Visual
2.3. Datasets: Special
3. Gemini capabilities and performance
3.1. Languages
3.2. Visual
3.3. IQ
4. Closing estimates
5. Implementing and applying Gemini
6. Conclusion
7. Further reading
8. Appendix
Get The Memo
by Dr Alan D. Thompson · Be inside the lightning-fast AI revolution.Thousands of paid subscribers. Readers from Microsoft, Tesla, Google AI...
Artificial intelligence that matters, as it happens, in plain English.
Get The Memo.
Dr Alan D. Thompson is an AI expert and consultant, advising Fortune 500s and governments on post-2020 large language models. His work on artificial intelligence has been featured at NYU, with Microsoft AI and Google AI teams, at the University of Oxford’s 2021 debate on AI Ethics, and in the Leta AI (GPT-3) experiments viewed more than 3.5 million times. A contributor to the fields of human intelligence and peak performance, he has held positions as chairman for Mensa International, consultant to GE and Warner Bros, and memberships with the IEEE and IET. He is open to consulting and advisory on major AI projects with intergovernmental organizations and enterprise.This page last updated: 8/Sep/2023. https://lifearchitect.ai/gemini-report/↑