Google Pathways (report)

An Exploration of the Pathways Architecture from PaLM to Parti

Alan D. Thompson
August 2022
24 pages incl title page, references, appendices.

Download PDF (2.3MB).

Updates to the Pathways family since publication (most recent at top)

Date Model Notes
13/Oct/2023 PaLI-3 PaLI-3: paper.
26/Jul/2023 Med-PaLM M Med-PaLM Multimodal: paper.
22/Jun/2023 AudioPaLM AudioPaLM = PaLM 2 + AudioLM: paper.
10/May/2023 PaLM 2 PaLM 2: announce.
29/May/2023 PaLI-X 55B PaLI-X 55B: paper.
10/May/2023 PaLM 2 PaLM 2: announce.
12/Mar/2023 Med-PaLM 2 Med-PaLM 2: announce.
6/Mar/2023 PaLM-E PaLM-E is PaLM Embodied with 562B params: paper, research site.
19/Jan/2023 Dr Jeff Dean provided extended commentary in his review of 2022: blog.
26/Dec/2022 Med-PaLM Med-PaLM, a medical finetuned model based on Flan-PaLM: paper.
20/Oct/2022 Flan-PaLM Flan-PaLM, based on Finetuning language models (Flan): paper.
20/Oct/2022 U-Palm U-PaLM, a version of PaLM using less power/hours of compute: paper.
15/Sep/2022 PaLI 17B PaLI 17B: Google Pathways Language and Image model: paper.
16/Aug/2022 PaLM-SayCan PaLM + Robots: Google PaLM-SayCan: announce, research site, video.

Received by several major governments; used in policy analysis.


With over a million subscribed users, GPT-3 and related models have received a lot of press coverage and public attention. Much like a flashy Porsche driving down the Autobahn, these models look impressive, and are performing well. However, it is only a matter of time until they are overtaken by a much larger supercar. And that vehicle is already rapidly approaching. Google Pathways was announced at the end of 2021, and we are seeing several of its components in 2022: beginning with PaLM, PaLM-Coder, Parti, and Minerva. While all of these models are closed—only available for Google’s internal research—it is anticipated that a future Pathways model will be publicly released. This report explores the accomplishments of the Pathways models so far, with PaLM and its related language models already at more than triple the size of GPT-3.


1. Overview
1. Google Pathways (Oct/2021)
2. Google Pathways: The Pathways System (Mar/2022)
3. Google Pathways: PaLM 540B (Apr/2022)
3.1. PaLM Dataset Summary
3.2. PaLM Capabilities & Performance
4. Google Pathways: PaLM-Coder 540B (Apr/2022)
5. Google Pathways: Parti 20B (Jun/2022)
6. Google Pathways: Minerva 540B (Jun/2022)
6.1. The Polish National Math Exam
7. Following the Trail to Transformative AI
8. Further reading
Appendix A: PaLM 540B vs GPT-3 175B vs Jurassic-1 178B vs human

More videos and images


1. PaLM 540B (Apr/2022)

2. Parti 20B (Jun/2022)

3. Report card referencing PaLM

Images (selected)

References, Further Reading, and How to Cite

To cite this report: 

Thompson, A. D. (2022). Google Pathways: An Exploration of the Pathways Architecture from PaLM to Parti.

Further reading

For brevity and readability, footnotes were used in this paper, rather than in-text citations. Additional reference papers are listed below, or please see for the major foundational papers in the large language model space.

Pathways System announcement (Pathways blog)

Dean, J. (2021). Introducing Pathways: A next-generation AI architecture.

Pathways System paper

Barham, P., Chowdhery, A., Dean, J., Ghemawat, S., Hand, S., Hurt, D., Isard, M., Lim, H., Pang, R., Roy, S., Saeta, B., Schuh, P., Sepassi, R., Shafey, L. E., Thekkath, C. A., and Wu, Y. (2022). Pathways: Asynchronous Distributed Dataflow for ML. 

PaLM announcement (PaLM blog)

Narang, S. & Chowdhery, A. (2022). Pathways Language Model (PaLM): Scaling to 540 Billion Parameters for Breakthrough Performance.

PaLM paper (includes PaLM-Coder)

Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., Schuh, P., Shi, K., Tsvyashchenko, S., Maynez, J., Rao, A., Barnes, P., Tay, Y., Shazeer, N., Prabhakaran, V., Reif, E., Du, N., Hutchinson, B., Pope, R., Bradbury, J., Austin, J., Isard, M., Gur-Ari, G., Yin, P., Duke, T., Levskaya, A., Ghemawat, S., Dev, S., Michalewski, H., Garcia, X., Misra, V., Robinson, K., Fedus, L., Zhou, D., Ippolito, D., Luan, D., Lim, H., Zoph, B., Spiridonov, A., Sepassi, R., Dohan, D., Agrawal, S., Omernick, M., Dai, A. M., Pillai, T. S., Pellat, M., Lewkowycz, A., Moreira, E., Child, R., Polozov, O., Lee, K., Zhou, Z., Wang, X., Saeta, B., Diaz, M., Firat, O., Catasta, M., Wei, J., Meier-Hellstern, K., Eck, D., Dean, J., Petrov, S., and Fiedel, N. (2022). PaLM: Scaling Language Modeling with Pathways. 

Parti paper

Yu, J., Xu, Y., Koh, J. Y., Luong, T., Baid, G., Wang, Z., Vasudevan, V., Ku, A., Yang, Y., Ayan, B. K., Hutchinson, B., Han, W., Parekh, Z., Li, X., Zhang, H., Baldridge, J., & Wu, Y. Scaling Autoregressive Models for Content-Rich Text-to-Image Generation. 

Parti demo

Google. (2022). 

Minerva announcement (Minerva blog)

Dyer, E., & Gur-Ari, G. (2022). Minerva: Solving Quantitative Reasoning Problems with Language Models. 

Minerva paper

Lewkowycz, A., Andreassen, A., Dohan, D., Dyer, E., Michalewski, H., Ramasesh, V., Slone, A., Anil, C., Schlag, I., Gutman-Solo, T., Wu, Y., Neyshabur, B., Gur-Ari, G., & Misra, V. (2022). Solving Quantitative Reasoning Problems with Language Models.

Minerva sample demo

Google. (2022).

Image credit: Thanks to jesssaysno for the header image on this page.

Get The Memo

by Dr Alan D. Thompson · Be inside the lightning-fast AI revolution.
Bestseller. 10,000+ readers from 142 countries. Microsoft, Tesla, Google...
Artificial intelligence that matters, as it happens, in plain English.
Get The Memo.

Dr Alan D. Thompson is an AI expert and consultant, advising Fortune 500s and governments on post-2020 large language models. His work on artificial intelligence has been featured at NYU, with Microsoft AI and Google AI teams, at the University of Oxford’s 2021 debate on AI Ethics, and in the Leta AI (GPT-3) experiments viewed more than 4.5 million times. A contributor to the fields of human intelligence and peak performance, he has held positions as chairman for Mensa International, consultant to GE and Warner Bros, and memberships with the IEEE and IET. Technical highlights.

This page last updated: 16/Oct/2023.