Google Pathways (report)

To cite this report: 

Thompson, A. D. (2022). Google Pathways: An Exploration of the Pathways Architecture from PaLM to Parti. https://LifeArchitect.ai/pathways

Further reading

For brevity and readability, footnotes were used in this paper, rather than in-text citations. Additional reference papers are listed below, or please see http://lifearchitect.ai/papers for the major foundational papers in the large language model space.

Pathways System announcement (Pathways blog)

Dean, J. (2021). Introducing Pathways: A next-generation AI architecture.

https://blog.google/technology/ai/introducing-pathways-next-generation-ai-architecture/

Pathways System paper

Barham, P., Chowdhery, A., Dean, J., Ghemawat, S., Hand, S., Hurt, D., Isard, M., Lim, H., Pang, R., Roy, S., Saeta, B., Schuh, P., Sepassi, R., Shafey, L. E., Thekkath, C. A., and Wu, Y. (2022). Pathways: Asynchronous Distributed Dataflow for ML. https://arxiv.org/abs/2203.12533 

PaLM announcement (PaLM blog)

Narang, S. & Chowdhery, A. (2022). Pathways Language Model (PaLM): Scaling to 540 Billion Parameters for Breakthrough Performance.

https://ai.googleblog.com/2022/04/pathways-language-model-palm-scaling-to.html

PaLM paper (includes PaLM-Coder)

Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., Schuh, P., Shi, K., Tsvyashchenko, S., Maynez, J., Rao, A., Barnes, P., Tay, Y., Shazeer, N., Prabhakaran, V., Reif, E., Du, N., Hutchinson, B., Pope, R., Bradbury, J., Austin, J., Isard, M., Gur-Ari, G., Yin, P., Duke, T., Levskaya, A., Ghemawat, S., Dev, S., Michalewski, H., Garcia, X., Misra, V., Robinson, K., Fedus, L., Zhou, D., Ippolito, D., Luan, D., Lim, H., Zoph, B., Spiridonov, A., Sepassi, R., Dohan, D., Agrawal, S., Omernick, M., Dai, A. M., Pillai, T. S., Pellat, M., Lewkowycz, A., Moreira, E., Child, R., Polozov, O., Lee, K., Zhou, Z., Wang, X., Saeta, B., Diaz, M., Firat, O., Catasta, M., Wei, J., Meier-Hellstern, K., Eck, D., Dean, J., Petrov, S., and Fiedel, N. (2022). PaLM: Scaling Language Modeling with Pathways. https://arxiv.org/abs/2204.02311 

Parti paper

Yu, J., Xu, Y., Koh, J. Y., Luong, T., Baid, G., Wang, Z., Vasudevan, V., Ku, A., Yang, Y., Ayan, B. K., Hutchinson, B., Han, W., Parekh, Z., Li, X., Zhang, H., Baldridge, J., & Wu, Y. Scaling Autoregressive Models for Content-Rich Text-to-Image Generation. https://arxiv.org/abs/2206.10789 

Parti demo

Google. (2022). https://parti.research.google/ 

Minerva announcement (Minerva blog)

Dyer, E., & Gur-Ari, G. (2022). Minerva: Solving Quantitative Reasoning Problems with Language Models.

https://ai.googleblog.com/2022/06/minerva-solving-quantitative-reasoning.html 

Minerva paper

Lewkowycz, A., Andreassen, A., Dohan, D., Dyer, E., Michalewski, H., Ramasesh, V., Slone, A., Anil, C., Schlag, I., Gutman-Solo, T., Wu, Y., Neyshabur, B., Gur-Ari, G., & Misra, V. (2022). Solving Quantitative Reasoning Problems with Language Models. https://arxiv.org/abs/2206.14858

Minerva sample demo

Google. (2022). https://minerva-demo.github.io/