AI Report Card

Designed in July 2022, the report card provides a standard template for assessing new large language models and multimodal models.

Download annotated papers (latest first)

Download report cards (latest first)

Model Lab Date Grade/ALScore Download
Llama 2 70B Meta AI Jul/2023 B+ (1.2) Llama 2 report card (PDF)
PaLM 2 340B Google May/2023 PX (3.7) PaLM 2 report card (PDF)
Updated to show poor performance on SuperGLUE=86.4%
GPT-4 1T OpenAI Mar/2023 PX (14.9) GPT-4 report card (PDF)
LLaMA-65B Meta AI Feb/2023 B- (1.0) LLaMA-65B report card (PDF)
Galactica 120B Meta AI Nov/2022 B- (0.8) GAL 120B report card (PDF)
AlexaTM 20B Amazon Alexa AI Aug/2022 C (0.5) Amazon AlexaTM 20B report card (PDF)
PaLM 540B Google Jul/2022 A (2.2) Google PaLM report card (PDF)
InstructGPT OpenAI Jul/2022 B- (-) OpenAI InstructGPT report card (PDF)


Report Card Template (PDF)

The report card template is open source, and available for download here.

Zoomed preview


Get The Memo

by Dr Alan D. Thompson · Be inside the lightning-fast AI revolution.
Thousands of paid subscribers. Readers from Microsoft, Tesla, Google AI...
Artificial intelligence that matters, as it happens, in plain English.
Get The Memo.

Dr Alan D. Thompson is an AI expert and consultant, advising Fortune 500s and governments on post-2020 large language models. His work on artificial intelligence has been featured at NYU, with Microsoft AI and Google AI teams, at the University of Oxford’s 2021 debate on AI Ethics, and in the Leta AI (GPT-3) experiments viewed more than 3.5 million times. A contributor to the fields of human intelligence and peak performance, he has held positions as chairman for Mensa International, consultant to GE and Warner Bros, and memberships with the IEEE and IET. He is open to consulting and advisory on major AI projects with intergovernmental organizations and enterprise.

This page last updated: 19/Jul/2023.