Baidu ERNIE (百度文心一言)

👋 Hi, I’m Alan. I advise government and enterprise on post-2020 AI like OpenAI GPT-4 and Google PaLM. You definitely want to keep up with the AI revolution this year. My paid subscribers (NASA, Microsoft, Google…) receive bleeding-edge and exclusive insights on AI as it happens.
Get The Memo.

Alan D. Thompson
March 2023, October 2023

Language model sizes

2024 optimal LLM highlights

Download source (PDF)
Permissions: Yes, you can use these visualizations anywhere, please leave the citation intact.

Summary

    • ERNIE is a multimodal model by Baidu. ERNIE stands for ‘Enhanced Representation from kNowledge IntEgration’. The Chinese name is 文心一言, or Wenxin Yiyan. – via MIT (16/Mar/2023).
    • 2022: Baidu spent total US$3.4 billion on research and development.
    • 650 companies have signed up to use ERNIE Bot (including Sina Finance and Baixin Bank), which can be implemented in a variety of applications such as searches, AI cloud, autonomous driving and in smart devices. – via WP (16/Mar/2023)
    • ERNIE Bot was released on 16/Mar/2023. Invitations to the event were written by ERNIE Bot (via Baidu).
    • ERNIE Bot = ERNIE Titan 260B + Plato-XL 11B.
  • Five major abilities:
      • Literary creation.
      • Commercial copywriting.
      • Mathematical logic calculation.
      • Chinese comprehension.
      • Multi-modal generation.

ERNIE Timeline

Papers

ERNIE 1.0 (2019): https://arxiv.org/abs/1904.09223
ERNIE 2.0 (2019): https://arxiv.org/abs/1907.12412
ERNIE 3.0 Titan 260B (2021): https://arxiv.org/abs/2112.12731
ERNIE-ViLG 2.0 (2022): https://arxiv.org/abs/2210.15257
ERNIE Bot (2023): Parameters not disclosed, based on ERNIE Titan 260B + Plato-XL 11B.
ERNIE 3.5 (2023): Retrieval capability only. Paper TBA.
ERNIE 4.0 (2023): Dense, >1T parameters. Paper TBA.

ERNIE Bot was trained on the Baidu Kunlun II AI Chip (equivalent to NVIDIA A100) produced by Samsung.

Demo

https://www.cnet.com/videos/baidu-unveils-ernie-ai-bots-multimodal-creation-capabilities/

Get The Memo

by Dr Alan D. Thompson · Be inside the lightning-fast AI revolution.
Bestseller. 10,000+ readers from 142 countries. Microsoft, Tesla, Google...
Artificial intelligence that matters, as it happens, in plain English.
Get The Memo.

Dr Alan D. Thompson is an AI expert and consultant, advising Fortune 500s and governments on post-2020 large language models. His work on artificial intelligence has been featured at NYU, with Microsoft AI and Google AI teams, at the University of Oxford’s 2021 debate on AI Ethics, and in the Leta AI (GPT-3) experiments viewed more than 4.5 million times. A contributor to the fields of human intelligence and peak performance, he has held positions as chairman for Mensa International, consultant to GE and Warner Bros, and memberships with the IEEE and IET. Technical highlights.

This page last updated: 15/Jan/2024. https://lifearchitect.ai/ernie/↑