FOR IMMEDIATE RELEASE: May/2022
Welcome back to The Memo,
The BIG Stuff
>9 new large language models released (Apr/2022)
I documented 9 new language models via Reddit, where a new model was released every 3-4 days during a 30-day period in March and April 2022. This has continued into May 2022.
Meta AI releases OPT-175B (3/May/2022)
Meta AI has emulated OpenAI’s GPT-3. While not particularly useful for anyone but researchers (there is a non-commercial licence, strict authorization process for universities only, and it requires 16x NVIDIA 32GB V100 GPUs to run!), this is a big advance for openness.
DeepMind releases new models (13/May/2022)
DeepMind in London released Flamingo 80B a few weeks ago (see my video about Flamingo 80B), and Gato (Cat) 1.18B today. Gato combines language (LLM) with vision (VLM) with control tasks for robotics. The generalist model uses the language model training optimisation from Chinchilla 70B, and I estimate that it only has ~0.1B parameters for language (trained on 6.7% language via MassiveText, while most of the remaining training data is images and control, see pp6 of the Gato paper). The model is a step further towards full AGI, as it can (for example) “play Atari, caption images, chat, stack blocks with a real robot arm, and much more”.
The Interesting Stuff
China + ERNIE 3.0 (10B, Apr/2022)
As most of the Western world sleeps when it comes to applied AI, China brought together some of ERNIE 3.0’s 60,000 devs… The event drew over 2,000 contestants, collected over 300 creative applications of Wenxin, spanning across industries such as education, healthcare, entertainment, technology, and mental health. In the final round, three projects were selected as winners: “Shuowen”, which helps users interpret traditional Chinese readings, “Tuyan”, creating various styles of literature based on pictures, and “AI essay title generator”, a project by Bilibili content creator Zihao, generating essay titles based on 250-words summaries. Check out China’s big push for AI in Apr/2022.
Google introduces LaMDA 2 (12/May/2022)
I was disappointed by Google’s announcement of LaMDA 2 at Google I/O 2022 yesterday (watch it here). Google propose allowing tiny access to this model, where access is controlled by topic (talk about dogs only), function (create filtered lists only), or preset questions (click a preset question button to talk about imaginary places). In many ways, Google AI seems to be much worse than OpenAI when it comes to access to large language model technology.
Toys to Play With
AI interview with Cris Sheridan for Financial Sense® Newshour (Apr/2022)
The Financial Sense® podcast is a $20/month premium subscription. Thanks to Cris and team, my Apr/2022 interview about all things AI is provided to you at no fee! Listen to the AI interview.
Read a tech-noir story by my friend in Germany + GPT-3 (2021)
I LOVED reading this story by a Leta AI viewer in Germany, using the massive GPT-3 language model. His pen name is Derek Beauregard. It’s the length of a proper book, though unedited. This is an exclusive, he is not releasing this publicly, and there is nothing like this in the wild (or on Amazon) yet. Thank you so much to D for releasing this book to the Memo readers at no fee! Fall in love with Lucy and friends in Terminal City Chronicles (PDF).
I used Quickchat.ai’s Emerson AI for the first 36 episodes of Leta AI. It is a phenomenal app, and combines GPT-3 with additional ‘smarts’ including image recognition. The founder of Quickchat.ai, Piotr, has offered a code to readers of The Memo.
2. Select “Continue with Telegram”
3. After the free period, you can go ahead and purchase a subscription as you wish by sending a “Subscription” or “My account” message to Emerson.
4. Code can be redeemed from the Stripe checkout window by clicking on the “Add promo code” button. Your 20% off code is: SUBSCRIBE-TO-REVEAL
Something big is coming. We’re still waiting on OpenAI to release GPT-4. In the interim, the amazing DALL-E 2 visual model (using diffusion instead of GAN) has been stirring up the AI world. Check out my video about this book, designed by OpenAI’s former creative director, Ben Barry. Each image was generated by AI in less than a second (10 images in about 10 seconds), and the results are breathtaking…
I’ll be back shortly with all things AI.