February Roundup
Hi, and welcome to February’s monthly roundup!
This is the place that I share some of my current work and reading.
Biden and Swift Deepfakes
Explicit deepfake images of Taylor Swift went viral on social media, showing how the generative AI is developing fast and becoming more accessible. As the technology improves, it is used against women and girls. The same technology is also used to influence political agendas - an artificial version of Biden’s voice was used to urge voters not to cast their votes.
Microsoft’s tool was identified as the source of the Taylor Swift images, and it swiftly made updates to prevent the same in the future. Twitter blocked searches for “Taylor Swift” to limit the spread. But, these show how keeping up and proactively preventing misuse is a difficult task for tech companies who have broad and general platforms.
New open source language models
The LLM arms race continues! The Allen Institute for AI just released OLMo, a new open-source LLM. The release includes the Dolma dataset it was trained on, code, model checkpoints, and evaluation code. The full release means that researchers can easily build on this model without needing to do a lot of their own work to get started. The release on HuggingFace has two 7B parameter versions, and a 1B parameter model. Just a week or so later, Cohere released Aya, which is a multilingual model complete with instruction tuning dataset, covering 101 languages worldwide. NLP research is typically focused in a few languages, and so a multilingual set is a great contribution for moving beyond just the ‘main’ languages.
Google are also in on the action. The next version of Google’s Gemini model is set to have a 1M token context window, and they released Gemma - a model with 2B and 7B parameter versions, under a licence that permits commercial use.
OpenAI’s Sora
OpenAI announced Sora, their text-to-video generation tool. It’s not publicly available yet, but they shared some examples. In my opinion, the quality is great, but they’re firmly in uncanny valley territory - complete with weird limbs and physics-defying movement.
2024 coaching program
I’m finishing up my qualification in coaching, and launching a new coaching program for 2024 aimed at tech leaders and AI teams. Take a look at my website to learn more and sign up.
What I’ve been reading
The EU AI Act is signed
Tackling Taboo Topics: A Review of the Three Ms in Working Women’s Lives
Allen Institute for AI releases ‘truly open source’ LLM to drive ‘critical shift’ in AI development
How to avoid machine learning pitfalls: a guide for academic researchers
Why the European AI Act transparency obligation is insufficient
The Curse of Recursion: Training on Generated Data Makes Models Forget