OpenAI o3 visual models 👓, Mistral Classifier Factory 🔍, Goodfire raises $50M 💰

OpenAI's latest visual models can reason with images through tool-augmented transformations, enabling a new level of multimodal understanding ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌  ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ 

Sign Up |Advertise|View Online TLDR Together With TLDR AI 2025-04-18 Is there a sustainable way to monetize AI? (Sponsor) You've built something powerful with AI. Now you're asking: how do we price it? Many startups are getting the answer wrong right now—and putting their business's viability at risk. In Metronome's upcoming webinar, pricing experts from 49 Palms Ventures and Metronome CEO Scott Woody will explore strategies and best practices for monetizing AI. Learn how to: → Find a logical starting point with the 9-step AI pricing framework → Assess and communicate your AI product's value → Create flexible, iterable pricing systems that can evolve with the market Register now to save your spot. 🚀 Headlines & Launches Mistral Classifier Factory (5 minute read) Mistral, a French AI startup, has launched a new product that allows users to very quickly build and deploy custom classifiers for a whole variety of tasks (e.g., spam, moderation, and more). Goodfire raises $50m series A to steer and understand models (5 minute read) Goodfire is a mechanistic interpretability company with strong expertise in SAEs, among other things. It is working closely with closed and open model providers to steer, control, and understand model motivations and behavior. Visual Reasoning with OpenAI o3 and o4-mini (5 minute read) OpenAI's latest visual models can reason with images through tool-augmented transformations, enabling a new level of multimodal understanding and step-by-step visual problem-solving. 🧠 Research & Innovation Efficient Line Art Colorization with Broader References (12 minute read) Novel efficient long-context fine-grained ID preservation framework for line art colorization, achieving high precision, efficiency, and flexible usability for comic colorization. It transforms black-and-white line art into vibrant illustrations by effectively integrating extensive contextual references. Scene Captioning (14 minute read) 3D CoCa is a unified framework that combines vision-language contrastive learning and captioning for 3D scenes. Large Reasoning Models as a Judge (8 minute read) JudgeLRM is a family of LLMs trained with reinforcement learning for judgment tasks. Unlike SFT, it excels in reasoning-heavy evaluations, outperforming models like GPT-4 and DeepSeek-R1. 🧑‍💻 Engineering & Resources DeepSpeed's DeepCompile (GitHub Repo) The DeepSpeed team has worked to bring compilation to their distributed training efforts. This compilation speeds up various bottlenecked operations by many times. It makes use of a patched version of torch compile. Speech Instruction Fine-Tuning Dataset (Hugging Face Hub) SIFT-50M (Speech Instruction Fine-Tuning) is a 50-million-example dataset designed for instruction fine-tuning and pre-training of speech-text large language models (LLMs). It is built from publicly available speech corpora containing a total of 14K hours of speech and leverages LLMs and off-the-shelf expert models. The dataset spans five languages, covering diverse aspects of speech understanding and controllable speech generation instructions. SIFT-50M augments existing speech datasets with instruction-based question-answer (QA) pairs for speech understanding and includes approximately 5 million examples for controllable speech generation. End-to-End Latent Diffusion Training with REPA-E (3 minute read) REPA-E enables stable, joint training of VAEs and latent diffusion models using a representation-alignment loss, achieving state-of-the-art results on ImageNet. 🎁 Miscellaneous Meta Releases Many New Artifacts (12 minute read) Meta has released an image Encoder, a VLM, a 3D object localization model based on JEPA, and weights for a BLT model that operates directly on bytes without tokenization. Hugging Face Inference Supports Cohere Models (9 minute read) Cohere became the first model creator to directly host and serve its enterprise-focused AI models on Hugging Face. Create AI-generated soundtrack in Shorts with Dream Track (3 minute read) YouTube's Dream Track is now available in the U.S. on YouTube Shorts and the YouTube Create app, enabling AI-generated instrumental soundtracks for content creators. These tracks can be remixed globally to create unique Shorts, fostering a collaborative ecosystem. The feature integrates directly with YouTube's creation tools and adheres to community guidelines. ⚡ Quick Links OpenAI Flex Processing (2 minute read) OpenAI has introduced Flex processing, a cost-saving API option that trades slower response times and intermittent availability for lower prices, ideal for non-production tasks. Conversational AI for Cells (1 minute read) C2S-Scale is a new family of LLMs that interprets single-cell data and translates biological signals into natural language for applications in personalized medicine and drug discovery. Anthropic enhances Claude with Research and Google Workspace integration (4 minute read) Anthropic has launched new Claude features: Research for autonomous multi-step search with citations and Google Workspace integration for context-aware assistance. Love TLDR? Tell your friends and get rewards! Share your referral link below with friends to get free TLDR swag! https://refer.tldr.tech/0b6a6dc1/2 Track your referrals here. Want to advertise in TLDR? 📰 If your company is interested in reaching an audience of AI professionals and decision makers, you may want to advertise with us. Want to work at TLDR? 💼 Apply here or send a friend's resume to jobs@tldr.tech and get $1k if we hire them! If you have any comments or feedback, just respond to this email! Thanks for reading, Andrew Tan, Ali Aminian & Andrew Carr Manage your subscriptions to our other newsletters on tech, startups, and programming. Or if TLDR AI isn't for you, please unsubscribe.