Airbnb’s Evals Approach 🏠, Spotify’s Fast Point Queries 🎵, Versioning Business Knowledge 🧩

Airbnb evaluates GenAI by reviewing 100 prototype outputs to find real failure modes before creating automated evals. It uses programmatic checks ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌  ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ 

Sign Up |Advertise|View Online TLDR Together With TLDR Data 2026-07-30 AI already sounds right. Cube makes sure it is right (Sponsor) You wanted non-specialists to chat with your data, so you connected your LLM to your warehouse. Now your team gets detailed numbers…that don't match how your business defines metrics. 🤦 Sound familiar? Cube is the agentic analytics platform that grounds every AI answer in your governed semantic layer. It ensures agents model, explore, and build reports on definitions you control.  💯 Week-long analysis already comes back in minutes. With Cube, it comes back right.  👀 Brex grounded its AI financial analyst in Cube and took answer accuracy from ~55% to ~90% for 35,000+ customers. Get started → 📱 Deep Dives Eval-driven development: Lessons from evaluating GenAI at scale (9 minute read) Airbnb evaluates GenAI by reviewing 100 prototype outputs to find real failure modes before creating automated evals. Its stack uses programmatic checks, calibrated LLM judges with agreement in the high 80% range against human reviewers. Agentic systems are assessed across tool calls, reasoning steps, and final outputs. Indexing the Data Lake for Online Point Queries (8 minute read) Spotify built an external index mapping keys directly to file offsets and row numbers in unmodified Parquet files, replacing distributed SQL scan chains with parallel ranged reads for lookups across petabyte-scale data lakes. Complementary write-time optimizations (per-key page flushing, ZSTD frame resets, column interleaving, covering index entries) reduce both bytes read and round trips further, making the same files serve both batch analytics and interactive queries. Pinner Progression: Better Use-Case Representation Driving Weekly Active User Growth at Pinterest (13 minute read) Pinterest replaced single-user embeddings with per-user agglomerative clusters over the last 500 engagements, with dynamic cluster count and lifecycle metadata tracking recency and frequency. It feeds retrieval, ranking, and blending layers, with lifecycle stages determining which utility signals get weighted during ranking. The retention benefit is nonlinear and accelerates sharply once users adopt enough distinct use cases. Scaling StreamHub: Transitioning from Kinesis to Kafka for 145 Billion Daily Events (8 minute read) Atlassian moved from Kinesis to Kafka on AWS MSK as traffic rose from 22 billion to 150 billion events a day. Tiered Storage cut costs, but incidents exposed broker, S3, control-plane, and disk limits. Quotas, sharding, spare capacity, backups, and failover were critical. 🚀 Opinions & Advice Your AI Agent Doesn't Know Your Business | Context Layers Explained (31 minute video) AI agents need a context layer that captures business definitions, exceptions, institutional knowledge, and rules. Without it, they will guess when interpreting company data. MotherDuck's upcoming Guides feature stores this context alongside the data and automatically surfaces it to agents, with database-level, personal and organisation-wide scoping, SQL support, and version control. The New Wave of the Streaming Log Technologies (5 minute read) New streaming platforms are dropping Kafka compatibility for different bets: Iggy uses Rust with TCP, QUIC, and HTTP; S2 uses HTTP and object storage for bottomless retention; and OpenData Log replaces partitions with key-based routing. Whether they complement or replace Kafka remains open. How LangChain Built an Agent-First Data Stack (10 minute read) LangChain rebuilt its stack for agent-native analysis, migrating fully off its previous BI tool in six weeks. Hex combines dbt definitions, semantic models, trusted datasets, guides, and LangSmith traces so agents generate reliable SQL while the data team controls modeling and governance. 💻 Launches & Tools Composable canonicals: from tribal data knowledge to versioned artifact (4 minute read) Composable canonicals encode domain knowledge in versioned models. Source layers define transformations and ontology, while business layers merge identities across systems. AI agents use stable concepts and identity rules, not simply more tables. How MVCC and Transactions Work in RocksDB (7 minute read) RocksDB tags keys with increasing sequence numbers, so readers snapshot a consistent view while writers append without blocking. Pessimistic transactions lock on write - optimistic ones check conflicts at commit using pre-hashed mutexes. TiDB and CockroachDB skip native transactions and build their own layers. From Python Script Hell to a Modern Data Integration Framework (11 minute read) Apache SeaTunnel is presented as an alternative to fleets of hand-written ingestion scripts that duplicate connection handling, retries, checkpoints, idempotency, concurrency, and observability. Its Source-Transform-Sink model separates declarative pipeline definitions from execution and centralizes connector/runtime behavior. 🎁 Miscellaneous TLDR is hiring a curator for TLDR Data! (TLDR Curator, ~3 hrs/week) Over 500,000 subscribers read TLDR Data to stay on top of the latest in data science and data engineering. If you work in data and want to help curate it, send your LinkedIn or resume to data@tldr.tech! The Real AI Risk is Inside the Labs (2 minute read) Frontier AI labs are more likely to cause the first serious AI incident than open models, since closed systems are vulnerable to insider leaks. Current open models also lack the domain depth to pose immediate risk in fields like biology, and restricting access helps malicious actors more than defenders. The 2026 Guide to Agent Observability Tools (20 minute read) Agent observability has moved from nice-to-have to a production requirement: Monte Carlo cites 73% of enterprises refusing to ship agents without monitoring and alerting, and 53% expecting major redesigns after deployment. The guide separates tracing-first, eval-first, gateway, and platform approaches, but highlights a persistent blind spot: most tools observe the agent while missing upstream data freshness, lineage, and quality failures. ⚡ Quick Links How to Consolidate Your Database Stack for Production AI (10 minute read) CockroachDB argues for consolidating transactional data, ephemeral state, and embeddings under serializable SQL, row-level TTL, changefeeds, and distributed vector indexing. Wide Column Stores Explained: Cassandra, Bigtable, and ScyllaDB (9 minute read) A clear refresher on when wide-column stores fit firehose workloads and predictable single-partition reads, and why they fail for ad hoc SQL or joins. Love TLDR? Tell your friends and get rewards! Share your referral link below with friends to get free TLDR swag! https://sparklp.co/32815a84/11 Track your referrals here. Want to advertise in TLDR? 📰 If your company is interested in reaching an audience of data engineering professionals and decision makers, you may want to advertise with us. Want to work at TLDR? 💼 Apply here, create your own role or send a friend's resume to jobs@tldr.tech and get $1k if we hire them! TLDR is one of Inc.'s Best Bootstrapped businesses of 2025. If you have any comments or feedback, just respond to this email! Thanks for reading, Joel Van Veluwen, Tzu-Ruey Ching & Remi Turpaud Manage your subscriptions to our other newsletters on tech, startups, and programming. Or if TLDR Data isn't for you, please unsubscribe.