30 Years of Postgres 🐘, Spotify Skips Bayesian 🎧, Kestra 2.0 Ships 🚀

Tom Lane, a committer for 25 of Postgres' 30 years, walks through the architectural bets that held: process-per-connection for code simplicity ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌  ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ 

Sign Up |Advertise|View Online TLDR Together With TLDR Data 2026-09-10 CostBench: 100 billion rows. Up to 83% lower read costs. (Sponsor) Fast queries are only part of your analytics bill. Ingesting and preparing fresh data costs money too. CostBench measures ingest, preparation, and reads. We streamed 1 million stock-market quotes per second, reaching 100 billion rows, with dashboard and drill-down queries running throughout. Against traditional cloud data warehouses we tested, ClickHouse delivered: Up to 82% lower ingest costs Up to 65% lower preparation costs Up to 83% lower read costs Open-source code. Full methodology. Public results. Inspect the numbers and reproduce the benchmark. Try ClickHouse Cloud on your own data. Start with $300 in free trial credits. Sign up for free → 📱 Deep Dives Tom Lane on the Architectural Decisions That Shaped 30 Years of Postgres (41 minute read) Tom Lane, a committer for 25 of Postgres' 30 years, walks through the architectural bets that held: process-per-connection for code simplicity, write-ahead logging in 8.0 that turned it from a plaything into a serious database, and MVCC pushing maintenance work into background vacuum. He credits Berkeley's liberal license for the project surviving where its contemporaries did not, names version 13 the buggiest release, and says he would delete partitioning given the chance. Built for reliability: How American Express processes payments at scale (12 minute read) American Express uses cells as complete payment-processing failure domains: microservices, databases, DNS, and reference data are local, while dynamic transactions are routed to the cell that owns their state. The design trades global coordination for controlled failure scope and deliberately rejects transactions when consistency cannot be established. Cloudfloe's query engine in a celld cell (9 minute read) Cloudfloe is a web interface for querying Iceberg data on S3 with DuckDB. This experiment tested whether a stateful, serverless-style cell could replace a per-query container: DuckDB WebAssembly read a 37,537-row Iceberg table in 148 ms, but cold or woken cells took 250–320 ms and retained most of their memory. Keeping the connection warm cut repeat queries to about 3 ms, making lifecycle policy, not raw startup, the core design tradeoff. 🚀 Opinions & Advice The AI Data Sprawl (12 minute read) Non-technical staff now write billions of lines of analytics code. The problem is where it lives: random context, porous permissions, and output scattered across one-off chats, Slack pastes, and someone's laptop. With no record of which table an agent grabbed or which filter it dropped, the only feedback loop left is people disagreeing. 5 CDC tools and the tradeoffs you should know (14 minute read) CDC evaluation should start with failure behavior, not connector counts. Compare capture method, initial snapshots and backfills, schema evolution, delivery semantics, observability, deployment model, and who owns recovery. A production proof needs representative data and a broken-destination test, especially for type fidelity and duplicate handling. Jitter is the cheapest reliability fix you are not using (5 minute read) Distributed systems synchronise themselves without being told to: shared events such as deploys, cache flushes, or connection loss turn independent workloads into recurring load spikes. Add jitter to initial retries, TTL refreshes, heartbeats, token renewals, and scheduled work. This preserves average load while preventing retry storms from repeatedly hitting a recovering dependency in lockstep. 💻 Launches & Tools Kestra 2.0: A new engine, workers anywhere, and flows as agent tools (16 minute read) Kestra is an open-source orchestration platform for scheduling and operating data, infrastructure, and business workflows. Its 2.0 LTS release separates control and data planes so workers can run across regions, clouds, or private networks over outbound-only connections. It also exposes flows as MCP tools for agents, decouples queue and storage choices, and includes migration tooling for 1.x users. Filament (GitHub Repo) Filament replicates data from sources to sinks with full, incremental, or CDC modes, verifying every bounded batch on both sides of the write and advancing checkpoints only after success. Sources, sinks, state storage, and event transport are all swappable. JupyterGIS 0.16: Collaborative story maps and remote geospatial workflows (3 minute read) JupyterGIS 0.16 adds real-time collaborative story maps, lazy tile rendering for remote openEO workflows, and native GeoZarr and GeoPackage support. Declarative symbology and Xarray tiling keep visualizations reproducible while allowing large geospatial datasets to stay remote instead of being materialized locally. 🎁 Miscellaneous Why Spotify is not using Bayesian A/B testing (12 minute read) Bayesian experimentation is not one method but a choice of prior, likelihood, and stopping rule. Spotify shows that common flat-prior posterior thresholds can reproduce frequentist peeking, while Bayes factors can control false positives under optional stopping. Choose guarantees for the experiment program first, then the inference configuration. Pretraining progress is mostly coming from data (11 minute read) A controlled comparison of 2019-2025 open-model recipes and data corpora attributes 12.0x compute-efficiency gains to data improvements versus 3.7x to model recipes at a 1e19-FLOP budget. The result frames dataset extraction, curation, and filtering as a major systems lever, while cautioning that larger-scale and synthetic-data effects remain untested. ⚡ Quick Links xlDuckDb (GitHub Repo) xlDuckDb adds a DuckDbQuery() function to Excel that runs DuckDB SQL and spills results into cells, querying worksheet ranges, tables, and local or remote JSON, CSV, and Parquet in one statement. TPC-H Query 15 - Scalar Constants and Materialisation (5 minute read) Referencing the same CTE twice tests whether the optimiser materialises the result. Love TLDR? Tell your friends and get rewards! Share your referral link below with friends to get free TLDR swag! https://sparklp.co/32815a84/11 Track your referrals here. Want to advertise in TLDR? 📰 If your company is interested in reaching an audience of data engineering professionals and decision makers, you may want to advertise with us. Want to work at TLDR? 💼 Apply here, create your own role or send a friend's resume to jobs@tldr.tech and get $1k if we hire them! TLDR is one of Inc.'s Best Bootstrapped businesses of 2025. If you have any comments or feedback, just respond to this email! Thanks for reading, Joel Van Veluwen, Tzu-Ruey Ching & Remi Turpaud Manage your subscriptions to our other newsletters on tech, startups, and programming. Or if TLDR Data isn't for you, please unsubscribe.