Zero-downtime Postgres upgrades with logical replication (25 minute read)
Logical replication offers a practical path to near-zero-downtime PostgreSQL major upgrades, with live rollback enabled by reverse replication. A Docker-based demonstration ran old and new clusters in parallel, freezing writes and waiting for replication to catch up before cutover. It achieved 181 ms of write interruption during upgrade and 179 ms during rollback, without data loss.
|
CASTLE: training natural language search before the first real query (6 minute read)
Cold-start natural-language search often lacks both real queries and relevance labels. Airbnb's CASTLE jointly generates synthetic queries and labels from de-identified booking sessions. Its queries matched real guest query patterns much better than baselines, and human reviewers agreed with its relevance labels >90% of the time. Carefully governed behavioral data can bootstrap search training before meaningful traffic exists.
|
|
Four PostgreSQL arguments that came out even (2 minute read)
PostgreSQL micro-optimization alternatives often don't matter. For 1,000 IDs, IN, ANY, and VALUES joins perform similarly; prefer parameterized ANY for stable query text. Short text, enums, and smallints use near-identical storage; mostly-NULL columns are cheap. Don't blindly rewrite but EXPLAIN on your own data.
|
Rewinding the exponential: How we keep hardware usage down (9 minute read)
Cloudflare held back exponential hardware growth by shrinking data earlier and moving duplication later: split monolithic schemas, switch to zstd, send each event once then replicate after sampling, sample at the edge, and pre-aggregate known queries. Optimize bytes, not just compute: prune fields, compress measured workloads, push reductions upstream, and replicate compacted data.
|
|
SQLBuild: safe refactoring for warehouse SQL (GitHub Repo)
SQLBuild is an open-source SQL and Python transformation framework for safely changing warehouses. It checks contracts and lineage before execution, runs graph-level tests and production diffs, and lets teams rename incremental models or columns while preserving history and compatibility views. A local DuckDB playground makes it easy to evaluate. Snowflake, Postgres, BigQuery, and Databricks adapters are available.
|
Phoenix (GitHub Repo)
Phoenix is a self-hosted, open-source AI observability platform for tracing, evaluating, and debugging LLM applications. It captures OpenTelemetry traces; offers retrieval and response evaluations, versioned datasets, and experiments; and supports prompt testing and replay. Framework- and provider-agnostic instrumentation makes it useful for teams that need to diagnose agent behavior and measure changes before production.
|
XERJ (GitHub Repo)
XERJ is an Apache-2.0, Elasticsearch-compatible local search engine that automatically indexes code, logs, PDFs, SQLite, and CSVs into typed datasets for agent retrieval, incident analysis, and RAG. Its autoindex workflow avoids schema setup and returns ranked passages with file and line locations. The project defaults to local operation, although optional reranking can send text to an external provider.
|
|
Redesigning source control at Anthropic (17 minute read)
Anthropic replaced Git with Scion, an in-house version control system built in five months, after daily code merges grew from 2,500 to over 30,000. By moving operations to servers and fetching only the files needed, it reports roughly 100 times faster checkouts and 200 times lower CPU and memory requirements than its previous Git mirror infrastructure.
|
Parsing JSON at tens of gigabytes per second… in Python (5 minute read)
Daniel Lemire's benchmark processed 7.6 GB of JSON in about 0.12 seconds using fastsimdjson on free-threaded Python across 64 cores, reaching roughly 66 GB/s. The speed comes from running threads in parallel and creating Python objects only for the fields needed, with data already decompressed in memory.
|
|
Love TLDR? Tell your friends and get rewards! |
|
Share your referral link below with friends to get free TLDR swag!
|
|
|
| Track your referrals here. |
|
|
|
0 Comments