Working Notes

Last week, I spent a morning at Snowflake's Data for Breakfast event, which is exactly what it sounds like: an early session where practitioners walk through what's new in the platform and where customers are actually getting value. The through-line across most of the presentations wasn't a specific feature. It was a concept that kept surfacing as a prerequisite for making AI work reliably against real business data. They called it the semantic layer. The framing was direct: without it, AI tools querying your warehouse will produce answers that are fast, fluent, and inconsistently grounded in what your business actually measures. With it, you give the AI something reliable to reason against. That framing stuck with me — partly because it applies well beyond Snowflake, and partly because it names something that data teams have wrestled with long before AI entered the picture. What a Semantic Layer Actually Is The term sounds more technical than it needs to be. At its core, a...

There's a specific moment most on-call engineers recognize. You're in a post-mortem, walking through a memory exhaustion failure that took longer to diagnose than it should have. Someone mentions they've seen this pattern before. Someone else says they filed a ticket after the last one. You go looking for the ticket. It's closed — marked resolved after the immediate fix, never connected to anything in the roadmap. The underlying failure mode is still there. You're just meeting it again. That moment is worth examining, because it reveals something about how reliability work actually moves — or doesn't — through most technology organizations. The on-call engineers usually have detailed, accurate knowledge of where the system is fragile. They know which failure modes the runbooks don't fully cover, which write patterns cause CPU plateaus the monitoring doesn't catch, which edge cases will eventually surface as customer escalations. That knowledge gets expre...

Nobody files a support ticket to say the database was up all day.  Users don't praise reliability. They don't notice it at all — until it's gone. And when it goes, they don't think "the SLO was breached." They think "I lost data" or "my app was down." They think it twice, and then they start looking for alternatives. That asymmetry is especially sharp in database products, where reliability isn't one thing — it's several, and they can fail independently. Not a Single Dial When teams talk about database reliability, availability usually dominates the conversation. Is the endpoint reachable? Is the cluster healthy? Can we fail over? Those questions matter. But availability failures have a particular character: they're visible, they're shared, and recovery brings relief. An outage is a trauma event. Everyone knows it's happening, everyone mobilizes, and when the cluster comes back, there's a collective exhale. It gets a...

Most OLTP databases are predominantly read workloads. SELECT statements outnumber INSERT, UPDATE, and DELETE by a wide margin — sometimes by an order of magnitude. Users browse and display far more often than they create or modify. Read scaling gets attention first because read pressure arrives first, and the solutions are well-understood: replicas, caches, connection pooling. It works. Writes arrive later and are more disruptive when they do. In the  previous post on schema as interface , the argument was that your schema has clients who depend on certain access patterns being cheap. Writes are a new kind of client the original design often didn't account for. Indexes that accelerate reads add overhead to every write. Constraints that protect integrity can increase contention. The schema doesn't change. The workload does. And suddenly the interface is expensive in new ways. Writes expose the actual shape of your system. Not the architecture diagram shape — the real shape. Wher...

Most teams adopt an ORM for good reasons. In the early sprints, having the database feel like a natural extension of the application — models mapping to tables, relationships expressed in code — lets teams move fast without needing deep SQL fluency on every story. The ORM earns its place well before the first release. What I've observed is what happens after. The product ships, sprint priorities shift toward feature work and production issues, and the ORM quietly becomes tribal knowledge. The engineers who set it up carry a mental model of how it works and why certain decisions were made, but that understanding rarely survives team rotation — much like documentation, it gets treated as something someone will formalize eventually. New engineers learn just enough to be productive, the ORM fades into the background, and by the time something goes seriously wrong with the database, it's effectively a museum piece. This is the Day 2 problem with ORMs — not that they're the wr...

When people say “schema as interface,” they usually mean naming and semantics. That’s part of it. But in a production system, the real interface is the set of access paths the query planner can execute cheaply. Every `SELECT`, join, background job, and dashboard query depends not just on tables and columns, but on index shape, cardinality, constraints, and data distribution. The schema is not only a contract of meaning; it is a contract of performance. I’ve seen more systems struggle from schema drift than from application bugs. Not because the code was wrong, but because the data model made correct code slow. A table starts simple and well-indexed. Features accumulate. Nullable columns get added “just in case.” Enums become free text. Indexes are added reactively. At 50k rows, everything works. At 50 million, joins that used to be cheap become dominant costs. Nothing is technically broken—but the interface has changed. We would consider an API broken if latency jumped from 40ms to 4 s...