On Second Thought — Episode 06 The ORM hides the SQL. The cache hides the ORM. The service mesh hides the services. The operator hides the YAML, which already hid the kubelet, which already hid the container, which already hid the process. By Tuesday, nobody quite remembers what the original problem was. They are too busy configuring its sixth wrapper. This is the post about that wrapper. When som
At 3:17 AM on a Tuesday in Q3 2024, our production Kotlin 2.0 microservice fleet hit a 92% memory utilization threshold across 140 nodes, traced to a silent coroutine leak in Ktor 2.2’s request pipeline that had been bleeding 12MB of heap per second for 72 hours. We lost $14k in SLO credits before we found the root cause. A Couple Million Lines of Haskell: Production Engineering at Mercury (78 p