On New Year’s Eve 2022, while many were watching the ball drop in NYC, or fireworks in Las Vegas, I found myself outside in the rain, holding a screw gun and cursing like a sailor, watching my pool fill with silt laden rain runoff.

You see, this New Year’s Eve, we received about three inches of rain here in Los Angeles.

To those of you living in more normal climates, this probably sounds pretty mild for December 31. But to those of us who have been living in Los Angeles for the past decade plus, this was unusual to say the least. We are in the midst of a massive drought, almost a decade long now, and rain beyond the occasional sprinkle, even in the winter, just doesn’t happen that often.

So, why did I go out in this rain? Well, our house was built in 1961. It has a pitched roof, and rain gutters, with ample drains, but nobody had really looked at those gutters or drains in a long, long time. We have lived here 9 years and had only really looked at them once when we first moved in. Cleared the leaves in them and went about our business enjoying Los Angeles’s notoriously perfect weather.

But the reality is, even with great weather, drains still need maintenance. Several of the gutters had filled with silt and detritus from the roof. Heck we even saw an ivy plant grow out of one corner.

And on NYC, one of the down spouts was clearly just not draining at all. The corner of it overhangs the pool, and we could hear it dumping into the pool like a hose had been turned on.

There was concern that the pool might overflow with this added in-flow, and a flooding pool is a flooded house, so, at 10pm, in the rain, this part time handyman decided it was time to check it out. This could be bad. What if the drains are clogged elsewhere?

So I grabbed my trusty drill-driven drain snake, dressed in waterproof clothing, and stomped out into the rainy night.

I stuck my snake in, and a big clump of dirt gave way immediately. Water started rushing down the down spout. Hooray! I moved on to the other down spouts and they were all clear.

But the splashing resumed a minute later. It was as if it had never been unclogged.

In went the drain snake again, and this time, it went 3 feet in. I jiggled it and pushed and it was stopped, so I began running the drill. It freed a few more feet down, twisting through a few curves. More water rushing through, but then a repeat, it backed up as I was removing the snake.

I shoved it back in and ran the drill at full speed, pushing and pulling, trying to remove the clog. It was bigger, and more stubborn. I knew in my head that if it’s really thick, more than a few inches, this snake will never release it. But I pressed on. It’s raining. I’m out here. It’s NYE and plumbers are gonna charge a fortune. Just Fucking Do It..

The drill strained. I pushed too hard, and I failed to notice the wire coil inside the snake starting to bind up. This should trigger a reverse, to ease the tension on the spring inside the snake, but I was too engrossed in pushing. In fact this snake is better with two people, one to watch the coil, and one to push and pull, but my wife was inside watching Miley and Dolly sing, and I was almost done, why call for help?

The snake twisted in on itself, and the wire pulled out, ruining the snake, and making the snake, all 8 feet that were in the drain pipe, as rigid as a steel bar. It was stuck. So now, we have a clogged drain, and a broken snake.

Fuck it. I detached the drill, packed up my stuff and hoped that what I had cleared would at least be helpful as the rain slowed later and we wouldn’t have the pool flood after all.

Fast forward to Jan 2, the rain has stopped and I dismantled the drain to remove the broken snake.

Inside the drain, I found six feet of what looked like 30 year old silt and roots, heavily compacted. Water drained through it at about 1mm per minute. I spent 2 hours dismantling and clearing the down spout, and then turned to the drain. The drain is also compacted, who knows how far. This is a job for professionals, and they’ll be here later in the week to finish the job.

This story reminds me of being on call for technical operations. Something alerts, and you open it up, employing your usual playbook of seemingly related remedies. But this time, it doesn’t work. In fact, it makes things worse. You aren’t an expert, just a reasonably capable person with access to tools and pressure to succeed. You may be in over your head, but escalation has psychological and maybe financial barriers, so you keep working on it, thinking you can do it, making things even worse.

So, make sure you clean out your services’ rain gutters. Even if there’s a drought, even when you aren’t expecting trouble, just remember why you have those systems. Fake an incident, do a table-top exercise, maybe make sure your backups are actually working. Whatever it is, allocate some time to it while the sun is shining. Because you’re much more likely to break something if you wait until a massive rain storm.

And also, remember that when there’s extra pressure, that’s the most important time to escalate. It’s raining, it’s a special night, you don’t want to be fixing stuff, and you know others don’t either. But escalate anyway. Call the other senior engineers. Call the plumber. Get the experts on the phone and make sure you don’t make one emergency in to two.