Mumford Advisory

Your logs exist. That doesn't make them searchable.

A customer reaches out because something that should have been sent from one system to another never showed up.

You know there are logs. Great. This should be easy.

Then you open them.

There are thousands of calls. You can't properly search them. You can't filter down to the thing you need. Sometimes you can't even see what happened without opening individual calls and digging through the payload.

So now you're scrolling.

I've worked with logs like this, and investigations that should have taken a few minutes can turn into hours of looking for one specific event — sometimes without finding it at all.

That's not really an observability problem in the way companies usually think about observability. The data exists. The problem is that the people who need it can't actually use it.

The expensive part isn't just the bad tool

One person spending an hour scrolling through logs is annoying.

The bigger problem starts when the ticket is difficult enough that multiple people get involved.

Support looks.

Escalations looks.

Maybe solutions engineering or engineering gets pulled in and looks too.

And because the tool itself doesn't give anyone a good way to narrow the search, you can end up with several expensive people independently digging through the same information.

Nobody decided that was a good use of their time. It's just what the process requires.

That's the kind of operational cost that's easy to miss because it never shows up as a line item.

You're already paying for the platform.

The additional cost shows up in people's time.

What could that actually cost?

The number is going to depend heavily on how often this happens, so I wouldn't pretend there's one universal figure.

But we can put some reasonable assumptions around it.

Say a manual log investigation takes about 1.5 hours and the loaded cost of the person doing it is $85/hour.

If this happens twice a month and only one person investigates:

2 investigations × 1.5 hours × 12 months = 36 hours/year

At $85/hour, that's about $3,060 per year.

Not particularly exciting.

But change the frequency and the number of people involved.

If you're dealing with roughly 13 of these investigations a month and three people or teams are spending time on them:

13 × 1.5 hours × 3 people × 12 months = 702 hours/year

At the same loaded cost, that's about $59,670 per year.

Same bad workflow. Very different business problem.

That's why I don't think every inefficient process needs an automation project attached to it. Sometimes the workaround genuinely is cheaper than fixing the workaround.

But you have to know which situation you're in.

And sometimes you still don't find the answer

This is the part the calculation doesn't capture particularly well.

You can spend all that time searching and still come up empty.

Then the ticket stays open.

Maybe you schedule a call with the customer and try to reproduce the issue. Maybe it doesn't happen again. Maybe the available logs don't go back far enough anymore.

Eventually you may have to tell the customer, essentially, we couldn't find it — let us know if it happens again.

That's a pretty terrible outcome after several people have already spent hours investigating.

What I'd do before trying to replace anything

If the logs belong to a platform you don't control, "get a better logging tool" isn't particularly useful advice.

So start smaller.

First, decide how much manual searching the investigation is worth.

If someone hasn't found the event after a reasonable amount of time, stop. Move to the next diagnostic step instead of letting someone spend half a day scrolling because the answer might be another hundred calls down.

Then look at whether the repetitive part of the investigation can safely be automated.

There are situations where a browser-based assistant or internal tool could do some of the tedious searching while the specialist works on something else.

But there's an important caveat: logs can contain customer data, personal information, tokens, identifiers, or other sensitive information. You don't point an AI tool at that data because it would be convenient.

Security and privacy requirements come first.

Sometimes the answer will be that automation isn't appropriate. That's fine. Even a simple search-time limit can prevent a bad tool from consuming unlimited specialist time.

Not every bad workflow needs fixing

If this happens twice a month and costs a few thousand dollars a year, I probably wouldn't build anything.

Document the limitation. Put a sensible limit on how long people search. Move on.

But if the same problem is regularly pulling multiple teams into the same investigation, that's different.

At that point the question isn't really whether the logs are annoying.

It's how much skilled time you're buying just to compensate for a feature the tool doesn't have.

Having logs and being able to use your logs are not the same thing.

Searchability is a feature.

And when it's missing, someone usually ends up doing the searching manually.


Want to see what workarounds like this may be costing your team? The Workaround Tax Calculator takes about ten minutes.

The cost is already being carried. The only question is whether it has been counted.

Request a fit conversation →