Workflow Professionals

Problem investigation example

A worked walkthrough from identification to known error and lasting fix. It shows how techniques are used in practice — timeline, Pareto, and 5 Whys — without turning the problem record into a novel.

Scenario

An internal finance application, LedgerOne, serves invoice approval for about 400 users. Over six weeks the service desk closed 38 incidents tagged “application slow” or “timeout saving invoice”. Each ticket was restored by asking the user to retry or by recycling an application server. Finance leadership asked why the same disruption kept returning at month-end.

1. Identification

Weekly incident trend review (Pareto by category and CI) showed LedgerOne timeouts as the top driver of priority 2 volume, concentrated in the last four working days of each month. That met the organisation’s criteria for recurring incidents with business significance. Problem PRB004201 was raised, owned by the problem manager, with the application owner as technical lead. All 38 incidents were linked.

Incident teams continued to restore users as before. The problem record tracked cause investigation separately so major-incident-style pressure did not redefine every slow day as a new major incident.

2. Situation appraisal and timeline

Before deep dives, the team wrote a short appraisal:

  • Is: save and approve actions timing out; month-end peak; app tier CPU high
  • Is not: login failures; report exports; non-finance apps on the same IdP
  • Where: both app nodes; worse on node B during peak
  • When: mainly 15:00–17:00 in the last week of the month
  • Changed recently: new invoice attachment feature released five weeks earlier

A chronological analysis of one severe afternoon lined up: batch posting job start → connection pool wait alerts → first user timeouts → app recycle → temporary relief. The timeline ruled out network change windows that afternoon and pointed at database connection handling under concurrent batch and interactive load.

3. Analysis

Pareto kept the investigation on LedgerOne save timeouts rather than scattering into unrelated “slow PC” tickets that shared a loose category.

Five Whys (evidence-backed) on a representative failure:

  1. Why did the save time out? — The app waited for a database connection beyond the request limit.
  2. Why was no connection available? — The pool was exhausted by interactive sessions and the month-end batch job.
  3. Why did batch and interactive share one pool? — The attachment feature reused the default pool with no separate batch pool.
  4. Why was that missed before release? — Performance test used average-day load, not month-end concurrency.
  5. Why was month-end load absent from test? — No non-functional scenario owned by the service for peak close.

Root cause recorded: shared database connection pool and missing peak-load test coverage for the attachment release, causing pool exhaustion and user timeouts at month-end.

4. Workaround and known error

While a permanent fix was designed, PRB004201 was moved to known error with an operator workaround:

  • During month-end peak, schedule the batch posting job to 18:30 (after the interactive peak), via the existing standard change for job schedule updates
  • If pool wait alerts fire during the day, pause non-critical batch steps using the runbook and notify finance ops — do not recycle app nodes as the first step
  • Service desk script: link matching incidents to PRB004201; avoid duplicate deep diagnosis

The known error article was published to the service desk knowledge base the same day the cause was confirmed.

5. Resolution through change

Change CHG009602 (normal) split interactive and batch connection pools, raised alert thresholds on pool wait time, and added a month-end concurrency scenario to the performance test pack. After implementation, two month-ends showed no recurrence of the timeout cluster. Post-implementation review closed the known error. A continual improvement action kept the peak scenario in the regression suite for future LedgerOne releases.

What this example demonstrates

  • identification from trends, not only from a single dramatic outage
  • incident restoration continuing while problem investigation owns the cause
  • technique choice recorded and proportionate: Pareto → timeline → 5 Whys
  • known error and workaround published before the permanent change landed
  • fix verified by evidence after change, then the known error closed
Problem management overview Root cause techniques Known error example