Problem Management Policy and Process

Defines how recurring or serious incidents are analysed for root cause, worked around, fixed permanently and recorded so that they do not return.

SXL-A23Policy1,658 words 8 min readOwner: Head of IT Operations
Sign in to download the PDFFree. You return straight to this document after you sign in.
Template draft. Replace everything in [square brackets] with your own decision, delete what does not describe what you really do, and approve it through your own document control. It is not legal advice and does not make an organisation compliant or certified.

1. Purpose

This document makes sure that the underlying causes of incidents are found and removed, so that the same failure does not return, and that known errors and workarounds are recorded for the service desk. Incident management restores service fast; problem management stops the repeat.

2. Scope

The process applies to problems that affect the confidentiality, integrity or availability of the Organisation's services and information, found through incidents, monitoring, testing, audit, supplier reports or staff observation.

  • In scope: repeated incidents, a single major incident with an unknown cause, trends found in monitoring or capacity data, recurring security events, defects in software built or run by the Organisation, and issues reported by suppliers.
  • Out of scope: restoring service for a live incident, which is handled under the Incident Management Procedure; and service requests.

3. Terms and Definitions

TermMeaning
ProblemThe unknown underlying cause of one or more incidents.
Known errorA problem whose root cause is known and for which a workaround or fix exists or is planned.
WorkaroundA temporary way to reduce or avoid the impact of an incident or problem.
Root cause analysis (RCA)A structured method to find why a problem occurred.
Known error database (KEDB)A searchable record of known errors and workarounds used by the service desk.
Proactive problem managementFinding and fixing problems before they cause incidents, using trends and tests.
Problem managerThe person who owns the process and each problem record.

4. Roles and Responsibilities

RoleResponsibilities
Head of IT Operations (document owner)Owns this policy and resources problem management, approves major problem reviews and reports trends.
Problem managerOwns each record, runs the analysis, coordinates fixes, keeps the KEDB and publishes the monthly report.
Problem owner (technical)The expert assigned to investigate and propose a fix.
Service deskLinks incidents to problems, uses the KEDB, and tells users about workarounds.
Information security managerReviews problems with security impact and checks that fixes close the weakness.
Change managerTakes permanent fixes through the Change Management Policy and Procedure.
SuppliersProvide analysis and fixes for products and services they supply, within contract times.
ActivityResponsibleAccountableConsultedInformed
Detect and log a problemProblem managerProblem managerService desk, security operationsHead of IT Operations
Classify and prioritiseProblem managerHead of IT OperationsSystem owner, information security managerService desk
Perform root cause analysisProblem ownerProblem managerEngineers, supplierSystem owner
Publish a workaroundProblem managerProblem managerService desk, problem ownerUsers
Implement the permanent fixProblem ownerSystem ownerChange manager, information security managerProblem manager
Close the problemProblem managerHead of IT OperationsSystem ownerService desk

5. Problem Management Process

5.1 Problem identification

Problems are raised reactively, from incidents, and proactively, from analysis. The problem manager reviews the sources below at least [weekly].

SourceTrigger to raise a problem
Incident recordsThree or more incidents of the same type, system or cause in [30 days]; any major incident with an unknown cause; any incident that needed a temporary fix
Monitoring and logsRepeated alert patterns, rising error rates, unusual traffic or resource trends
Security eventsRepeated phishing success, repeated malware on one route, repeated policy bypass
Change recordsA change that failed, was backed out or caused an incident
Testing and auditVulnerability or penetration test findings that point to a systemic cause; audit non-conformities
Suppliers and communitiesVendor advisories and defect reports that match the Organisation's systems
PeopleA reasoned request from staff, customers or a service owner

5.2 Problem logging

  1. RecordLog the problem in [the ticketing tool] within [1 working day] with title, description, symptoms, services and assets affected, date, source, and linked incidents. Output: problem record with unique number.
  2. LinkLink all related incidents and changes, and tag any with security or privacy impact.
  3. AssignThe problem manager assigns a problem owner and sets the next update date.
  4. Check the KEDBSearch for an existing known error; if one is found, link it and apply the recorded workaround.

5.3 Classification and prioritisation

PriorityCriteriaStart analysis withinTarget for root causeTarget for permanent fix plan
P1Critical service or security control failed more than once; major incident cause unknown[1 working day][5 working days][10 working days]
P2Important service degraded repeatedly, or confirmed weakness being exploited[2 working days][10 working days][20 working days]
P3Limited impact, workaround available[5 working days][20 working days][40 working days]
P4Minor or cosmetic, low frequency[10 working days]Next reviewAs capacity allows

Priority is set from the number of users affected, business impact, frequency, security and legal impact, and cost. The problem manager may raise priority if the pattern worsens.

5.4 Root cause analysis

  1. Collect evidenceGather logs, timelines, configuration records, change history, and interviews. Preserve evidence if the problem could be a security incident.
  2. Choose a methodUse a technique that fits: five whys for simple chains, cause-and-effect (fishbone) diagrams for several possible causes, fault tree for complex systems, timeline analysis for sequences of events, and change analysis when a recent change is suspected.
  3. Find the root and contributing causesDistinguish the root cause from contributing factors (process gap, training, design, monitoring, supplier). Verify by reproducing the fault or proving the cause with data.
  4. Write the findingRecord the cause, the evidence, the services affected and the recommended fixes. The problem owner and the problem manager review it.
Note. Worked example, five whys: customers could not upload files (incident). Why? The storage service refused writes. Why? The volume was full. Why? Old export files were never purged. Why? The clean-up job failed silently after a change to its schedule. Why? The change had no verification step and job failure had no alert. Root cause: missing monitoring and verification for a scheduled job. Fix: add an alert, add a verification step to the change template, and purge the volume.

5.5 Resolution and workarounds

  1. Define a workaroundWhere the fix will take time, define and test a workaround with the service desk. Record it in the KEDB with steps, limits and expiry.
  2. Plan the fixAgree options, cost, risk and date, with the system owner's approval. Raise a request for change for the permanent fix.
  3. ImplementCarry out the change under the Change Management Policy and Procedure and update the knowledge article.
  4. VerifyMonitor for [30 days] (or one full business cycle) to confirm that incidents have stopped.
  • A workaround that carries a security risk, such as turning off a control, needs approval from the information security manager and an end date.
  • A problem that cannot be fixed economically is closed as an accepted risk, with approval of the system owner and Head of IT Security, and recorded in the risk register.

5.6 Closure and documentation

  1. ConfirmThe problem manager checks that the fix is in place, that no related incident has occurred in the verification period, and that the KEDB entry is marked resolved.
  2. ReviewFor P1 and P2 problems hold a review within [10 working days] of closure: what happened, what worked, what to change in detection, design or process.
  3. DocumentUpdate the knowledge base, runbooks, monitoring, configuration records and risk register. Share lessons with relevant teams.
  4. CloseSet the status to Closed with closure code and date. Notify the linked incident owners and the service desk.

6. Problem Management Policy Statements

  1. Every incident that meets the trigger criteria must be linked to a problem record within [1 working day].
  2. Root cause analysis must be completed for every P1 and P2 problem, with a written finding reviewed by the problem manager and the system owner.
  3. The known error database must be kept current and used by the service desk as the first source for workarounds.
  4. Permanent fixes must follow the Change Management Policy and Procedure.
  5. Problems with a possible security or privacy cause must also be reported under the Information Security Incident Management Policy, and evidence preserved.
  6. Problem trends are reviewed at the monthly operations meeting and quarterly by the information security forum.

7. Major Problem Review

A problem is treated as major when it is linked to a major incident, a data breach, an outage beyond the recovery time objective, or a customer complaint to a regulator. The problem manager convenes a review within [5 working days] of service restoration.

Review itemContentOwner
TimelineMinute-by-minute record from first sign to restoration, with who did whatProblem owner
Cause analysisRoot and contributing causes, with evidenceProblem owner
ImpactUsers, customers, data, money and reputation affectedSystem owner
Detection and responseHow quickly the problem was detected, escalated and contained; what slowed itSecurity operations
ActionsCorrective and preventive actions with owners and dates, tracked to closureProblem manager
ReportBlameless written report to management within [10 working days]Head of IT Operations

8. Compliance and Monitoring

The problem manager reviews open problems against their targets each [week]. The Head of IT Operations reports on the measures below to management each [quarter], and internal audit samples problem records, KEDB use and closed-problem verification once a [year].

9. Measures

MeasureTargetHow measured
P1 and P2 problems with a completed RCA in target time[90%]Record dates
Repeat incidents for a closed problem within [90 days][under 5%]Incident links
Incidents resolved using a KEDB workaround[Rising trend]Service desk tags
Open problems older than [90 days][under 10%]Ageing report
Closed problems with a documented lesson[100% of P1 and P2]Record sample

10. Records and Evidence

  • Problem records, RCA reports and review notes, kept [3 years].
  • Known error database with version history, kept while current.
  • Monthly problem report and trend analysis, kept [3 years].

Annexure A: Problem record

FieldEntry
Problem number and date
Title and description
Source and linked incidents
Services and assets affected
Priority and rationale
Problem owner
Root cause and evidence
Workaround and expiry
Permanent fix and change number
Verification result and date
Lessons learned
Closure code and approver

Annexure B: Known error register

KE no.SymptomsRoot causeWorkaroundLinked problemStatusReview date

11. Exception

Exceptions to this document are allowed only where compliance is not practicable and the risk has been assessed. A request must state what is to be excepted, why, the risk it creates, the compensating controls, and an end date no later than [6 months]. It is approved in writing by the document owner and the risk owner, recorded in the exceptions register, and reviewed at the end date. An exception does not change the document.

12. Enforcement

Compliance is monitored through internal audits, management review and routine checks by the document owner. Non-compliance is reported through the incident or non-conformity process and handled under the organisation's disciplinary procedure, and under the terms of any contract for people who are not employees. Serious or repeated non-compliance may lead to loss of access and, where the law is broken, to referral to the authorities.

13. Review

This document is reviewed at least every [12 months], and when there is a significant change to the organisation, its systems, its risks or the law, or after an incident that shows it was not effective. Each review is recorded in the revision history.

ISO/IEC 27001:2022 references

ReferenceTitle
A.5.27Learning from information security incidents
A.5.26Response to information security incidents
A.8.32Change management
A.5.37Documented operating procedures
10.2Nonconformity and corrective action
9.1Monitoring, measurement, analysis and evaluation
Sign in to download the PDFFree. You return straight to this document after you sign in.