ET1: Experiencing outage

Incident Report for NRP

Postmortem

Symptom

Failing smoketest

Impact

Initially all services across NRP was affected by issues with database This was resolved before business hours. Afterwards, it was discovered that prepayment service was causing further issues.

Root cause

‌The used prepayment bond price was was not valid due to a faulty version of the serving legacy module

Resolution

‌A rollback of the legacy module to a working version was done

Actions

N/A

Event started at: 08:00
Event ended at: 13:28

Posted Nov 19, 2025 - 15:04 CET

Resolved

Roll back was performed on mainframe (legacy side). ET1 is fully operationel and service has been restored.

Posted Nov 17, 2025 - 13:28 CET

Update

Still investigating / working on resolution in legacy side.

Next status:
15:15

Posted Nov 17, 2025 - 13:12 CET

Update

Database issue was resolved around 08:30 and test environments was restarted and up around 09:00. Environment is up and running, but we are seeing issues with prepayment from legacy side. We can't confirm if issues with prepayments affect all loans or just a limited set.

Next status:
13:15

Posted Nov 17, 2025 - 11:24 CET

Identified

Symptom:
Because of database outage this morning, ET1 has not been started correctly, and most API services are responding with errors.

Reference:
N/A

Technical impact:
Failed morning start due to Oracle database (insufficient log storage - ORA-257). We expect a complete restart of the renvironment will resolve the issue.

Business impact:
Not possible to perform test

Next status:
11:00

Posted Nov 17, 2025 - 08:18 CET

This incident affected: ET1 environment (ET 1 environment smoke test status).