Sporadic Gateway issues resulting in 500/502 errors

Incident Report for Codat

Resolved

The incident has now been resolved and our ongoing monitoring has shown the platform has returned to normal stability.

All users previously affected should be able to query our API without getting 500s or 502s errors.

If you are still affected by this issue, please contact support@codat.io
Posted Sep 30, 2022 - 17:06 BST

Update

We are still actively monitoring the situation and affected users should no longer see these errors.

We will keep monitoring the situation for the next hours. If you experience the same issues, please contact support@codat.io for further assistance.
Posted Sep 28, 2022 - 15:00 BST

Monitoring

We have implemented a fix in both Production and UAT.

Users should no longer see 500/502 errors.

Our teams continue to monitor the platform to ensure continued stability. If you experience any issues please contact support@codat.io for further assistance.
Posted Sep 27, 2022 - 23:18 BST

Update

We are aware that a subset of clients are still affected by this issue.

We are currently working on implementing a fix to remediate the issue for the remaining clients affected.

This incident will be updated as soon as we have successfully tested and rolled out a fix in Production, and no later than 00:30 BST.
Posted Sep 27, 2022 - 22:35 BST

Update

We are aware that a subset of clients are still affected by this issue.

We are currently working on implementing a fix to remediate the issue for the remaining clients affected.

This incident will be updated as soon as we have successfully tested and rolled out a fix in Production, and no later than 22:00 BST.
Posted Sep 27, 2022 - 20:24 BST

Identified

We are aware that a subset of clients are still affected by this issue.

We are currently working on implementing a fix to remediate the issue for the remaining clients affected.

This incident will be updated as soon as we have successfully tested and rolled out a fix in Production, and no later than 16:00 BST.
Posted Sep 27, 2022 - 14:16 BST

Monitoring

We have implemented a fix in both Production and UAT.

Users should no longer see 500/502 errors.

Our teams continue to monitor the platform to ensure continued stability. If you experience any issues please contact support@codat.io for further assistance.
Posted Sep 26, 2022 - 22:24 BST

Update

We have identified the cause of the incident and are currently implementing a fix to remediate the issue.

This incident will be updated as soon as we have successfully tested and rolled out a fix.
Posted Sep 26, 2022 - 21:54 BST

Investigating

We are currently investigating

This incident will be updated as soon as we have identified the issue.
Posted Sep 26, 2022 - 21:43 BST
This incident affected: General Connectivity & Portal (API Connectivity).