Some of the users started the experience issues with Singa & Singa Pro around 5.30pm (UCT +2). We are investigating the issues at the moment.

Major incident API Web Business Lite Business Admin Business Check-In
2022-06-17 14:57 · 1 hour, 12 minutes

Updates

Times are shown in Africa/Abidjan timezone
Post-mortem

Our backend utilizes caching to improve the performance. When the cache is used the backend connects to cluster of cache nodes (i.e. servers). As soon as the incident began we saw that the cluster reached outbound bandwidth limit and dropped request from our backend apps.

To increase available bandwidth we tried to add more nodes to the cluster but the cluster did not response to the request. The hosting service we use did not show any errors and no new nodes were ever created.

Finally we initiated cluster reboot which restarted all the nodes and the cluster began responding to requests from our backend apps.

June 17, 2022 · 18:57
Resolved

Issue fixed. Happy singing!

June 17, 2022 · 16:09
Monitoring

We are investing and fixing the issue. Multiple different web pages and services went down at the same time. Some of the users are already able to use Singa normally.

June 17, 2022 · 15:44
Issue

We have detected issues and we are currently investigating, we’ll update you soon.

June 17, 2022 · 15:00

← Back