Platform API - Log View Degradation
Incident Report for Yext
Postmortem

Summary

On July 16th, starting at 7:50 PM EST, the backing datastore for our Developer Console logs became overloaded, preventing access to multiple features in the Developer Console. The datastore was restored at 9:00 PM EST, and logs became available again. Customers were unable to see the usage graph or view logs in the details view of the Developer Console during this time.

Root Cause

We store up to a month of our API logs and provide this data in the Developer Console. A bug in our log configuration had been accidentally introduced which caused the backing datastore to keep multiple copies of incoming data. When attempting to gather and delete the duplicate logs, the server was overwhelmed. We have since flushed the duplicate logs from the system, and will be adding alerts to prevent regression.

Posted Aug 27, 2018 - 18:50 EDT

Resolved
This incident has been resolved.
Posted Jul 17, 2018 - 12:51 EDT
Monitoring
We have fixed the issues around viewing logs. We will continue to monitor for any problems.
Posted Jul 16, 2018 - 21:37 EDT
Investigating
We are investigating some issues around viewing logs in our Platform API. Some logs may be unavailable for viewing at this time in the Developer Console. No logs are lost and the API is fully functional.
Posted Jul 16, 2018 - 20:27 EDT