Elevated Error Rates in Platform Api
Incident Report for Yext
Postmortem

Summary

On Friday, March 22nd, beginning at 11:56 PM EST, specific portions of the Platform API began returning errors. Investigation began right away. The cause of errors was identified by March 23rd, 12:30 AM EST. Mitigations were implemented by 1 AM, at which point errors subsided.

Only the Analytics and Knowledge Manager APIs were affected. The majority of other API services continued to operate normally.

Root Cause

A bug in our deployment infrastructure caused the backend servers for the aforementioned services to fail to start up after a routine shutdown. In addition to fixing the bug, we plan to add additional monitoring and alerting that will help us anticipate this failure mode.

Posted 3 months ago. Apr 01, 2019 - 14:53 EDT

Resolved
This incident has been resolved.
Posted 3 months ago. Mar 23, 2019 - 08:39 EDT
Monitoring
We have implemented a fix and error rates are back to normal. We will continue to monitor for any issues.
Posted 3 months ago. Mar 23, 2019 - 01:27 EDT
Identified
We have identified the cause of the errors and are implementing mitigatory actions. We will update as soon as we have more information.
Posted 3 months ago. Mar 23, 2019 - 00:52 EDT
Investigating
We are currently investigating reports of elevated error rates in the Platform API. We will update as soon as we have more information.
Posted 3 months ago. Mar 22, 2019 - 23:56 EDT
This incident affected: Platform API.