Degradation in Live API service
Incident Report for Yext
Postmortem

Summary

On Thursday, July 5, from approximately 10:48 AM to 11:25 AM EDT, LiveAPI v2 requests had higher than average latencies and returned errors for some calls due to timeouts. All users, including Pages Store Locators, could have potentially been affected.

The issue was isolated to the Live API; the Platform API was not affected. Pages for individual locations and static location directory pages on Yext Pages sites were also unaffected.

Root Cause

The errors were caused by a failure in a third-party search infrastructure provider used by the Live API and a spike in load. Our mitigation was to fail over to a backup cluster in another region. We returned to normal operations using our primary cluster at around 12:10 PM EDT.

In response to this issue we have increased capacity in all regions and are in the process of automating load management without need for human intervention.

Posted 5 months ago. Jul 13, 2018 - 11:11 EDT

Resolved
We have completed our monitoring period, all services continuing to function normally. This issue is resolved.
Posted 5 months ago. Jul 05, 2018 - 14:12 EDT
Monitoring
We have identified and mitigated the issue and mitigated. We will continue to monitor serving.
Posted 5 months ago. Jul 05, 2018 - 11:36 EDT
Update
We are continuing to investigate some Live API calls that are experiencing larger than average latencies.
Posted 5 months ago. Jul 05, 2018 - 11:26 EDT
Investigating
We are currently investigating a degradation in the Live API service. Some customers may be experiencing longer than normal latencies and intermittent failures. We will provide an update in the next 15 minutes.
Posted 5 months ago. Jul 05, 2018 - 11:10 EDT
This incident affected: Live API.