Summary

October 24th, 2019, beginning at around 4:00 a.m. Eastern, Yext engineers began observing high error rates in the Analytics service in Sandbox. Service was restored at 12:41 p.m. Eastern. During this outage, Analytics was unavailable in Sandbox. There was no impact to our production services.

Root Cause

During a routine cluster maintenance, a configuration change failed to be deployed to all networking layers, preventing requests from reaching the backend cluster.

Remediation

We will improve our deployment procedures in our sandbox infrastructure to avoid this class of problem in the future.

Posted Nov 01, 2019 - 16:00 EDT

Resolved

We have completed our monitoring period, all services continuing to function normally. This issue is resolved.

Posted Oct 24, 2019 - 13:48 EDT

Update

We are continuing to monitor for any further issues.

Posted Oct 24, 2019 - 12:49 EDT

Monitoring

We have resolved the incident with Analytics services in the sandbox environment. We are monitoring to ensure proper system functionality. We will provide another update by 2:00 p.m. US Eastern Time.

Posted Oct 24, 2019 - 12:48 EDT

Investigating

We are investigating reports that the Analytics service in sandbox is unavailable, affecting both the Analytics section as well as graphs in other sections of the Customer Portal, and Analytics endpoints in the Platform API. Only the sandbox environment is affected; Analytics in the regular production environment is operating normally.

We will provide an update within the next hour.

Posted Oct 24, 2019 - 12:18 EDT

This incident affected: Sandbox.