Increased errors on Pages sites
Incident Report for Yext
Postmortem

Summary

On November 12, 2019, from 6:57 p.m. until 7:24 p.m. US Eastern Time, some images, like social media network icons, went missing on web pages built with the Yext Page Builder. Webpages relying on Yext Event Calendar Tags also failed to display any events during this time.

Additionally, from 7:07 p.m. until 7:17 p.m., some webpages hosted by the Yext Pages platform were inaccessible. Visitors to the affected pages during this time saw an error message from their browser.

Root Cause

A change in our CDN configuration settings to increase the security of the hosting for Yext Pages inadvertently broke the serving of files hosted under https://assets.sitescdn.net/. The outage was prolonged by a bug that caused our standard rollback procedure to fail, leaving our configuration in a state that additionally affected some Yext Pages sites. The issue was resolved by manually reverting our configuration to the state before this change.

Remediation

We are updating our processes for CDN configuration changes to include more testing of rollback contingencies. We are also modifying our CDN configuration to more thoroughly isolate different systems, to reduce the likelihood of similar unintended side effects from future changes.

Posted Nov 20, 2019 - 11:27 EST

Resolved
Starting at 6:57 p.m. US Eastern Time, some webpages hosted on the Yext Pages platform presented a degraded experience or showed a browser error page. The issue was resolved at 7:24 p.m.
Posted Nov 12, 2019 - 19:00 EST