Pages Generation Delays
Incident Report for Yext
Postmortem

Summary

On December 1st, 2021, beginning at 4:15PM ET, Yext engineers observed and began investigating delays in a server of our asynchronous messaging service. This degradation affected a portion of sites in the Pages Generation system. Multiple mitigation efforts were applied, and the Pages Generation pipeline fully recovered at 4:39PM ET on December 3rd. Engineers continued monitoring the service over the weekend before declaring the incident resolved on December 6th, 2021.

Root Cause

A configuration setting was incorrectly applied on the asynchronous messaging service for pipelines associated with the Pages Generation system, causing a node failure to stall the pipeline.

Remediation

The configuration setting has been updated and we have also rolled back recent updates to our asynchronous messaging service to conduct further testing for similar issues.

Posted Dec 20, 2021 - 09:21 EST

Resolved
This incident has been resolved.
Posted Dec 06, 2021 - 18:07 EST
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Dec 02, 2021 - 13:24 EST
Investigating
We are investigating reports of delays for some sites in Pages Generation.
Posted Dec 01, 2021 - 16:55 EST
This incident affected: Pages Generation.