Between 12:20 AM ET on March 8, 2023 and 10:20 PM ET on March 20, 2023, Entity Relationship Indexing and asynchronous syncing of entity references were delayed because of a backup in the indexing queue caused by a large volume of updates from a single large customer. This led to the reflection of new or updated entity references in downstream systems such as Knowledge Graph, Search, Pages, and Yext’s APIs to be delayed for several hours on average but up to 24 hours in some cases. No data was lost and all updates were finished processing by 10:20 PM ET on March 20th.
Our indexing system’s parallelization was inadequate to handle the sudden volume of updates in a single account that was several orders of magnitude above normal.
Yext will be updating our indexing system to increase isolation between accounts so that large batches of updates from a single account will not impact other accounts. Yext will also be introducing new concurrency mechanisms designed to increase our overall indexing throughput.