Starting on April 29th, 2021 at 7.09 p.m. ET, the Locations API (a legacy equivalent to the Knowledge API) started returning a higher than average number of errors, accounting for about 1-2% of all requests. This was remediated by our team by April 30th at 10.55 a.m. ET. The Live API as well as requests to the current Knowledge API were unaffected.
A series of large requests made to the Locations API resulted in increased latencies in responding to health checks, which caused restarts of the services providing the API. These restarts resulted in the errors seen by consumers of the API.
The issue was remediated in the short term by providing increased resources to the affected services. We will further follow up by adding caching to reduce the load on these services, and improving our monitoring of these specific endpoints to catch such problems earlier.