Plan Changes May Block on "Waiting for the Index Curation Lock" Stage
Incident Report for ESS (Public)
Resolved
Our monitoring indicates that this issue is no longer impacting cluster plan changes. We will continue to monitor overall plan change success rates as per normal.

This issue is now resolved.
Posted Jul 10, 2019 - 04:14 UTC
Monitoring
We've deployed the fix to all regions. No new deployments should be impacted by this issue. Plan changes should now succeed for any deployments which had been seeing plan changes stuck in a "Waiting for the Index Curation Lock" state.

We'll continue monitoring plan change failures as we verify the fix. We will update this incident within the next hour when the fix is verified in all regions.
Posted Jul 10, 2019 - 03:33 UTC
Update
We've deployed and verified the fix on all Asia Pacific and Europe based regions. We're continuing to deploy the fix to the US based regions.

We will update this issue when the fix is completely deployed, which we expect to be within the next 30 minutes.
Posted Jul 10, 2019 - 03:13 UTC
Identified
We've verified a fix in our staging environment. We're preparing to release this fix into production now. Deploying this fix will also fix any deployments which are currently impacted by this issue.

We expect the fix to be deployed to production within the next 2 hours. We'll update this incident within the next 2 hours, or when the fix is available in all regions.
Posted Jul 10, 2019 - 02:45 UTC
Update
We are still validating this fix. We'll have an update in the next 4 hours.
Posted Jul 10, 2019 - 01:14 UTC
Update
We have identified a potential fix for this and are working to confirm that it will resolve the issue and will have an update in 4 hours.

While we work on a fix, You can avoid this issue by not cancelling plans - especially in the early stages of a plan while it's performing initial steps.
Posted Jul 09, 2019 - 22:05 UTC
Investigating
Starting on July 3, customers in all regions may have started seeing plan changes getting stuck in "Waiting for the Index Curation Lock" state preventing further plans from being applied. We are investigating a fix and will update in 30 minutes.
Posted Jul 09, 2019 - 21:43 UTC