New notices not being created from a triggered automation

Retrospective

Last Wednesday, January 10th, at around 4:30 PM, we released some changes which, although unrelated to monitoring automations, inadvertently broke them.

Once we became aware of this, early on January 11th, we immediately rolled back our changes to get things working again while we tried to chase down the details of what we had broken.

We have now found the code which broke things and have corrected it; we have also improved our automated testing process around monitoring integrations to help prevent this kind of thing from happening again.

Sorry to everyone who was impacted, and as always, we've tried our best to learn from our mistakes.

We've identified the issue and have implemented a fix. Monitoring automations are now triggering and publishing correctly. Thanks for your patience during this incident.

We are investigating an issue with monitoring automation, specifically inbound mail, where the automation is triggered, but notices are not created or published.

We will provide updates as we continue to investigate the issue.