Current Status

Resolved

Scope

Some applications in the us-central1 region experienced a complete to partial loss of incoming traffic from approximately 11:31 UTC (06:31 CDT) to 11:53 UTC (06:53 CDT).

Applications that are running on a Dedicated Ingress were not affected.

gigalixir.com, the Gigalixir console and use of the Gigalixir CLI and APIs were also affected during this time.

Timeline

15:00 UTC / 10:00 CDT

At this time all systems have returned to normal operation. We are continuing to add to our system to help prevent similar issues in the future.

This issue is resolved.

If your application is mission critical, please consider looking into our Dedicated Ingress offering. The dedicated ingress helps isolate your application traffic from other applications on the platform.

You can read more on the benefits of dedicated ingress here.

13:15 UTC / 08:15 CDT

We have identified the root cause of the issue and are working on preventative measures. At this time we do not expect any further issues in the short term, but will continue to monitor the situation.

12:30 UTC / 07:30 CDT

We believe we have identified the root cause. We are investigating it further at this point to verify.

More to come as we investigate the root cause of the issue and continue to monitor.

11:53 UTC / 06:53 CDT

At this time we show everything is operational and we are continuing to monitor the situation. Please report back if you are still having any issues. Contact us at help@gigalixir.com if you have any questions.

11:31 UTC / 06:31 CDT

We experienced a heavy spike of traffic into our default ingress system. Our system should have been able to handle the spike, but did not. We are investigating why the load caused an outage.

Our main site and console were down as well, because they are running on the same default ingress system. We will be looking to move our infrastructure to one of our dedicated ingress setups so that we can communicate with users via the website in case of a similar issue in the future.