Intermittent DNS Resolution Errors

lowAtlassianOct 22, 2019 22:32Duration: 5h 29m
dns
DNS Failure

Summary

Between 23:45 UTC to 02:19 UTC, some customers experienced intermittent failure connecting to Atlassian Cloud. The root cause was an increased DNS error rate from our infrastructure supplier. The supplier fixed the upstream issue and we have verified that the services have recovered. The conditions that caused the issue have been addressed and we are actively working on a permanent fix. The issue has been resolved and the service is operating normally.

Impact

none

Timeline

Oct 22, 2019 22:32

[investigating] We are aware that some customers may be experiencing intermittent failure connecting to Atlassian Cloud. We are working with our cloud hosting provider (AWS) to resolve the DNS related errors. We will provide hourly updates.

via statuspage
+1h 40m
Oct 23, 2019 00:12

[identified] We continue to work on resolving intermittent failure connecting to Atlassian Cloud for some cloud customers. We will provide hourly updates.

via statuspage
+56m
Oct 23, 2019 01:08

[identified] We continue to work on resolving intermittent failure connecting to Atlassian Cloud for some cloud customers. We will provide hourly updates.

via statuspage
+24m
Oct 23, 2019 01:32

[identified] We have identified the root cause of the intermittent failure connecting to Atlassian Cloud for some cloud customers and have mitigated the problem. We are now monitoring closely.

via statuspage
+1m
Oct 23, 2019 01:32

[monitoring] We have identified the root cause of the intermittent failure connecting to Atlassian Cloud for some cloud customers and have mitigated the problem. We are now monitoring closely.

via statuspage
+2h 29m
Oct 23, 2019 04:02

[resolved] Between 23:45 UTC to 02:19 UTC, some customers experienced intermittent failure connecting to Atlassian Cloud. The root cause was an increased DNS error rate from our infrastructure supplier. The supplier fixed the upstream issue and we have verified that the services have recovered. The conditions that caused the issue have been addressed and we are actively working on a permanent fix. The issue has been resolved and the service is operating normally.

via statuspage

Lessons Learned

馃搳Incidents related to dns have occurred 13 times across all providers in the past year.

馃挕This incident is categorized as: DNS Failure. Consider implementing preventive measures specific to this failure category.