Summary

Media caption,

Watch: BBC's Lucy Woodham asks Cardiff students about Snapchat outage

  1. AWS returns to normal as internet reels from mass outagepublished at 02:24 BST 21 October

    We're wrapping up our live coverage of a mass outage that caused swathes of the internet to fall apart today.

    Amazon Web Services says it has fixed the underlying cause - but many of the world's biggest websites will be reeling from the outage which put them out of action for much of the day.

    The outage hit early today, and affected companies as varied as Snapchat, Roblox, Zoom, Coinbase, HMRC and even banking services like Lloyds, Bank of Scotland and Halifax.

    The root of the problem was a malfunction at Amazon Web Services, the tech giant's cloud computing division, whose infrastructure underpins millions of large companies' websites and platforms.

    While outages affecting web platforms and services are nothing new, multiple sites going down in one single outage are somewhat less common - but experts say they are becoming more frequent, according to our technology reporter Liv McMahon.

    With much of our lives being online these days, the pressure on cloud services is only increasing all the time, writes technology reporter Shiona McCallum - and events like these highlight the fragility of these systems.

    AWS now says its cloud division has "returned to normal operations" and Downdetector reports of AWS issues have trailed off. Now the blame game starts as companies start to try and recoup losses from today, says our North America technology correspondent Lily Jamali.

    Check out our additional coverage for more information on what went wrong and why.

  2. What caused the outage?published at 02:17 BST 21 October

    Wondering why your ride-share app froze, digital wallet was not working, or a game failed to load on your server?

    Here is what we know about the cause of an outage that ground more than 1,000 companies' digital services to a halt:

    First, the outage originated in AWS's northern Virginia cluster, known as US-EAST-1, which is the world's largest cloud provider.

    It's the third time in five years a major internet outage has stemmed from the northern Virginia data centre.

    Second, we know the Domain Name System (DNS) is also the likely cause of the fault. Likened to the internet's "phone book", the DNS can "paralyse" entire applications and services if it domain name resolution stops working, according to experts.

    We have been promised a detailed rundown by AWS of what went wrong, known as a "post-event summary", although that could take weeks or months.

    A brightly-lit AWS sign at an expo-displayImage source, Getty Images
  3. All AWS services returning to normal, Amazon sayspublished at 01:16 BST 21 October

    Amazon Web Service programs have "returned to normal operations", bringing an end to the 15-hour long outage.

    In an earlier update the company said the underlying cause of the outage had been fixed, while noting there were still connectivity issues on some AWS services.

    This had included its Lambda service, which helps smartphones and computers run apps from the cloud.

    AWS now says the final technical problems have been fixed and it will later provide a full rundown detailing the cause and scope of the outage.

  4. 'It's always DNS' - engineer on major outagespublished at 00:22 BST 21 October

    An IT security officer at Thomas Cook's parent company says issues with a server's Domain Name System (DNS) typically triggers major outages.

    Marek Szustak, who works at eSky Group, explains that AWS's outage is to do with DNS - explained as the internet's 'phone book' - which is the "foundation of network communication".

    He says that when domain name resolution stops working, entire applications and services can come to a halt, no matter how well they are designed.

    Though a "seemingly minor piece of infrastructure", Szustak points out that DNS can "paralyse" the largest cloud environments.

    "As engineers say, it's always DNS," he says.

  5. Consumer champion warns of scamspublished at 23:40 BST 20 October

    Consumer champion Matt Allwright warns that scammers could take advantage of people during an internet outage.

    "People will be struggling to access things they enjoy using, or eally need to use, when it comes to things like banking," he told Morning Live.

    He urges users not to panic if they see an error message when they're trying to use their banking services.

    "Just ask yourself how important it is to make that transaction right now," he says. "If the answer is that it's very important, then get in touch with your bank directly using a number that you trust."

    "The last thing you want to do is entertain a phone call out of the blue right now saying I can offer you a solution," he says.

    Allwright adds that it's best to stay calm and wait until the issue resolves.

  6. Game developers describe difficulty working amid outagepublished at 22:49 BST 20 October

    Freya Scott-Turner
    Live reporter

    A man in a white suitImage source, CHARLES OSITA ODILI

    As we've been reporting today, one of the platforms that has been affected is Roblox.

    On the platform, users can create their own games using a development tool called Roblox Studio. People like Charles Osita Odili have built entire careers doing this.

    As a result of the outage, many have been twiddling their thumbs today.

    Odili explains that both the Roblox game and the development tool went down for many developers, both in the UK and in the US. "Once it all went down, we couldn't access either, meaning we couldn't work on our respective games for a couple of hours," he says.

    "Sometimes the Roblox Studio still works so you can at least get work done, but that wasn't the case today," he adds.

    "Back in the older days of development... we all manually saved work," he says, so the consequences of the outage would have been much worse.

    Developers are used to outages though, he adds: "Everyone just goes to do something else for a few hours then as soon as someone gives the green light that its back up again, everyone hops back on."

  7. MyFitnessPal health app reports 'increase in errors'published at 22:14 BST 20 October

    In a new update, MyFitnessPal - a health app that allows users to track their food intake, exercise and weight - says it's seeing an increase in errors.

    "We have started to see an increase in errors resulting in issues with log in, account creation, delays in search, and using smart logging and meal planning," it says.

    "We will continue to monitor closely and provide updates."

    This comes as Amazon Web Services says there are still "significant errors" for some services, despite the underlying problem having been fixed.

  8. Duolingo promises to save users' streakspublished at 21:48 BST 20 October

    a screenshot from the Duolingo app saying that 'profiles are currently unavailable'

    Users of the language-learning app Duolingo will be familiar with the concept of streaks.

    They're a marker of how many days in a row you've practiced on the app. Indeed, some dedicated users rack up streaks in the thousands.

    Duolingo has promised it will protect these streaks in spite of the outage.

    "PSA: your streaks will be saved!" it says in a post on social media, adding thanks to users for their patience.

  9. What we know about today's internet outagespublished at 21:32 BST 20 October

    Sakshi Venkatraman
    Reporting from New York

    For those just joining us, we've been covering the outages on some of the world’s biggest apps and websites today.

    If you've been having trouble with technology platforms like Zoom, Lyft or Snapchat, you're not alone. Here's what we know so far:

    • The underlying problem today was a malfunction at Amazon Web Services, where something called "DNS resolution" was not working
    • Since a massive number of companies rely on Amazon, the issues were widespread
    • Amazon says the underlying issue has been fixed, but consumers are still reporting problems
    • Reports of apps being down started at 08:00 BST, and Downdetector told the BBC it had received reports stating it got 6.5 million reports and over 1,000 companies were facing problems
    • Popular financial apps like Venmo and Coinbase are still experiencing problems. Gaming giants like Roblox and Fortnite were impacted, but are back up and running
    • Even Amazon's online shop experienced some interruptions, one of our US reporters finds

    We'll continue to bring you the latest as this situation unfolds. If you are being impacted by the outages, get in touch.

  10. 'They have millions... They should be able to cover the basics'published at 20:50 BST 20 October

    Your Voice Your BBC News Banner
    An error screen for Hargreaves Lansdown
    Image caption,

    Mark tells us he was greeted with this error screen when he tried to take a look at his investments

    Retired banker Mark from Essex tells us he has been trying to access his account, which has thousands of pounds of investments, with financial services company Hargreaves Lansdown today but cannot log in.

    He says: “I used to work for a large corporate and we had an offsite system in case we ever had issues like this.

    “I think it is sad that Hargreaves Lansdown does not have a backup. They don’t seem to have a disaster recovery system."

    He's concerned that while the outage continues, he's unable to check on his investments.

    “It’s in my terms and conditions that I can have access to it whenever I want," he explains.

    “My worry is there is no disaster recovery and just pathetic excuses for not having a platform in case it goes wrong.

    “They have millions, or billions. They should be able to cover the basics in these cases.”

    • A notice on Hargreaves Lansdown's site now says: "Earlier problems clients may have experienced logging onto the website have now been resolved. These were caused by a widely reported third-party issue".
  11. Amazon says recovery continues but problems persistpublished at 20:50 BST 20 October

    Lily Jamali
    North America Technology Correspondent

    Amazon has been posting updates about the AWS outage roughly every half hour or so. In the latest, they say they continue to see signs of recovery across all AWS services.

    But problems persist, including with its Lambda service, which computer science experts say helps ensure your cell phone, computer, or television can run apps from the cloud.

    Amazon says users might see error messages when they try to run these outside services. They’re working on it.

    While it seemed the worst had been resolved, a new series of “cascading failures” seems to have arisen during the last few hours, according to Mike Chapple, an information technology professor at Notre Dame University.

    “It’s like when you have a large-scale power outage,” Chapple said. “Crews start working to try to bring it back on line. The power might flicker a few times,” but it’s possible “they’d only addressed the symptoms” and not the root cause.

  12. Premier League confirms Semi-Automated Offside Technology unavailable for start of Monday's gamepublished at 20:42 BST 20 October

    A West Ham player battles a Brentford player for the ballImage source, PA Media

    Monday’s premier league fixture between West Ham United and Brentford has started without the use of Semi - Automated Offside Technology, (SAOT) due to today's Amazon Web Services outage.

    SAOT is a system used to assist VAR (Video Assistant Referee) in making quicker and more consistent offside decisions. It became approved by clubs towards the end of the 2024-25 season.

    The Premier League confirmed that, should SAOT become available during the game, it will be implemented when appropriate.

  13. Educators struggle as Canvas service interruptedpublished at 20:28 BST 20 October

    Kayla Epstein
    US reporter, in New York

    Kadri Brogi, a technical support manager who also teaches at universities in New York and New Jersey, was hit by the AWS outage on several fronts.

    This morning, he realized he could not check the Ring doorbell camera he had installed on a second home in Albania for security purposes.

    And Canvas, a widely used education software that allows teachers to assign and review students' work, was down.

    “It impacts a huge amount of students and faculty," Brogi says.

    Instructure, the company that owns Canvas, alerted users on X that their programmes were experiencing "service disruption".

    Brogi, who uses the software as an educator for his online classes and helps maintain university infrastructure, warned about the implications for higher education.

    "You understand if its a power outage," he says.

    “But for AWS to have such a widespread outage is concerning."

  14. Users report issues with Grok, Lyft and more, according to Downdetectorpublished at 20:02 BST 20 October

    Though Amazon Web Services (AWS) has said it has fixed the underlying issue that has disrupted numerous apps and websites today, some users are still reporting issues.

    Downdetector now says users have been reporting issues with Grok, Lyft, Claude AI, Hulu and Reddit in updates posted on its X account., external

    It's unclear if the recent issues are related to the AWS outage.

    As we've been reporting, Downdetector says it has seen more than 6.5 million reports globally, affecting more than 1,000 companies, since the start of the outage.

  15. ‘No cars detected’ on Lyft left me scrambling to get to workpublished at 19:47 BST 20 October

    Sakshi Venkatraman
    Reporting from New York

    I’m embarrassed to admit how much I rely on rideshare apps, and my commute to join this page hit a roadblock this morning when I called a Lyft - in Texas where I got an early flight before work - and no cars were detected.

    Lyft was one of the many apps affected by today’s outages, with service going out for thousands of users and drivers.

    Despite scheduling my ride the night before, the app delayed me by nearly an hour, showing an eerily empty map devoid of the normal little cars buzzing around in the screen.

    When I finally got a match, the Lyft driver told me he had several rides scheduled that morning that were ultimately cancelled.

    He woke up at 04:00 to make the pickups – but the app didn’t let him accept any rides, he told me.

  16. Amazon says underlying problems fixed, now the blame game beginspublished at 19:23 BST 20 October

    Lily Jamali
    North America technology correspondent

    Amazon Web Services says the underlying problems that caused today’s outage are fixed. Now the blame game has begun.

    One computer science expert says some of the responsibility rests with the companies who use AWS.

    “Companies using Amazon haven’t been taking enough adequate care to build protection systems into their applications,” says Ken Birman, a computer science professor at Cornell University in New York.

    Outages like the one on Monday occur all the time, although not always at this scale.

    Birman tells the BBC that app developers should take care to invest in backing up mission-critical applications that live in the cloud.

    “We know how to make these systems stronger, and we know how to do it securely,” Birman says.

    The question of responsibility could well land in the courts.

    More than a year after the massive CrowdStrike outage, Delta Airlines is still wrangling with the company to recover more than $500m in losses.

    Even after CrowdStrike had fixed the issue, the airline said it had to manually reset 40,000 servers, leading to major flight delays over several days.

    Given how integrated these systems are, determining fault isn't always straightforward.

  17. Amazon shop has been struggling from outagepublished at 18:56 BST 20 October

    Madeline Halpert
    US Reporter

    An Amazon error message with a dog is displayedImage source, Amazon

    Amazon - the e-commerce platform – has been a victim of its company's web services outage too.

    I've tried to purchase several items on the e-commerce platform today, but each time, I get a message that "something went wrong on our end", often accompanied by a photo of a cute dog.

    On Downdetector, thousands of people have reported issues with the site, although the number seems to be decreasing since a peak of problems around noon US time.

    People are experiencing issues with Amazon's Alexa products too.

    We are hearing now that some customers have been able to make purchases on Amazon in the last few moments.

  18. Outage may lead to attempts to recover losses from big businesses, lawyer sayspublished at 18:24 BST 20 October

    After today's Amazon Web Services outage impacted many of the world's biggest businesses, some customers might be asking whether they can take legal action for any disruption they might have suffered.

    Henna Elahi, a senior associate at Grosvenor Law in London, explains that whether money can be recovered will depend on "several factors", including the contracts between the various parties and the severity of the outage.

    For instance, banking apps are among those that saw thousands of reports of issues.

    And, with users unable to make payments, this might have meant customers failed to complete purchases and breached contracts of their own.

    "This may very well lead to customer complaints and attempts to recover any loss caused by the outage from the business," Elahi explains.

  19. Outage hits US financial apps and media sitespublished at 18:06 BST 20 October

    In the US, users are reporting issues with a variety of websites.

    People are having trouble sending and receiving payments on the platform Venmo.

    Coinbase, the largest US cryptocurrency exchange, has been having issues with its services since 03:00 EST (08:00 BST).

    And trading app Robinhood, as well as language learning company DuoLingo, also appear to be experiencing technology problems.

    Several media organisations were hit by the outage too, including the Wall Street Journal and the New York Times.

  20. Starbucks says app saw 'limited interruption' after users report issuespublished at 17:42 BST 20 October

    Starbucks has confirmed its app experienced disruptions earlier in the day but says it is now working again.

    "I can confirm that our mobile order ahead app and pay features are all operating normally," a spokeswoman for the company says. "We experienced very limited interruption and were back up and running quickly."

    We have asked the company if the disruption was linked to the AWS outage.

    More than 1,800 people reported issues with the mobile app earlier in the day, according to outage monitor Downdetector.

    A line chart by the monitor now shows a steep decline in user reports since 12:00 BST.