Microsoft Outlook/365 still down after 12 hours

(twitter.com)

243 points | by player85 4 hours ago

40 comments

  • nickdothutton 29 minutes ago
    Microsoft is unable to run a reliable hosted email service, given infinite resources. What people don't realise is that the former is in part a consequence of the latter. Without a forcing function, complexity multiplies, silent fragility accumulates, end-to-end understanding dwindles. There is no incentive to simplify.
    • zkmon 6 minutes ago
      Great insight. I would add that forcing function comes from the self, not from outside. The force is essentially the communication between the parts. When the communication breaks down, things fall apart, establishing a new kind of communication across the pieces to achieve equilibrium. The physical structures also have the same rule. They will crumble into a pile of lego bricks to achieve a new state of equilibrium and a simplified network of communication across the pile.
    • torben-friis 26 minutes ago
      Today is sounding like one hell of an incentive.
      • silvestrov 12 minutes ago
        Why?

        Do they lose any significant number of customers? Do anybody in leadership get fired?

        Do reading the twitter status updates give you the impression of an organization that takes responsibility seriously?

        Or do it sound more like an organization that is optimized for evaporating responsibility?

        I think this will change nothing.

        • torben-friis 2 minutes ago
          Well, at the very least there's going to be a lot of people pointing at their broken SLAs.

          >Do reading the twitter status updates give you the impression of an organization that takes responsibility seriously?

          Not implying otherwise, to be clear. Very far from defending MS.

        • steve1977 3 minutes ago
          And why would it? Most people using Microsoft 365 have bullshit jobs anyway. All they do is performative. They are not productive, with or without access to their e-mail and calendar. It doesn't make a real difference.
      • bigfudge 18 minutes ago
        Seriously, who will change provider because of this? Our institution is so brainwashed and hidebound to Microsoft it would take a lot more than this. It seems like an abusive relationship: the bad times just increase fear generally, so make change less likely.
        • throwawaybackag 15 minutes ago
          It's the same with AWS, Cloudflare. There is also this weird effect, often commented on here, where if half the Internet is down, everyone understands when you are down. If I use some random new start up for my infrastructure or services, and it goes down, customers and executives are furious. So it seems we are stuck here.
        • eterm 14 minutes ago
          It might not force a change away, but it definitely reduces the chance of people switching to them.
  • magnio 2 hours ago
    Apparently this single outage means Office 365 uptime this year drops to 3 nines (2 nines for this month). You can claim SLA credit worth 50% service fee by applying before December 31.
    • tuom1s 28 minutes ago
      Do you have official reference for this?
      • Arbortheus 3 minutes ago
        There's two parts, the actual downtime and the SLA agreement...

        At my day job, we have a calendar integration that polls Microsoft's Graph API across hundreds of Microsoft 365 tenants. We saw our first metric spike in 5xx errors at 2026/11/12 17:33 UTC. As of writing it's 2026/11/13 09:44 UTC and we are still encountering a full outage, so 16 hours and 11 minutes.

        With a SLA calculator [1], assuming no other downtime this month and entering 16h11m, we get:

        * Yearly SLA: 99.82%

        * Monthly SLA: 97.78%

        Next, what is the actual agreement? I found an SLA document list here [2].

        I downloaded Service Level Agreement for Microsoft Online Services (WW) English (2026/10/01). There's a heading for Exchange Online that says:

        Uptime Percentage : Service Credit

        < 99.9% : 25%

        < 99% : 50%

        < 95% : 100%

        Which seems to agree with what the original commenter said for this month.

        [1] https://uptime.is/reverse

        [2] https://www.microsoft.com/licensing/docs/view/Service-Level-...

      • agilob 12 minutes ago
    • igleria 1 hour ago
      Thanks for the tip!
    • tyingq 1 hour ago
      "Office 364" is apparently a trending term.
    • JumpCrisscross 2 hours ago
      I couldn't specifically say since my admin.microsoft.com fails pukes on a wildcard certificate for azureedge.net.

      Also, multiple bankers I'm speaking to about Anthropic's IPO can't send or receive work e-mails. So that's a thing.

  • vld_chk 47 minutes ago
    This is arguably one of the biggest outages in modern history. AWS region level outage was more visible, but email is a backbone of the modern world.

    BBC is running live coverage of it at the level of government and ministers, feels surreal

    • arlcode 32 minutes ago
      I wonder whether this will help the already strengthening trend of some governments to unstick themselves from Microsoft (in particular their cloud).

      I don't expect a fundamental shift but the boss having no access to his mails seems certainly like something I would bring up quite a bit.

      • vld_chk 26 minutes ago
        I am certain this is the direction of thought at least for Europeans. IIRC France recently moved away from Microsoft. If right now they do not have this outage, I can only imagine how much it boosts confidence in such decisions
    • devsda 26 minutes ago
      Felt bad for the engineers that it had to happen on a Friday but for a multi day outage I guess it's a blessing in disguise.

      Barring corona & year end holidays, this is probably the closest we can get to a global work holiday.

      • vachina 14 minutes ago
        Even consumer Outlook is down.

        This means you cannot do anything that requires email auth (OTP, signup etc)

    • oblio 7 minutes ago
      Who wants to bet that this is ultimately due to AI, either an action taken by AI or just general AI code rot coupled with developer knowledge decay?
  • aucls1 4 minutes ago
    https://thenational.co.za/news/2026-11-13-cisa-confirms-hack...

    This is not related to this outage, right?

    16 hours and nothing...

  • BLKNSLVR 2 hours ago
    How many Microsoft employees are working on this? It's concerning that one of the biggest, most valuable companies on the planet, that runs services internationally, can have one of its primary services have gone down for this long.

    6 hours ago they seemed to not exactly understand what the problem was, and that was 6 hours into the outage.

    This is concerning.

    • EmX_3000 0 minutes ago
      Mythical man month, maybe throwing everybody in the company at it isn't the right thing to do. But they must be in a panic, and their own communications internally may well be hampered, could be MS everything is out for MS as well as everybody else.
    • pjmlp 1 hour ago
      Let me put this way, the Visual C++ standard library has officially one FTE working on it, everything else are open source contributions.

      WinUI 3.0, WinAppSDK, and related CsWinRT, C++/WinRT are in a ridiculous position six years after Project Reunion announcement, due to ever meagre resources.

      XBox app keeps under delivering user experience, and bugs on patch Tuesdays are getting a joke by now.

      All this in a four trillion valued company.

      • ctmlr 1 hour ago
        Maybe acting like jerk-offs got them to four trillion but there are consequences to burning all your social capital. A large camp of software developers have genuinely hostile feelings towards Microsoft, I would bet that this affects the quality of talent they can hire and these are downstream affects.

        Lately it seems like more non-techies are souring on the company.

        • sznio 31 minutes ago
          >A large camp of software developers have genuinely hostile feelings towards Microsoft

          which, besides hatred, manifests in malice, like Nightmare Eclipse. I wouldn't be surprised if this outage was an engineer leaving a parting gift, using a bug they probably knew of for years but management didn't allow them to fix.

        • sterlind 1 hour ago
          talent they can hire? large swaths of the company have been in a hiring freeze. they just sent 25% of the company voluntary severance offers.
          • rsalus 1 hour ago
            they've still been hiring, just on an attritional basis - headcount has remained flat for a few yrs now. it is pretty clear though that they want to operate even leaner.
        • mihaaly 50 minutes ago
          Jerk-offs can skim the profit of going big yesterday while the consequences affect everyone, but only in the future of yesterday. Who does such things, one might ask? Jerk-offs, off course. And PE too (likely not a disjoint set).
        • kakacik 42 minutes ago
          Was there ever a time when this was not true? M$ was always among top hated companies since 90s, arrogance, fuck-U to users all around, absolutely ignoring any standards larger than them, and famous 'Embrace, extend, and extinguish' which was pathetic to watch for any engineer worth 2 cents.

          Top notch user experience peak was Windows XP and every subsequent release was massively larger, clunkier and slower while requiring way more resources, was spying on the users more and more. Office has similar story, peak of usability was maybe 2007 with ribbon introduction (which still a lot of power users hated with passion since it was far from clean design, more seemingly designed by committee of marketing guys).

      • Cthulhu_ 22 minutes ago
        This is what I don't get about big companies and their software; you'd think every application, every moving part would have a (small?) team of core maintainers, people that get paid well to know everything there is to know about something, ranging from (in microsoft's case) Paint to a particular configuration panel.

        I get that this doesn't scale though - a lot of features are just "done" at some point and keeping one or more people whose sole job is to maintain this may lead to them bolting on features just to stay relevant / stop being bored. But big tech can easily afford to keep people like that on.

        (I mean they keep (kept?) tons of middle management on, who knows what they even do? Not emailing right now that's for sure am I right?)

      • logicchains 30 minutes ago
        >the Visual C++ standard library has officially one FTE working on it

        One guy but it's Stephan T. Lavavej the guy who the "STL" was literally named after.

    • sobiolite 21 minutes ago
      Not so long ago, I was assigned to a project that had to be built on Azure due to client requirements. Because we were underhanded and it was an MVP, we'd elected to use PaaS as much as possible: Postgres Flexible Server, Static Web Apps, Easy Auth, etc. On paper, these all looked fine. They had lots of documentation, example projects, GitHub repos, blogs, roadmaps, etc. Then we started building and running into issues, and we realised a bunch of these offerings were basically unmaintained. There were features that had been stuck in "preview" status for literally years. The blogs and roadmaps were no longer updated, and issues were piling up in GitHub.

      I started digging into the historical blog posts, GitHub comments, etc. and I realised everybody at MS who'd been working on them, SWEs, PMs, architects, etc. had either left the company, or moved to other teams. And the one constant about the internal moves was that people were now working in some "AI" related project or product. Literally, I didn't find a single person who wasn't now working on AI.

      Clearly, there has been a huge internal push to get into AI. Resources have been reallocated, and anything non-AI related has been starved. And since working on an AI project is probably considered the only route to advancing your career, anybody with any ambition has moved to a team doing something AI-related. The result is that huge numbers of products and teams doing vital but non-AI related work have been hollowed out, and now we're seeing the consequences.

    • nikanj 32 minutes ago
      In our field, once you understand what the problem is, the problem is usually solved. Unless it's broken hardware, obviously
    • InsideOutSanta 1 hour ago
      The ones who actually knew how the system works were too expensive to retain, so now it's just some interns typing frantically into Claude Code.
      • digikata 50 minutes ago
        "Too expensive" as measured by bean counters that don't understand the actual risk profile - or do understand it but have such a short window of strategic concern it leads to this. It would be an interesting analysis to stack up costs to Msoft vs cost of labor cut as this outage evolves.
      • StingyJelly 1 hour ago
        Company policy is to use copilot but copilot is down so they are just hitting F5 and hoping
        • aucisson_masque 1 hour ago
          Haha this is ridiculous and yet there is some truth in it.
          • kakacik 36 minutes ago
            It may not be true now but 10 years down the line? This is disaster in the making yet while most people recognize that (when filtering out most fanatical pro-ai folks claiming everywhere how they get 10x or 100x work done) nobody is able to do anything. Sort of legacy technical debt being delivered but of different kind than usual, that train that will hit you since you stand on the rail line yet its moving 5kmh and you see it from 200m, but unable or unwilling to move away (Austin powers scene with steamroller does it better than me [1]).

            Imagine if say BMW decided they don't want to manufacture their motors, suspensions, drivetrain etc. Outsourcing 100% of that knowledge to external suppliers, for all models, from now on forever. How do you think market would reacted to such move ie by valuation of stocks?

            [1] https://www.youtube.com/watch?v=y_PrZ-J7D3k

        • SillyUsername 1 hour ago
          This should be the top comment, my eyes are watering to this one
    • micw 2 hours ago
      > How many Microsoft employees are working on this?

      all 3!

      • digikata 49 minutes ago
        They're more effective than ever because with this outage, every manager above them can't email to ask for status every 5 min
        • pferde 41 minutes ago
          Alas, Teams are working, or at least the chat and voice call portions of it. So managers are in a calling and chatting frenzy right now, pulling every body else in.
          • kakacik 17 minutes ago
            search in teams doesn't, very convenient for our Windows team to be unreachable for those of us still on WFH
        • motionlessveloc 30 minutes ago
          Fortunately they can still ask on Teams!
    • karlkloss 2 hours ago
      How may Microsoft employees are left anyway?
      • phoronixrly 1 hour ago
        The lights are on but there's no one home
        • theflyingelvis 40 minutes ago
          Your mailbox is not your own
        • rdtsc 57 minutes ago
          The wheel is still spinning but the hamster is dead.
      • wolfi1 1 hour ago
        I guess the old light bulb jokes about MS will not be applying anymore
    • boudin 54 minutes ago
      /s Their internal AI is full steam on it. The issue is just that their internal AI infrastructure is down too, so AI just need to fix itself first. Then it will focus on fixing the issue. Oh, wait...
    • wooger 56 minutes ago
      And which made a load of staff redundant in the last month.
    • surgical_fire 54 minutes ago
      This is HN. We should be asking why frontier models couldn't still dolve this! How many parallel agents does MS have working on this?
    • DarkCrusader2 2 hours ago
      I don't understand why they haven't asked copilot to fix it yet, make no mistakes.
      • mint5 1 hour ago
        Don’t be silly, they tried that but copilot is/was down!

        That’s why it’s taken over 12 hours now, no one’s left that knows without copilot’s help.

      • cokeandpepsi 2 hours ago
        [dead]
    • xyst 47 minutes ago
      the vibe coding group in charge of this product has decided to delete the production database and backups
    • krige 1 hour ago
      Maybe Nadella is still trying to figure out which of the 8000+ sacked employees knew how the system works /s
      • sterlind 1 hour ago
        he was such a nice and effective CEO the first few years. he's really changed.
        • kakacik 14 minutes ago
          Was he? I struggle to see it even in hindsight, and definitely his moves were bunch of FU to common users. Maybe in some specific area you care about, but not their core products like windows and office
        • cyberax 48 minutes ago
          Not really. He killed the QA department immediately and consciously put MS onto the "slide into mediocrity" path.
          • motionlessveloc 28 minutes ago
            Google killed its test engineers (or, rather, forcibly merged them into SWE) at around the same time without measurably denting uptime.
        • pjmlp 58 minutes ago
          Even with some of the Microsoft <3 Open Source, I do miss the Steve Balmer era in regards to Windows development experience.
        • bell-cot 1 hour ago
          Executive compensation corrupts, chief executive compensation corrupts absolutely.
  • daitangio 39 minutes ago
    Is for this reason that my email is hosted on a cheap VPS:

    https://gioorgi.com/2020/mail-server-on-docker/ https://github.com/docker-mailserver/docker-mailserver

    It is very easy to set up (just a bit of time on average every 6 month to follow upgrades) and work out of the box.

    Europe outlook users are experiencing problems too.

    • u1hcw9nx 23 minutes ago
      When your most important contacts use outlook, it does not matter what you do. Sending and receiving email when almost nobody else is sending and receiving is only small help.

      Some problems are not solved by individual choice. If 30-40% of your important contacts use service that is down, your ability to work goes down dramatically.

      I have currently important 3 daily contacts down they use MS services extensively contacted and said they can't do much of anything because their work process and the process of their contacts is down.

      • mahboi 19 minutes ago
        Enough of my contacts use Gmail. But both of these have more uptime than my self-hosted email.
      • vachina 18 minutes ago
        Personal email is almost exclusively for OTP, reminders/invoice.
    • theK 31 minutes ago
      There probably are some very good reasons not to self host your email server but apparently my own, self hosted, email server's uptime over the last decade is now better than that of outlook.com :-D

      Quick lets collect all options people have to self host! in addition to docker-mailserver im aware of

      - https://github.com/stalwartlabs/stalwart

      - https://github.com/mailcow/mailcow-dockerized

  • pitabas106 1 hour ago
    I have always felt it's less about email redundancy and more about having a backup plan. If outlook goes down tomorrow, does everyone know how to keep work and client communication moving? that is something worth thinking about.
    • emil-lp 24 minutes ago
      Who wants to bet that Microsoft is using slack to communicate now?
      • enopod_ 15 minutes ago
        Fire up that old IRC server!
    • SubiculumCode 52 minutes ago
      For the amount of uneeded emails about future emails about other emails, probably no real work was lost.
  • throwawaybackag 18 minutes ago
    Something I've been thinking about with the last few major outages is that as there is more and more consolidation (fewer large companies running a higher share of the world's infrastructure and critical services) is that we become increasingly exposed to massive disruption like this. Would some sort of mandated maximum occupancy per instance resolve this? So Outlook.com is allowed to support 100m accounts per instance. Then you are required to spin up Outlook2.com. You are not allowed to have any shared infrastructure. Yes it's wasteful and expensive, but these are companies with billions in revenue, so that's the price you pay. It just seems that keeping these huge, incredibly complex services running is turning in to an impossible task. Very much a hot take so interested in the opinions of peope who have worked at this scale.
  • enopod_ 1 hour ago
    Good morning and happy Friday the 13th everyone
  • jruohonen 29 minutes ago
    "Okay? Nothing. It’s just terrible. Every day is a nightmare. As recently as 1972, most buildings collapsed and rusty nails were found in cereal boxes all the time. Okay, so when you’re faced with statistics like this, it’s very difficult to be optimistic. So, hey, why not think about the statistics? I want to just put that out of your mind."

    https://www.centennialsoftwaresolutions.com/help/transcript-...

  • bob1029 1 hour ago
    All 3 of my outlook accounts are still dead.

    I am looking at other current events and beginning to wonder if there is some kind of correlation.

  • karlkloss 2 hours ago
    Why would anyone introduce a single point of failure out of their control into their company?

    We're doing anything to avoid single points of failure in our products, but our IT is doing the opposite.

    • baliex 2 hours ago
      Outside of tech and disaster recovery, people _really_ don't understand that some redundancy--in this context--is a good thing.

      The idea of paying for a backup system, or even having one on standby for situations like this is just not on people's minds, or if it is, it's simply not affordable because it would cost you too much compared to your competition.

      Razor thin margins and generally getting away with it is what makes this work. Until it doesn't.

    • srdjanr 20 minutes ago
      I don't see a reasonable alternative here. Pay for another email/office suite provider, and duplicate all mails and files? Twice the cost and exposure surface just to cover 0.01% of downtime.

      Sure, it makes sense for mission critical companies (utilities, emergency services), but I don't think it does for an average company. Especially that you don't take much blame if everyone is down.

    • Arainach 2 hours ago
      The difference between a distributed system and a monolith is that in a monolith you have some idea what your single point of failure is, but in a distributed system you don't know your dozen single points of failure until they happen in production.
      • cowlevel 1 hour ago
        [dead]
      • adzm 2 hours ago
        I love the dependency cycles that prevent things from being started up again from scratch.
  • eddythompson80 1 hour ago
    Having friends who work at Microsoft, this is not really surprising at all. The company has always had deep structural problems. The AI change has really just exposed it and is aggressively accelerating the inevitable implosion. It's truly in a free fall that is planned and orchestrated perfectly by their upper management. Their CEO and VPs are convinced you don't need engineering know-hows any more because of AI, and they got all the management layers to sincerely agree with them. Not in a malicious or subverting way, but in a "you can't get a man to understand something his salary depends on him not understanding it" way.

    Microsoft has a moat, but their moat is having billions of dollars to spend on data centers. It's a failure everywhere else. It'll continue to rake in billions in revenue because people need compute now, but it's rotten at the top.

  • MisterMunchkin 1 hour ago
    This is going to cause absolute chaos. Basically every government organisation and business in the UK runs on outlook. The whole day is going to be a write off. It’s like a bank holiday!
    • robjan 44 minutes ago
      We're coming to the end of the Friday 13th business day in East Asia. Can confirm it caused chaos for the whole day.
    • hi_hi 32 minutes ago
      Same is Aus. Wife works for government, they got sent home early (typical gov that means 30 minutes before official clock off)

      The biggest issue has been so many security and approval flows depend on email, so all processes have ground to a halt.

    • erdoc69 1 hour ago
      Just the UK? Lol, you can add many, if not most government and private companies and educational institutions in the US and countless other countries to that list. The government and corporate world is grinding to a halt.
    • mattlondon 23 minutes ago
      Citation needed.

      Government I have no idea, but loads of businesses don't use Outlook - alternatives do exist. I've personally not used Outlook for well over a decade now.

      Quick google AI mode check says:

      - MS 365/Outlook: ~40%

      - Google Workspace: ~25%

      - Zoho: ~2%

      And then a bunch of others - Rackspace, Zimbra, IBM WebSphere Lotus Notes Blue For Enterprise XML Watson Edition et al

      • kakacik 8 minutes ago
        Every government globally with all parts and agencies is purely microsoft. Some EU states are trying to move away from it for obvious reasons (not this event, more like national security and sovereignty which you don't have with MS products).

        Exchange and Active directory are the biggest obstacles by far, bureaucrats can learn how to work with open source office alternatives easily, and underlying OS can be anything as long as you have few icons to click on.

      • oblio 5 minutes ago
        > Zimbra, IBM WebSphere Lotus Notes Blue For Enterprise XML Watson Edition et al

        I see that you're a hater. Outlook sucks but your solution is to cure cancer (Outlook) with Alzheimer's (Zimbra, Lotus Notes)?

    • byronsharman 1 hour ago
      This observation applies to many other countries, too. Mmmm centralization is pretty fun!
  • SubiculumCode 50 minutes ago
    ..but why is it so centralized? It makes these events so final.

    MY University's whole email system is Outlook/365, and that also seems kind of dumb to me.

    • srdjanr 11 minutes ago
      It's much better UX if everyone within the company uses the same office suite/email, it makes collaboration much nicer when everything is well integrated. And people get used to one system.

      My guess is that having duplicate system would cost (in money for double subscriptions and lost productivity) much more than the benefit of being available 0.01% more. Especially for universities where almost nothing is really urgent and can't be delayed by a day.

  • msie 23 minutes ago
    They used AI to create something they didn't wholly understand to debug.
  • ReDress 1 hour ago
    [dead]
  • DavCreator 1 hour ago
    • cornfieldlabs 1 hour ago
      The amount of racist memes targeting Indians on the replies.
      • xyst 42 minutes ago
        it's twitter, what do you expect?
  • fuglede_ 56 minutes ago
    Self-host your email.
    • murphy1312 39 minutes ago
      That would be a good idea, until you realize that everyone you send emails to (in a business context) also uses MS 365.
    • Arbortheus 22 minutes ago
      Big tech are trying to take this option away from us.
  • sedan_baklazhan 2 hours ago
    26 Oct: Microsoft offers voluntary buyouts to 60,000 employees as AI leads to efficiency push

    It's just a wild theory, but maybe there are still too much employees left, standing in the way of AI? I would try firing another 60,000 first.

    • gb2d_hn 2 hours ago
      No one knows if this is AI related, but it's easy to imagine that where changes written by AI fail in production, if AI can't find the solution (which will be common as AI access to production systems is typically reduced), then the devs won't either, or at least it will take them some time.
      • karmakurtisaani 1 hour ago
        Perhaps the velocity is the problem, not that AI inherently produces worse code (you just need one bad or just inexperienced engineer have their PR slip through to have bad human written code in you code base). When the velocity is high, there's less time to battle test each change and new code piles up on the old one. When the velocity is low, shitty code fails in production, but people are aware of what was recently changed and can find and fix the problems.
        • _kulang 1 hour ago
          Same thing, now, just faster: one bad engineer can now ship a million lines
          • karmakurtisaani 50 minutes ago
            Well, yes. My point was more that even if the quality was better, the speed of change is a problem. In particular in mature systems, I should add.
      • lkramer 1 hour ago
        The bigger issue with having your systems written by AI, is that nobody knows how the system works anymore and can't pinpoint where the likely failure points are.
      • gbalduzzi 1 hour ago
        Maybe the error itself isn't event AI related, but the engineers that knew how to fix it were let go
      • solid_fuel 1 hour ago
        The status updates have a pretty strong "LLM-driven agent failing to debug a complex system" feel to them. I hope it's not the case, but I could imagine a scenario in which inexperienced engineers - or experienced ones who have lost touch with the codebase - are deferring to LLMs to drive the troubleshooting and fix here.
        • koyote 1 hour ago
          They sound exactly like frustrating conversations I'd have with a cheap model and an overextended context window.
    • Zecc 1 hour ago
      Ugh. My sarcasm detector is down. Shouldn't have hosted on MS servers.
  • binaryturtle 5 minutes ago
    Nothing could go wrong if you force/bind millions or billions of people to a single point of failure. Of course, I doubt Microsoft ever will have to pay a single cent for lost work hours/ money across the globe. They probably have a fine print against that.

    Hopefully some of their "customers" learn a lesson and switch to a software without this side effect in worst case. There's plenty of alternatives (LibreOffice, OpenOffice, etc.)

  • whatever1 2 hours ago
    Who needs senior engineers. We are going to have LLMs troubleshoot billion-scale systems!

    What could go wrong, said all of the hyperscalers. Enjoy your multi day downtimes.

    • gb2d_hn 1 hour ago
      I do actually believe that LLMs and process around them will continue get better and become the accepted, but that there's going to be a 'fossil record' of a lot of problems having been seeded in these early years of AI that will take some time repair.
  • aucisson_masque 1 hour ago
    > We're aggressively pursuing all pathways to reduce impact

    I love this kind of wording. It makes everything suddenly more serious lol.

    • byronsharman 1 hour ago
      What a fancy way to say, "we highkey have no idea why it's broken"
  • keyle 1 hour ago
    At least Microsoft 360 still has a nice sound to it.

    No Outlook, they should make this a public holiday!

    • vincnetas 13 minutes ago
      Its microsoft 360 not 365 (days in year). So they have four more days.
  • pelagicAustral 1 hour ago
    If Teams is down I take that as a service upgrade
    • pferde 34 minutes ago
      Alas, Teams is working, although parts of it which are integrated from other services are not, like calendar (exchange), or file sharing (share point).
  • yusaydat 8 minutes ago
    > We're investigating reports of users experiencing issues accessing Exchange Online mailboxes through multiple connection methods in the Americas.

    Globally, not just in the Americas. Other Microsoft 365 services are also affected.

    > We determined the elevated load isn't driven by an increase in traffic, but by a number of servers being unexpectedly unavailable; straining the remaining infrastructure.

    > We found a core process on affected servers is becoming overwhelmed, causing capacity loss.

    > We’re identifying alternate workstreams that can be applied to increase capacity in the face of sustained traffic management imbalances.

    Hmm maybe they can migrate their infra to a hyperscaler to prevent capacity related outages like this in the future while working on a fix.

    > Our investigation indicates the failed health checks may originate from a specific component that's responsible for processing server health.

    Someone prompted "health check not healthy please fix MAKE NO MISTAKES!!!".

    > we're moving additional traffic off the affected infrastructure, and we're restarting affected servers to return them to service

    > In parallel, we've initiated a restart of all affected servers and are monitoring health as they return to service.

    > We're also preparing automated recovery systems to continually detect and restart unresponsive servers, to maintain service stability while this change progresses.

    > We're actively restarting and validating the affected servers to return them to service.

    > To validate a potential recovery, we're isolating a portion of infrastructure, restarting affected components

    Jeez, have you tried turning it off and on again?

    It seems that not just the engineers but also the people responsible for incident communications were replaced by AI.

  • hilbert42 24 minutes ago
    Ho-hum, my LibreOffice on my local machine hasn't skipped a beat all day!

    Isn't there an old saying about putting all one's eggs into one basket?

    • srdjanr 7 minutes ago
      Well then your one basket is your machine, or your backup system. And Office and Onedrive still work IIUC, so you're not really comparing apples to apples.
  • harvey9 1 hour ago
    This is very serious: I cannot access Viva Engage.
    • asimovDev 47 minutes ago
      TIL that Yammer turned into that
  • danielsoto317 2 hours ago
    The repeated comments on their twitter feed are weird, is it an LLM writing it? why can't they just wait until its fully fixed?
    • Arainach 2 hours ago
      Regular communication is an important part of outage/crisis communication, even if that communication is "nothing new to share".
    • InsideOutSanta 1 hour ago
      It's actually nice to have some updates on what they're doing that go further than just "we're investigating."
    • a77ec820a40f6 1 hour ago
      [flagged]
  • venzaspa 43 minutes ago
    From where I'm sitting, everything is working but Exchange Online which is absolutely fine by me.
  • redrix 1 hour ago
    “Co-pilot, fix the outage. Make no mistakes.”
  • ieie3366 57 minutes ago
    Just by looking at the O365 UI changes they have been doing tells me there’s a massive competency crisis within that team.

    It would not surprise me if the entire project is complete and utter (human-written) slop

  • mrheosuper 38 minutes ago
    Token maxxing can't help huh ?
  • aktuel 2 hours ago
    Welcome to Outlook/364
    • mint5 1 hour ago
      If only this year were a leap year Microsoft would be in the clear
  • vachina 2 hours ago
    Funny, my self hosted email has a better uptime than this pos
    • cowlevel 1 hour ago
      Email is actually designed with intermittent connectivity in mind. The sending server buffers outgoing emails until they're accepted by the receiving server. Typical settings are to retry every 4 hours, with a permanent failure being counted after 3-7 days.
    • gb2d_hn 2 hours ago
      I would take 99.0pct self hosted over having to use outlook!
  • devsda 1 hour ago
    Their current engineers must have stored their KT notes from their laid off and leaving senior engineers in OneNote on their Mac, and might have been working with copilot to find the root cause /s