What HubSpot's Outage Taught Us About Effective Crisis Communication

If you’re a HubSpot user, then you’re well aware of the recent outage the platform experienced last week.

On the afternoon of Thursday the 28th, HubSpot reported that it “experienced an issue with one of our critical infrastructure systems that supports many parts of our platform, causing issues with tools such as email, page publishing, sales tools, reporting, and more.”

Unfortunately, it wasn’t a quick fix, and the issue resulted in a multi-day outage for certain tools in the platform, and delays in others.

The outage, along with HubSpot’s response to it, teaches valuable lessons for both SaaS companies and companies that rely on platforms like HubSpot to power their business.

For SaaS companies, it teaches about effective crisis communication strategies when your product has outages, or even more minor issues.

It is also an important reminder of how reliant our businesses are on these all-in-one platforms, and begs the question - what would we do if a key system crashed entirely?

HubSpot’s Response

Upon the initial system malfunction, HubSpot directed users to a status page that was updated hourly (sometimes more) with the status of progress that had been made in addressing the underlying problems causing the outage.

This practice isn’t uncommon for many SaaS companies that experience outages.

However, affected users became increasingly frustrated with the gaps in time between updates, and the limited information they held.

They took to social media to share their concerns.

.@HubSpot I love you and everything, but you have got to do a better job of creating transparency and expectations when something goes wrong.

It's been essentially the same message for 14 hours. pic.twitter.com/m8MRO5Zrib

— Doug Davidoff (@dougdavidoff) March 29, 2019

Again, this type of reaction isn’t uncommon for software outages - particularly ones that businesses rely on so heavily to conduct their day-to-day operations.

However, by actively listening to customer feedback, HubSpot was able to utilize crisis communication skills and lessen the impact of what could have been a PR disaster.

First, they took note of users’ frustrations with stagnant updates and began adding more detailed explanations of progress that the team was making.

Crisis Communication Done Right

Additionally, they implemented a well-crafted communications plan consisting of a blog post and two emails where they explained the situation to customers with transparency, empathy, and a clear plan of action to prevent future mishaps.

The plan seemed to be successful, with customers sharing positive feedback on social media after the first email was sent out.    

— K. Lovisa Hogstrom (@k_lovisa_91) March 30, 2019

The HubSpot Outage email was so good. 1. To the point - we messed up. 2. Why it happened. 3. next steps. Proactively acknowledging the point of failure and doing it quickly is key to customer success.

— Emilia D'Anzica (@emiliadanzica) March 29, 2019

took full responsibility for the situation in your recent blog posts. Being a business owner myself, I know issues happen. The way you handled it reaffirms why I became and continue to be a @HubSpot customer. Thank you 🙏

— Andrew Breiter-Wu 🌎☀️💰 (@AndrewBreiterWu) March 30, 2019

Let's breakdown their initial email to show how to do crisis communication the right way.

Catering To The Emotional Impact

One of the first pillars of good customer service is knowing how to apologize. Often, when issues arise in business, too many companies hide behind dry business jargon to explain the technical reasons for the problem.

Though widely used, that approach is rarely effective because customers’ frustration stems from human emotions, and they need to know that you empathize with and understand those grievances.

In HubSpot’s case, the opening sentence in the email reads:

“We’re sorry. HubSpot’s only mission is to help businesses grow better, and today we weren’t able to do that for you and many of our customers.”

Before getting into any details, HubSpot started off with a straightforward apology and assumed full accountability for the issue.

Having this right in the opener frames the rest of the email as much less robotic, and shows customers that above all, HubSpot understands that they fell short, and aren’t making excuses.

Additionally, they also addressed the criticism surrounding their status updates in the beginning hours of the outage.

“We also know that many of you were often left waiting for a response longer than we would have liked. The entire HubSpot team has been working tirelessly to solve the issue and to give you the communication you deserve. In the times when we didn't have reliable updates to give you, we didn't want to make any promises that we couldn't keep. We understand the frustration that you must have felt, and we hear you.”

Here, they explained their reasoning for the seemingly infrequent updates without being defensive. Again, like the example above, they displayed empathy to their customers in this situation and let them know the concerns they had were valid.

Answering The Key Questions

Of course, the most important part of crisis communication is providing an explanation.

HubSpot provided a brief “What Happened” paragraph that covered the basics of what caused the situation to occur, and what impact it had on customers:

“On the morning of March 28 EDT, we experienced an issue with a critical infrastructure system that supports many of the tools that our customers rely on. It caused widespread delays across much of our core functionality, including email, pages, sales tools, reporting and CRM tools.

These processing delays resulted in further issues for customers attempting to use the tools.

For you, this meant that you may have experienced delays across HubSpot and were unable to use parts of your HubSpot functionality.”

While some were frustrated with the vagueness of “an issue with a critical infrastructure system,” I actually thought that it was better to skip the highly technical language and instead focus on what actually impacts customers.

The brevity and straightforwardness of this section allowed both technical and non-technical users to gain an immediate understanding of why the outage occurred, and what the impact would be to their accounts.

Additionally, HubSpot answered another question: Why is it taking so long to fix?

“At this point, we had to make the decision on where we wanted to direct our processing power, either between restoring key data points that were at risk, or enabling customers to use the tools but abandoning the data. We made the decision to focus our efforts on restoring as much data as possible.

This decision meant that it has taken us longer than we would have liked to fully recover.”

Again, this explains their team’s decision-making process while still taking accountability, and without being defensive.

While it reads simply, this is a very fine line to walk down, and one that many businesses find difficult to execute successfully.  

Sharing Prevention Plan & Next Steps

Most importantly, HubSpot shared what they’re planning to in the future to prevent this from occurring again.

“In the coming days we will be running a full retrospective of what caused this issue and our response. To give you to the transparency you deserve, we’ll publish these findings publicly in an in-depth post that provides more information about the cause of this issue, as well as the steps we’re taking to make sure this doesn’t happen again.”

While empathizing and being transparent about an issue are important, demonstrating that you have a plan in place to not repeat this situation in the future is key to gaining back your audience’s trust.

Additionally, they made good on their promise to follow-up on the specific changes that are being made.

On Thursday afternoon, they published the in-depth retrospective they spoke of. The post shared exactly what went wrong that caused the outage, and a clear list of the steps they’re taking to ensure it doesn’t happen again. If you want more insight on the technical issues, you can read their full blog post here.

Here are the preventative measures they’ve vowed to take to reduce risk moving forward:

• Splitting up key clusters within their infrastructure - The outage was largely caused by systems being interconnected so much so, that if one failed, it impacted a lot of different aspects of the tool. Now, HubSpot is going to split up the clusters so if one of them fails in the future, “an outage of this size and scope won’t happen,” and it will make recovery much faster.

• Invested in a dedicated reliability team - While HubSpot did have a team that worked on reliability, they weren’t dedicated to that alone. Moving forward, HubSpot has made investments to increase resources for a team that is “solely focused on reliability, upgrades, and testing”
• Increase audits and scenario testing for massive failures - To ensure that these issues are caught early on, the reliability team will increase the frequency and thoroughness of its tests of critical infrastructure so they’re prepared for different scenarios that can occur.  
• Commit to better communication moving forward - Again, HubSpot references the frustration with their update process at the start of the outage. From now on, they stated that they “promise to provide more consistent updates on the status page and give greater information about tools affected” in the event of any issues moving forward.

Why Effective Crisis Communication Is So Vital For Business

If you work with technology, you know that issues like the one HubSpot experienced are inevitable.

Functionality that was once working perfectly can break suddenly, systems can lag, and as we’ve seen in HubSpot’s case...those can have big consequences.

All-in-one platforms like HubSpot are often great because they’re so interconnected...but the downside is, if it breaks, it can have a huge impact on all areas of your business.

HubSpot’s approach demonstrated that they understood the gravity of this problem and that it was more than a simple inconvenience.