<img height="1" width="1" style="display:none" src="https://www.facebook.com/tr?id=749646578535459&amp;ev=PageView&amp;noscript=1">

KaiNexus Blog

Everything Continuous Improvement

Subscribe

Twitter’s “Blame Processes, Not People” Moment

Posted by Danielle Yoon

Sep 28, 2020 3:12:13 PM

AdobeStock_68281158With the blistering speed of this summer’s news cycle, we can’t blame you if you’ve already forgotten about the single largest security incident in Twitter’s history that took place on July 15 but was a doozy. It was also a teachable moment for Twitter and every other organization that involves people and technology. 

What Happened?

Early in the day on July 15, 2020, Twitter’s automated detection systems flagged some suspicious activity. Several Twitter accounts with short handles, like @USA, @drug, @love, and others, seemed to be compromised. These short handles are valued among certain hacker communities, so Twitter knows to keep an eye out for unusual access. The appropriate people were notified, and they began looking into the issue. Security incidents like this are common when you have a platform with hundreds of millions of users.

But later, at about 3:00 PM EST, the cryptocurrency platform, Binance, sent a tweet announcing that it would give back about $52 million of bitcoin, an unlikely scenario. Within the next hour, 11 other cryptocurrency exchanges posted the same message. And then, at about 4:00 EST, Elon Musk tweeted a classic bitcoin scam in which followers are told that if they send bitcoin to a particular address, they will receive double the amount back in return. Soon other accounts, including those of Bill Gates, Barack Obama, Joe Biden, Uber, Apple, and more, did the same. Twitter was clearly under attack.


A System-Wide Approach to Process Improvement

But How?

Earlier in the day, some employees reported receiving suspicious phone calls. Someone from IT was calling customer service and IT employees and asking them to log in to reset their passwords. Many employees reported the calls, but somewhere between four and eight of them went to the hacker’s fraudulent website and entered their credentials, allowing the bad actors to gain access to Twitter’s back-end systems. The attack was so severe and widespread that Twitter was forced to stop all verified accounts from tweeting.

Now What?

This hack was not technically sophisticated at all. It relied on a technique called “social engineering,” where malicious actors target something much more vulnerable than an organization’s network, its people. So it may seem that the way to prevent a future similar assault would be to find the people who fell victim to the scam and replace them with smarter people who would recognize and report suspicious activity. Get better people and solve the problem. Right?

Not exactly.

Hiring people who will never make mistakes is not realistic and certainly not scalable. Executives at Twitter knew that the response to this problem required a much more systemic review and response. (There is no public information about what, if any, personnel actions were taken following the incident.)

After solving the immediate problem, Twitter dug in and found that its systems and culture contributed to an environment that made this attack possible. As a senior Twitter employee told Wired Magazine, “There was a systems-level failure. The whole thing should not have happened. The issue isn’t that someone got phished; it’s that once they got phished, the company should have had the right systems in place.”

On Thursday, September 24, 200, Twitter announced the new protocols it has implemented to protect its systems in advance of the upcoming election, including:

  • Limiting the number of employees who have access to critical systems
  • Improving threat detection and monitoring capabilities
  • Investing in additional training for employees and contractors
  • Rolling out phishing-resistant physical security keys
  • Dramatically increasing the number of privacy reviews
  • Implementing new pre-launch product security procedures

In short, what Twitter is doing is applying the Lean principle of mistake-proofing, or its Japanese equivalent poka-yoke (pronounced PO-ka yo-KAY). The goal is to use automatic devices or methods that make it impossible for an employee to commit an error or make the error immediately obvious once it has happened.


[Watch Now] How to Leverage Lean for Long-Term Success

In any organization with humans, errors are possible. No matter how attentive and brilliant folks are, once in a while, we all make mistakes. Twitter set an excellent example of how to step back and look at the conditions that made the problem possible.

Topics: Lean, Improvement Culture, Improvement Process

Recent Posts