For several hours one early Monday afternoon in mid-December 2009, a Google intern teased apart the equivalent of a sonar blip on his screen.
Someone had tripped an alarm.
He sighed. “Probably another intern.”
Google had just introduced new tripwires across its network, and alarms were going off incessantly. The company’s security engineers were now spending all their time trying to decipher which blips marked an imminent attack, an engineer accessing a spammy poker site, or simply an intern stumbling down the wrong digital hallway. Almost always, it was the latter.
“There’s a Fog of War, but there’s also a Fog of Peace,” Eric Grosse, Google’s affable vice president of security engineering, told me. “There are so many signals triggering, it’s hard to know which ones to go after.”
Some inside the company likened it to Pearl Harbor. That Sunday morning in December 1941 on the Hawaiian island of Honolulu had started peacefully enough. Lieutenants were still familiarizing themselves with the naval base’s new radar system when a radar operator on the far end of the island informed the on-duty lieutenant of an unusually large blip on his radar screen—signs of a fast-approaching aircraft fleet over 100 miles away. The lieutenant’s first reaction was, “Don’t worry about it.” He assumed the blip was a squadron of B-17 bombers due in from San Francisco, not the first wave of Japanese bombers.
With so many new blips popping up on Google’s screens that December, it was simply human nature to prefer the simple, benevolent explanation—a disoriented intern—to the reality, an imminent nation-state attack.
“We weren’t trained to think about spies,” Heather Adkins, the freckled, thirtysomething director of Google’s information security team, would later recall. That Monday afternoon, Adkins was just wrapping up another Google meeting about China. The company had tiptoed into the Chinese market three years earlier and was still struggling to navigate Beijing’s draconian censorship rules.
Adkins was something of an anomaly among the mostly male, testosterone-fueled coders she managed. Most had a deep distaste for authority. They buried their heads in code by day and lived vicariously through virtual role-playing games by night. Adkins was more of a history buff, who spent her off hours reading up on the Middle Ages. She saw her security gig at Google as the digital equivalent of stopping medieval invaders in the ancient world. Her job was simple: “Hunt down evil.”
As her meeting came to a close, Adkins glanced at the clock. It was 4 pm. She might just be able to beat the rush-hour traffic if she left work early. But as she headed for the door, her intern beckoned, “Hey, Heather, check this out.”
The blip on his screen had metastasized and was now moving at dizzying speeds in and out of employees’ computers, across Google’s network. Whoever was on the other side of the screen was no intern. “It was the fastest cyberattack we had ever seen,” Adkins recalled. “Whoever they were, they were clearly practiced. This was not their first rodeo.”
As late afternoon turned to evening, the blip grew more animated. It was now bouncing from computer to computer, winding its way through Google’s systems in unpredictable patterns, in search of something. The intern stayed glued to his screen through dinner, when he broke to join the rest of the team in Google’s café. There, over burritos, he relayed the strange trail of the blip that was taking on a life of its own. Seated at the table that evening was Grosse, Adkins’s boss, and several other security engineers.
She saw her security gig at Google as the digital equivalent of stopping medieval invaders in the ancient world.With his glasses and graying hair, Grosse had a Socratic, professorial quality. He was one of the few Google directors to forgo an office so he could sit with his engineers. It was not uncommon to find him sloped on a couch, computer on lap, or staying late to dine with twentysomething engineers. That night Grosse listened intently to the intern’s account, asking questions, trading notes with others at the table. A consensus emerged: Whoever this was, they appeared to be in the beginning stages of reconnaissance. An insider? What were they after? Salary records? As the men wrapped up their meals and walked out to the volleyball court that evening, nobody had so much as guessed a foreign nation.
As Mountain View retired for the night, the sun was just peeking up over the Swiss Alps in Zurich when Morgan Marquis-Boire, the dreadlocked then 30-year-old hacker, logged in. Google’s engineers in Zurich—or “Zooglers,” as they called themselves—referred to their offices as the “real Mountain View” for its Alpine backdrop. But Marquis-Boire always felt that Google’s Swiss headquarters, with its oversize rainbow-colored Google logo, stood out like a leering, oversize clown in Zurich’s old Hürlimannplatz.
For years, Hürlimannplatz had been home to an old Swiss brewery. But once the brewers discovered a spring bubbling up inside the building’s brick walls, they started producing their own mineral water. Europeans from across the continent made weekend pilgrimages to the square’s mineral-fed fountain to taste the purest water Europe had on offer. These days, the well had been converted into a thermal bath and spa. It was an oddly Zen setting in which to be triaging the very beginnings of a cyberwar.
That morning, Marquis-Boire picked up where the Mountain View intern had left off, following the blip as it ping-ponged across Google’s network, looking more and more ominous. He barely noticed the snow silently blanketing Zurich’s rooftops and steeples.
This was no intern. “Google isn’t a nuclear enrichment facility,” he told me, “but in terms of security, it comes pretty damn close.”
Whoever this was, they had managed to bypass the toughest security measures he had yet to see. And now they were riding roughshod across Google’s networks, indiscriminately accessing systems that did not fit the typical employee’s digital path. The list of possible explanations for the anomalous blip was getting shorter and shorter, until there was no other explanation: Google was under attack.
“We’ve caught a live one!” Marquis-Boire shouted. It was hard not to jump on top of his desk, pound his chest, and yell, “Shit is on!”
For years, he’d been chasing imaginary ghosts and pointing out the dangers of weak security. Now he finally faced something real. It felt like vindication.
By the time he relayed his analysis back to Mountain View and left the office for the night, it was 11 pm. The streets were muffled with snow. Usually he biked to his apartment in Langstrasse, Zurich’s equivalent of Amsterdam’s red-light district. But that night he decided it would be better to walk. He needed time to process. As his combat boots crunched through the streets, his mind flashed back to a presentation in Las Vegas two years earlier, where he’d boldly declared to a large audience of hackers that “threats from Chinese hackers are overrated.” Recalling his words now, Marquis-Boire could only smile: “History sure has a way of coming to bite you in the ass.”
Come morning in Mountain View, it was clear that this was no fire drill.
By 10 am, Google’s entire security team had been briefed on the attack. As morning turned to afternoon, however, the activity flat-lined. Whoever was behind their screens had retired for a few hours. But that evening the blip returned with a fervor. Several engineers elected to pull an all-nighter, tracing the attackers’ movements into the wee hours of the morning.
The intruder was clearly a night owl—or operating in a different time zone. By the time bleary-eyed engineers briefed their fresh-faced cohorts the following day, there was no question they were facing the most sophisticated cyberattack Google had ever seen.
It was time to call in the specialists. Google’s first call was to a cybersecurity shop in Virginia called Mandiant. In the messy world of security breaches, Mandiant had carved out a niche for itself responding to cyberattacks, and was now on the speed dial of nearly every chief information officer in the Fortune 500.
Kevin Mandia, Mandiant’s founder, was like Harvey Keitel’s meticulous, fast-talking character, the Wolf, in Pulp Fiction, called upon by corporate America to clean up the aftermath of the bloodiest digital breaches, extortion attacks, and cyberespionage campaigns. Google asked Mandiant to get to Mountain View as soon as possible. “Just one thing,” Google’s executives said. “Don’t wear suits.”
The following day Mandiant’s forensics team arrived at the Googleplex. They’d foolishly ignored their clients’ advice and showed up in dark suits and glasses. Googlers in hoodies took one look at the men and concluded they had to be federal agents.
Grosse and Adkins ushered the men into their improvised war room, a small, nondescript conference room overlooking Moffett Field, the former naval air base. In the distance, Mandiant’s team could just make out the glimmer of the San Francisco Bay before someone drew the shades and slapped a sign on the door: this conference room offline until further notice. The next hour played out in what Kevin Mandia affectionately calls “Upchuck Hour.” Mandiant’s team insisted that Google fork over everything: Firewall logs. Web logs. Emails. Chats. They grilled Grosse and Adkins’s team about everything they knew so far, blasting them with a series of questions that could be best summed up as “Who the hell do you think might have done this?”
Time was of the essence. With every second that passed, the blip was gathering more data, more code. There was a good chance that the attackers had already planted backdoors in Google’s systems for quick return access. Google’s employees were practically vomiting information on the table—anything and everything that might offer Mandiant’s investigators a digital crumb or fingerprint to trace the attackers’ identity and motive.
Mandiant’s founder was like Harvey Keitel’s meticulous, fast-talking character, the Wolf, in Pulp Fiction, called upon by corporate America to clean up the aftermath of the bloodiest digital breaches, extortion attacks, and cyberespionage campaigns.At Google offices across the globe, internal investigators started summoning employees in for interrogation. Why had their device accessed that file, that system, that one piece of data? What were they after? But by the end of the day, it was clear that this was no insider. An attacker had infiltrated their machines from the outside. Mandiant’s investigators honed in on the logs, looking for any malicious links or attachments that employees might have clicked on, inadvertently granting attackers entry to their systems.
They had seen this a thousand times. Mandiant’s clients could spend millions of dollars on the latest and greatest in newfangled firewalls and antivirus software, but security was only as good as the weakest link. And usually the weakest link was a human who clicked on a simple phishing email or message containing something nasty. The messages could be quite persuasive. The attacker might mimic a FedEx tracking notice or an HR manager. Somebody, somewhere in the organization, almost inevitably fell for it and clicked.
As Mandiant’s investigators wound their way from infected machine to machine, they picked up on a common thread: several Google employees in the company’s Beijing office were trading messages with colleagues, partners, and clients using an external Microsoft chat service. As investigators sifted through their chats, they found one blaring red flag. Each had clicked on a link attached to the same menacing three-word message: “Go Kill Yourself.”
For those next few days in December 2009, Google’s war room became a tangle of data and minds as Grosse and Adkins began pulling engineers from every corner of the company, telling them to ask any friends with any security experience to come work at Google. They started poaching digital spies from Fort Meade and the Australian outback and security engineers from Google’s competitors up and down Highway 101, offering immediate, no-questions-asked $100,000 signing bonuses.
The war room quickly became something of a curiosity to other Googlers, particularly as Sergey Brin, the company’s energetic cofounder, became a regular presence on their floor. Brin, who spent his spare time performing as a trapeze artist, was hard to miss. Often he’d come speeding into the office on Rollerblades or clownish outdoor elliptical bikes, wearing full-body luge racing suits or, at a minimum, neon slippers.
Brin, a Russian-Jewish émigré, had taken a personal interest in the attack. He was a picklock. As a student at Stanford, he’d experimented with various lock-picking techniques. He also happened to be one of the world’s foremost experts in data mining, extracting meaningful patterns from mountains of data. This kind of forensic pursuit was in many ways what Brin did best. But he also began to take the assault personally. Brin’s identity—and, one could argue, Google’s corporate identity too—was inextricably linked with his family’s escape from the Soviet Union in the late 1970s. He saw the attack as a direct assault on the founding principles of Google itself, summed up by its three-word motto: “Don’t be evil.”
With each visit to the war room, Brin was further convinced that this was not the work of some basement dweller; this was a well-resourced attack. Baked inside that three-word clickbait, “Go Kill Yourself,” was a link to a website hosted in Taiwan that executed a script containing a zero-day exploit in Microsoft’s Internet Explorer browser. Once the employees in Google’s China offices clicked on the link, they inadvertently downloaded encrypted malware that gave Google’s attackers a foothold and allowed them to plug in and out of Google’s network.
No kid—Brin didn’t care how good they were—was burning a Microsoft zero-day exploit on Google and encrypting their attack code out of curiosity. This attacker was after something bigger. And they’d taken unusual care in hiding their tracks. The level of obfuscation alone suggested that this was the work of a highly trained, well-funded adversary. Brin made it his personal mission to find out who.
As more engineers joined their effort, the investigation moved to a second, larger conference room, then a third, and finally to an empty building across campus, where some 250 employees were now tasked with finding who had broken into Google’s network, what they were after, and why. Their quest had taken on such purpose that engineers were now refusing to go home. Several took to sleeping on campus.
“When the building is on fire, it’s hard to keep the firefighters away,” Adkins recalled.
As the holidays neared, Adkins encouraged her team to go home to sleep and shower, even as she was becoming a permanent fixture on campus herself—and a slightly ridiculous sight at that. She’d run out of clean clothes just as Googlers were raiding the on-campus merchandise store for last-minute holiday gifts. At five-foot-three, Adkins found herself managing the digital investigation of a lifetime in an extra-large neon-green Google sweatshirt.
Holiday trips were canceled. Employees were not permitted to tell their loved ones what was keeping them away. Adkins managed to make it to Vegas to see her mother for Christmas, but she spent the entire time chained to a computer. Grosse managed only a brief cameo on Christmas Day.
“I just had to tell my mom, ‘Something big is going on. Trust me. It’s important,’ ” Adkins said.
The level of obfuscation alone suggested that this was the work of a highly trained, well-funded adversary.The obsession with the attack started giving way to paranoia. On the way to campus one morning, Adkins spotted a utility worker emerging from a manhole. “I thought, ‘Oh my God, that person is trying to backdoor our fiber on campus.’ I started to wonder if someone was listening to our phone calls.’ ”
In Zurich, engineers began to worry about their personal safety. They wondered how personal Google’s assailants were willing to make this. They were civilians effectively doing counterintelligence against what was clearly a well-funded adversary. Several took to watching their backs on their late-night commutes.
As the weeks went by, Google’s security team was learning that they had good reason for concern. The attacker had begun to show the telltale signs of a sophisticated adversary: a Chinese government contract group Mandiant had come across before, a group the National Security Agency tracked by the classified pseudonym Legion Yankee.
Legion Yankee was among the murkiest—and most prolific—of the more than two dozen Chinese hacking groups that NSA hackers tracked, as they raided intellectual property, military secrets, and correspondence from American government agencies, think tanks, universities, and now the country’s most vibrant technology companies.
Chinese cyber theft took two tacks. The majority of hacking crusades were conducted by the China’s People’s Liberation Army’s Second and Third Departments. It was clear from their targets that various PLA units were assigned to hack foreign governments and ministries in specific geographic locales, or to steal intellectual property in distinct industries that benefited China’s state-owned enterprises and economic plans.
The other approach was less direct and more episodic. Increasingly, high-ranking Chinese officials at China’s Ministry of State Security started outsourcing attacks on high-profile targets—political dissidents like the Dalai Lama, Uighur and Tibetan ethnic minorities, and high-profile defense contractors in the United States—to freelance hackers at Chinese universities and internet companies.
The state identified these hackers for their skills, which often far exceeded those of their PLA counterparts. Plus, if anyone ever traced back the attacks to these individuals, Beijing could claim ignorance. “That way Beijing can say, ‘It’s not us. It’s these hackers we can barely control ourselves.’
__________________________________
Excerpted from This Is How They Tell Me the World Ends: The Cyberweapons Arms Race. Used with permission of the publisher, Bloomsbury. Copyright © 2021 by Nicole Perlroth.
