Exim: the email address that took over mail servers

On June 5, 2019, the Exim project published a fix. The flaw it closed let attackers run any command on a mail server as root, the administrator account. They only had to hide the command in an email address. Exim is a mail server: the program on a Linux or Unix machine that accepts email from the internet and delivers it. The security company Qualys found the flaw and called it “The Return of the WIZard”. Eight days later, Microsoft warned of a worm that used the flaw to move from server to server. A worm is a program that spreads on its own.

How it worked

Exim has a small built-in language for its settings. Text written as ${...} is filled in while a mail is handled, much like a form letter. One of these items, ${run{...}}, starts a program.

Since version 4.87, released on April 6, 2016, Exim ran the recipient address through this function when delivery failed in certain ways. An attacker only had to write a command into the address, as in ${run{...}}@localhost, and make the mail fail. One way was to add more than 30 “Received” lines, the stamps that each mail server adds on the way. Exim then treated the mail as stuck in a loop. It ran the command as root, because by default it keeps these rights while it delivers mail. No memory trick was needed.

On the server itself, any user could do this at once. From the internet it was harder, because Exim’s standard settings accept only the addresses of real users on the server. But some changes to these settings removed that obstacle, for example on a backup mail server that accepted mail for another domain. Even with the standard settings, Qualys found a way. The attacker made Exim send an error message to a server under the attacker’s control. The attacker then kept that connection open for seven days, sending one byte every four minutes. After seven days, Exim gave up on the message and ran the command.

The name refers to WIZ and DEBUG, two old flaws in Sendmail, an older mail server, that also let outsiders run commands.

Reach and attacks

Because nobody had recognized the bug as a security flaw, most operating systems still shipped an affected version, Qualys wrote. The company showed the attack on an up-to-date Debian 9.9. One day after the fix, the security company Tenable searched Shodan. This search engine records what servers on the internet reveal about themselves. Tenable found more than 4.1 million systems with Exim 4.87 to 4.91 and 475,591 with the fixed version 4.92. So nearly 90 % of the Exim systems found were open to the flaw.

Such counts rely on the version number that a server shows. Debian, for example, fixed its Exim 4.89 with an update on June 5, 2019, without changing that number. So not every system counted stayed open.

The attacks began quickly. On June 9, four days after the fix, a security researcher reported the first attempts to exploit the flaw. On June 13, Microsoft confirmed an active worm. It urged customers of its Azure cloud to patch their virtual servers or restrict access to them. On June 14, the Canadian Centre for Cyber Security described the attacks. A first email made the server download malware. A scheduled task then kept the attackers on the server. An added key let them log in as root over SSH, the service for remote logins. A script searched the internet for more vulnerable Exim servers and attacked them. To hide, the attackers used Tor, a network for anonymous connections.

Most of these attacks installed crypto miners, the UK’s National Cyber Security Centre (NCSC) wrote in October 2019. Crypto miners are programs that use a hijacked server to create cryptocurrency. Since August 23, 2019, a ready-made module for Metasploit, a widely used attack toolkit, had made the attack easy. The NCSC counted more than 174,000 Exim servers in the UK alone that were still open to this flaw or to two Exim flaws found in September 2019.

In May 2020, the US National Security Agency (NSA) reported that Sandworm had exploited the flaw since at least August 2019. Sandworm is a hacking group of the GRU, Russia’s military intelligence service. The NCSC backed the finding and notified the affected providers in the UK. A few days later, the Canadian cyber centre reported Canadian victims from as recently as May 2020. In 2020, the security company Rapid7 still found about 87,500 servers with a vulnerable version on the internet. How much damage the attacks caused is not documented.

The fix and the response

The strangest part: the flaw had already been fixed. Exim 4.92, released on February 10, 2019, had changed the code, but nobody noticed that the change closed a security hole. Qualys found the flaw while reviewing the latest Exim changes and reported it on May 27, 2019. The Exim team wrote a patch the same day.

On June 3, the team warned of a “possible remote exploit”, still without details. It planned a coordinated release for June 11, so that Linux distributions could prepare their updates. After details leaked, it published the fix early, on June 5 at 15:15 UTC. Debian released its update the same day. Qualys published its advisory that day, at first without the attack methods, and the full version on June 6.

The Exim team made no new release for the old versions, because it considered them outdated. Operators had to install their distribution’s update or move to version 4.92. In May 2020, the NCSC advised providers to install version 4.93 or newer at once.

The National Vulnerability Database rates the flaw 9.8 out of 10 on CVSS 3.1, “critical”. Red Hat gives it 9.0 on the older CVSS 3.0 and rates the attack as harder to carry out. The US cyber agency CISA added the flaw to its catalog of exploited vulnerabilities on January 10, 2022.