Exim: the email address that took over mail servers

On June 5, 2019, the Exim project published a fix for a flaw that let attackers run any command on a mail server with the rights of root, the administrator account, by hiding it in an email address. Exim is a mail server: the program on a Linux or Unix machine that accepts email from the internet and delivers it. The security company Qualys found the flaw and called it “The Return of the WIZard”. Eight days later, Microsoft warned of a worm, a program that spreads on its own, that used the flaw to move from server to server.

How it worked

Exim has a small built-in language for its settings. Text written as ${...} is filled in while a mail is handled, much like a form letter. One of these items, ${run{...}}, starts a program.

Since version 4.87, released on April 6, 2016, Exim ran the recipient address of a mail through this function when it could not deliver the mail in certain ways. An attacker only had to write a command into the address, as in ${run{...}}@localhost, and make the mail fail. One way was to add more than 30 “Received” lines, the stamps that each mail server adds on the way, so that Exim treated the mail as stuck in a loop. Exim then ran the command as root, because by default it keeps these rights while it delivers mail. No memory trick was needed: Exim simply ran the command.

On the server itself, any user could do this at once. From the internet it was harder, because Exim’s standard settings accept only the addresses of real users on the server. But some changes to these settings removed that obstacle, for example when a server accepted mail for another domain as its backup mail server. Even with the standard settings, Qualys found a way. The attacker made Exim send an error message to a server under the attacker’s control and kept that connection open for seven days, sending one byte every four minutes. After seven days, Exim gave up on the message and ran the command.

The name refers to WIZ and DEBUG, two old flaws in Sendmail, an older mail server, that also let outsiders run commands.

Reach and attacks

Because nobody had recognized the bug as a security flaw, most operating systems still shipped an affected version, Qualys wrote; it showed the attack on an up-to-date Debian 9.9. One day after the fix, the security company Tenable searched Shodan, a search engine that records what servers on the internet reveal about themselves. It found more than 4.1 million systems with Exim 4.87 to 4.91 and 475,591 with the fixed version 4.92, so nearly 90 % of the Exim systems found were open to the flaw.

Such counts rely on the version number that a server shows. Debian, for example, fixed its Exim 4.89 with an update on June 5, 2019, without changing that number, so not every system counted stayed open.

The attacks began quickly. On June 9, four days after the fix, a security researcher reported the first attempts to exploit the flaw. On June 13, Microsoft confirmed an active worm and urged customers of its Azure cloud to patch their virtual servers or restrict access to them. The Canadian Centre for Cyber Security, Canada’s cyber security agency, described the attacks on June 14. A first email made the server download malware. A scheduled task then kept the attackers on the server, an added key let them log in as root over SSH, the service for remote logins, and a script searched the internet for more vulnerable Exim servers and attacked them. To hide, the attackers used Tor, a network for anonymous connections.

Most of these attacks installed crypto miners, programs that use a hijacked server to create cryptocurrency, the UK’s National Cyber Security Centre (NCSC) wrote in October 2019. Since August 23, 2019, a ready-made module for Metasploit, a widely used attack toolkit, had made the attack easy. The NCSC counted more than 174,000 Exim servers in the UK alone that were still open to this flaw or to two Exim flaws found in September 2019.

In May 2020, the US National Security Agency (NSA) reported that Sandworm, a hacking group of the GRU, Russia’s military intelligence service, had exploited the flaw since at least August 2019. The NCSC backed the finding and notified the affected providers in the UK. A few days later, the Canadian cyber centre reported Canadian victims from as recently as May 2020. In 2020, the security company Rapid7 still found about 87,500 servers with a vulnerable version on the internet. How much damage the attacks caused is not documented.

The fix and the response

The strangest part of the story is that the flaw had already been fixed. Exim 4.92, released on February 10, 2019, had changed the code, but nobody noticed that the change closed a security hole. Qualys found the flaw while reviewing the latest Exim changes and reported it on May 27, 2019, and the Exim team wrote a patch the same day.

On June 3, the team warned of a “possible remote exploit”, still without details, and planned a coordinated release for June 11, so that Linux distributions could prepare their updates. After details leaked, it published the fix early, on June 5 at 15:15 UTC. Debian released its update the same day. Qualys published its advisory that day, at first without the attack methods, and the full version on June 6.

The Exim team made no new release for the old versions, because it considered them outdated: operators had to install their distribution’s update or move to version 4.92. In May 2020, the NCSC advised providers to install version 4.93 or newer at once.

The National Vulnerability Database (NVD) rates the flaw 9.8 out of 10 on CVSS 3.1, “critical”. Red Hat gives it 9.0 on the older CVSS 3.0 and rates the attack as harder to carry out. The US cyber agency CISA added the flaw to its catalog of exploited vulnerabilities on January 10, 2022.