VMWARE FUSION FOR SECURITY RESEARCHERS



2019 Nov 11

Glowing human silhouette falling through holographic VMware logo

Of the many complex and interconnected software components that make up the modern, computerized world the operating system (OS) bears special significance and demands heightened attention from cybersecurity researchers and practitioners. In a force-on-force context, operating systems constitute the ‘terrain’ in which attackers and defenders coalesce and coevolve while vying for supremacy and achievement of their respective goals. It stands to reason that a competent infosec professional should seek to maximize his or her capabilities with respect to installing, configuring, using, administrating, exploiting, analyzing, and fully understanding the OSes he or she is likely to encounter.

Virtualization products aren’t new. Creating a virtual machine (VM) to run more than one OS at a time has long been a fantastic alternative to the inconveniences of dual-booting for computer hobbyists, power users, and IT workers. Moreover, virtual machines are the basic unit of modern data centers, and the general population of non-technical internet users have all unwittingly benefited from virtualization in the form of ubiquitous cloud services.

With that said, you might be wondering why I’m writing about this topic in 2019. Isn’t virtualization played out? Aren’t all the cool kids working with microservices, infrastructure-as-code, or similar nowadays? Fair enough. I readily admit that there’s absolutely no secret sauce (nothing new, flashy, revolutionary, etc.) in this post. Regardless, having the ability to quickly and efficiently provision and pilot any mainstream OS is a core competency for anyone working in infosec, and doing so with VMs is often the best approach. Handling VMs is a fundamental skill worth sharpening and an indispensable prerequisite before any OS-oriented security research can be done.

I’ve seen firsthand how too many stop short of fully realizing the benefits of virtualization products consistently. I hope my advice (read: not dogma, but merely what’s worked in my situations) can assist someone to go further than their current methodology and evolve their daily research habits into something even better than before. Here’s a breakdown of what I cover in this post:


  1. Hardware and Software Selection
  2. Planning & Organizing
  3. Isolation & Usability
  4. Full Clones & Linked Clones
  5. Snapshot Strategy
  6. Use Case: Daily OPSEC
  7. Use Case: Filesystem Forensics
  8. Use Case: Traffic Analysis
  9. Use Case: Memory Forensics
  10. Backup & Automation


This post focuses on configuration and operation of one’s personal workstation (as distinct from designing a standalone lab environment) with the everyday needs of security professionals in mind. I’ll describe how to manage and utilize VMware VMs to improve your day-to-day OPSEC for common tasks, and I’ll also present a few ideas for how to get the most out of these VMs while performing a variety of security research tasks.


Hardware and Software Selection


Modern operating systems are highly complex and undergo constant change, so it can be quite a challenge for security professionals to master OS internals. I think it’s fair and uncontroversial to say that it’s impossible for a single practitioner to pull this off for every distribution, version, and build of all the OSes installed across the world. All one can do is narrow the set of OSes one chooses to study and prioritize them according to his or her real-world interests or responsibilities.

If you work in corporate security, then this certainly means Microsoft Windows and probably Apple’s macOS too. If your organization maintains an online presence or has a non-trivial internal network then your list could also include other OSes like ones based on Linux, other Unix-like systems (the BSD family, Solaris, etc.) as well as network operating systems like Cisco IOS. For some, mobile OSes (Apple iOS, Android-based, etc.) and/or embedded systems are relevant. To better focus on the most essential parts of this topic let’s narrow our scope to the most commonplace desktop and server OSes like Windows, macOS, and various GNU/Linux distributions. Installing each in a VM is a great starting point for convenient and repeatable experimentation.

Windows, macOS, and Linux icons

Doing so for Windows or Linux-based systems is usually uncomplicated, but macOS, on the other hand, presents a more inconvenient licensing obstacle. I’m no lawyer, but it’s generally understood among IT workers that Apple’s software license for macOS requires that one install and run it on genuine Apple hardware. The license is believed to permit running macOS in virtual machines insofar as the VMs are, again, running on Apple hardware.

If, like me, you intend to follow the rules, then you too should be using a Mac at least for the macOS portion of your setup. I found it most convenient to run all of my VMs on the same machine (a MacBook Pro) instead of buying a dedicated, single-purpose Mac. Either tactic could be perfectly valid in your situation, as could cloud-hosted macOS as a service, but you’ll have to assess your own needs to decide.

For specs, think of the likely workload and which day-to-day activities are the bedrock of your productivity. For me, there isn’t a time that I’m not simultaneously running at least three VMs, and it’s not uncommon for me to run five or six at a time. Furthermore, my VMware library contains around 20 VMs, and all are stored on the internal SSD of my Mac. I advise making your storage and RAM selections according to your own predictions, but no matter what, you almost always want to avoid running VMs from a mechanical hard drive. An SSD is a hard requirement.

Also, if you have a consistent workspace (i.e. you don’t live out of a suitcase) then get yourself real peripherals and construct a thoughtful, cohesive ‘station’ for maximum productivity. If you plan on putting tons of hours into your studies, then you should consider items that enhance the usability of and interface to your machine like an external mouse, keyboard, and monitor. The biggest productivity boost I’ve experienced firsthand (specifically when running many VMs at once) has been thanks to my ‘super ultra wide’ display. At any rate, just get peripherals you know you’ll appreciate. You’ll thank yourself later.

Desk with large curved monitor & glowing keyboard

As this post’s title indicates, I chose to use VMware Fusion for my virtualization provider. Although my general rule is to prefer open source software whenever possible, VMware Fusion has a track record of generally outperforming other desktop virtualization solutions that run on macOS (e.g. Virtualbox, Parallels, etc.) especially with VM suspend/resume operations which I’m constantly doing. VMware is relied upon in enterprise production environments, and I’ve personally had better experiences (e.g. stability, responsiveness, etc.) using VMware products over others.

At the time of this writing the latest version of VMware Fusion is 11.5.0. I’m using VMware Fusion Pro for some important features not available in the standard version like VM cloning (linked clones are vital to my setup), custom virtual networking, and the ability to connect to VMware ESXi servers.


Planning & Organizing


Before you start creating VMs take the time to assess your needs and how to fulfill them as easily as possible. Here are five best practices to keep in mind during VM planning:

  1. Don’t. Repeat. Yourself. Eliminate redundant efforts. Solve a small problem then build on top of it.

  2. Eliminate or reduce manual labor. Your time, energy, and attention is limited. Be judicious.

  3. Focus on purpose. Don’t waste time or disk space on VMs that serve a vague, ill-defined goal.

  4. Embrace as-needed work. Create the VMs you need today. Deferring non-critical work is okay.

  5. Expect changes. OSes will change. Your needs will change. Don’t get complacent with ‘version 1’.


Planning doesn’t just mean brainstorming which OSes you need to install. The configuration of your VMs is just as important. The virtual hardware settings (amount of storage, RAM, network config, etc.) as well as the intra-OS software configuration (system settings, VMware Tools, application programs, etc.) should all be set up to fully meet your needs. At the same time, you’ll want to know how you’ll use your VMs and manage their lifecycle to avoid repetitive, manual reconfiguration. Cloning VMs can help here, so don’t dive into customizing your VM software before you see the big picture and all the tools at your disposal.

Once you have a few VMs you’ll realize that organizing your library is essential for fast, effortless day-to-day operations. But organization shouldn’t be an afterthought. Before you build your first VM, add your own folder on your host OS filesystem that will contain them all instead of using VMware’s default folder. I recommend going with a very short folder name and placing it outside of your user folder in a location that results in the full path being quick to type. On macOS, in either bash or zsh, you can run this command to make such a folder with normal ownership settings:

$ 
sudo mkdir /opt/vms/ && sudo chown $(echo $USER) /opt/vms/ && sudo chgrp staff /opt/vms/

Next, you should name your VMs so their underlying folders and internal files are less painful to type at the command line. For example, instead of naming a VM ‘Windows 10 Professional 64-bit’ try something shorter that doesn’t contain spaces such as ‘Win10_Pro_x64’ instead.

Finally, the VMware Fusion Virtual Machine Library program window allows you to visually organize your VMs by grouping them into nestable folders. Know that this folder system is for visual neatness only, and it doesn’t correspond to your actual filesystem. Consider creating a small number of these display folders as you please. I organize my VMs into these folders by lifecycle status (Archived, Templates, etc.) and their role (R&D, Operations, DFIR, Offense, etc.) in my overall workflow.

VMware Fusion Virtual Machine Library program window


Isolation & Usability


VMware’s desktop products are hosted (A/K/A “type 2”) hypervisors meaning they run atop a conventional operating system. In the case of VMware Fusion this means Apple’s macOS is the host OS. This is distinct from so-called bare-metal hypervisors (A/K/A “type 1”) which run on the computer’s hardware directly, and the difference has security implications. Generally speaking, hosted hypervisors present a larger attack surface that malicious guest VMs can potentially exploit relative to the often smaller attack surfaces of bare-metal hypervisors. Put simply, software vulnerabilities in programs like VMware Fusion are more plentiful, easier for attackers to exploit, and can lead to entire compromise of the host OS and, in turn, all the guest VMs.

That’s not to say bare-metal hypervisors are impervious to attacks, though. Every well-known virtualization platform has had publicly-disclosed vulnerabilities, including fatal ones. I only mention this aspect of hosted hypervisors to inform your own risk assessment. With all this said, I still perform hazardous operations inside my VMware Fusion VMs (e.g. intentionally executing known-malicious software) after putting the requisite safeguards in place.

WannaCry ransomware program window

Photo by 황승환 / CC BY-SA 4.0

You should evaluate VM settings and only enable the attack surface-exposing features that you truly need. Four major options for you to choose stand out here:

  1. Shared folders allow VMs to read and/or write to the files on your host OS filesystem.

  2. Shared clipboard allows VMs to read and write to a buffer managed by code running on the host.

  3. Drag and drop allows file transfer operations between the host OS and guest VMs.

  4. Unauthenticated promiscuous mode allows VMs to sniff a vmnet’s packets without explicit approval.


Determine whether these usability features can be safely enabled and to which VMs or vmnets they should be applied. If you use shared folders try to place them in a consistent location within macOS for future scripting or automation to be more reliable. Also, like your VM names, select folder names and locations so that manually typing out the path is speedy and less error-prone. Basically, avoid space characters, long names, and excessive folder nesting.

If you decide to enable the shared clipboard feature (e.g. to copy/paste URLs, CLI commands, credentials, etc.) consider only turning it on for the VMs that actually need it on a day-to-day basis. The same goes for drag and drop which I personally do not use.


Full Clones & Linked Clones


In VMware Fusion Pro your VMs can be readily copied, or cloned, to save you from repetitive OS installation and configuration tasks. Full clones are effectively exact copies of the source VM and all its contents (virtual disks, hardware config, usability settings, etc.) except for some small, deliberate differences, for example, to virtual hardware identifiers like the MAC address. These whole-copy clones take up the same amount of storage space as the source VM, so ensure you have enough free space before initiating a full clone.

Linked clones, on the other hand, produce a functional copy of an existing VM but without duplicating the source VM’s entire virtual disk. The result is that you can run a copy of a VM much more quickly and without the need for so much free disk space. The downside is that your linked clone may experience reduced disk I/O performance. Also, linked clones are completely dependent on the source VM, so you can’t move or delete the source VM’s files without breaking your linked clone.

But how does one know which type of clone is appropriate for a given situation?

Make a linked clone if you:

Make a full clone if you:


In practice, I end up spinning up linked clones far more frequently than full clones. The disk I/O performance difference is negligible for my use cases, and the conserved disk space allows me to retain more source VMs.

To create a clone, power off the source VM, take a snapshot, and clone the snapshot by selecting the type of clone, full or liked, that you need.

VMware VM snapshot program window


Snapshot Strategy


Taking snapshots of my VMs is probably the VMware feature I rely on the most. It allows me to set my own restore points so I can confidently rework the OSes and applications. Snapshotting also helps me to reduce repetitive, manual setup tasks. If I know I’m going to be restoring to a known-clean state over and over as part of a testing procedure, then I can stage all the files, configure all the settings, and place all the program windows exactly how I want them. Then, I take a snapshot, and when I eventually restore the VM’s state, everything is already staged, configured, viewable, and ready to go.

Every time a snapshot is generated the execution context (disk, memory, CPU, etc.) of the VM is saved to disk, and so one shouldn’t overuse this capability. Making excessive snapshots can have a negative impact on the VM’s performance. Here are four more best practices to inform your VM snapshot strategy:

  1. Don’t take a snapshot without a clear purpose. Know when and why you’ll restore to it.

  2. Reduce the number of chained snapshots. Remove intermediate snapshots whenever you can.

  3. Use brief names. VMware already saves snapshot date/time and VM name/state/size metadata.

  4. Don’t rely on AutoProtect snapshots as an alternative to a true backup solution.


Snapshotting with purpose requires you to know which aspects of your VMs you’ll be changing between each snapshot, and to group and order those changes to maximize each snapshot’s value as a restore point. The strategy that has worked for me has been to customize my VMs starting with settings that are unlikely to be modified and then changing aspects of the configuration that are progressively more and more likely to require tweaking in the future. Here are some example steps I might perform when building a new Windows VM from scratch:

  1. Install and activate Windows, install VMware Tools, and join Windows to the domain.

  2. Take a ‘base install’ snapshot.

  3. Install all applications and perform OS and application updates.

  4. Take a ‘running config’ snapshot.

  5. Make any test-specific changes, and stage your starting point.

  6. Take a ‘testing’ snapshot.


If you follow these steps then each snapshot has obvious value serves a clear purpose in scenarios like the following:

Restore to ‘testing’ if the current test iteration:


Restore to ‘running config’ if you need to:


Restore to ‘base install’ if you need to:


Each time you restore a snapshot you can choose whether or not to save or delete child snapshots. More often than not I end up deleting mine, but there’s no rule that says you can’t or shouldn’t keep them. If you save child snapshots, VMware Fusion will retain a hierarchical snapshot tree with multiple branches, and this may be valuable if you need to maintain more than one unique software configs or restore to more than one type of ‘testing’ snapshot back and forth.


Use Case: Daily OPSEC


Okay, now that we covered the fundamental VMware Fusion primitives and personal workstation design we can move on to practical applications for an infosec practitioner’s use cases. To start, let’s review a few methods by which a security professional can utilize VMware to strengthen his or her OPSEC with respect to daily workstation activities.

The first main OPSEC benefit of VMware VMs is hazard isolation. Users can disaggregate their otherwise commingled tasks into separate operating systems. For example, a user can create a different VM (e.g. a linked clone from a known-clean base install) for each aspect of their digital life such as social media, online banking, casual web browsing, and school or work. Then, by only performing the appropriate activities within each VM, the user can reduce the likelihood of cross-domain confidentiality, availability, or integrity harms while also decreasing the severity of harm if any single VM becomes compromised.

Orange and white barriers

Photo by ProjectManhattan / CC BY-SA 3.0

Another approach is to disaggregate tasks by perceived risk. You could separate workflows that require public internet access from tasks that have no such requirement, or you can login to your public online accounts with one VM but only interact with non-attributed ‘anonymous’ accounts or services within a different VM that might use a VPN. Another idea is to set up a dedicated VM only for working with your password manager or other secrets.

The second security improvement from using VMware VMs is that malware persistence can be made more difficult for an attacker to maintain. By regularly reverting your VMs to trusted, known-clean snapshots any undiscovered infections can be undone thereby reducing the long-term security harms of persistent spyware or other threats. I revert all my VMs to known-good snapshots whenever meaningful software updates are available, but at the very least, I restore whenever I complete a project or otherwise reach a good stopping point.


Use Case: Filesystem Forensics


Whether your discipline is offensive or defensive security research you’ll have to inspect filesystems eventually. It’s often enough to rely on the running OS to tell you what you need to know regarding files, folders, and other filesystem attributes, but, in terms of forensic reliability, nothing beats using a clean, trustworthy OS to handle disks from the target system. Your specific type of research activities (e.g. automated testing, software development, etc.) may benefit much more from the live interrogation approach, but so-called ‘dead disk’ forensics is a fundamental investigative skill worth practicing.

In a physical-world scenario, a forensic investigator may (after many steps I’m omitting for brevity) acquire a copy of a machine’s hard drive by mounting it on his or her forensic workstation or imaging device. However, we’re working with VMs here, and we control the host OS, so physical acquisition is unnecessary. VMware products implement a VM’s virtual hard disk(s) as one or more VMDK files, and working with them is much faster than imaging physical HDDs.

However, dead-disk forensics means the machine has to be powered off. Even though we’re working with virtual, software-defined machines, the target VM must be shut down to successfully work with its virtual disk(s). One cannot mount VMDKs that are actively in use by a running or suspended VMware VM. Additionally, virtual hard disks cannot be added to linked clones, so that leaves you with three options for acquisition:

  1. Use forensic software to handle the VM’s VMDK(s) directly from the host OS’s filesystem.

  2. Boot the target VM from a live CD/ISO file, and safely mount its existing disk(s).

  3. Add the existing virtual disk(s) (copy, share, or take away) to a standalone forensics VM.


I have firsthand experience with the last two methods, and I found them to be fairly straightforward to do. Another item of note is that, by default, VMware Fusion splits new virtual disks into multiple files. If you plan to try the first option, or you care strongly about ease of copying or cryptographic hashing, then you might want to disable this for the virtual disks you expect to analyze. Either way, once you mount the disk(s) you can search the filesystem(s) and inspect artifacts of interest such as web browsing history, password hashes, and deleted files. VMware Add Device program window


Use Case: Traffic Analysis


Another crucial capability for security researchers is the ability to monitor and capture network traffic for all sorts of analysis like discovering malware infections, uncovering protocol flaws, observing sensitive data in transit, anomaly detection, reverse engineering command and control mechanisms, diagnosing bugs in offensive tools, and more. Fortunately, VMware Fusion ships with a binary program, vmnet-sniffer, that writes a vmnet’s packets into a file.

Before you run this tool you should understand the basics of how VMware Fusion implements virtual networks on top of macOS. Every VM with a virtual network adapter can be configured to join the network in one of three predefined ways that correspond to a virtual adapter in macOS. Here are the options and their associated vmnets:



With VMware Fusion Pro you can also define custom virtual networks in the global VMware Fusion preferences. Doing so instantiates a new vmnet (e.g. vmnet5), and if you opt to “Connect the host Mac to this Network” then packet capturing with the vmnet-sniffer program is possible. Once you know the target VM’s vmnet (or you’ve configured it to your liking) you can follow these steps to analyze its network traffic:

Step 1. On your host macOS create a folder and empty file to store the packets VMware captures.

$ 
sudo mkdir /opt/PCAP/ && sudo chown $(echo $USER) /opt/PCAP/ && touch /opt/PCAP/vmnet8.pcap

You can optionally generate a symbolic link to the vmnet-sniffer executable to conserve keystrokes.

$ 
ln -s /Applications/VMware\ Fusion.app/Contents/Library/vmnet-sniffer /opt/PCAP/vmnet-sniffer


Step 2. Set up a forensic analysis VM that has read access to the PCAP folder. VMware Shared Folders program window Step 3. Run vmnet-sniffer specifying the target vmnet and destination PCAP file.

$ 
sudo /opt/PCAP/vmnet-sniffer -w /opt/PCAP/vmnet8.pcap vmnet8 &> /dev/null


Step 4. Open the file in the forensic analysis VM with Wireshark or similar.

Wireshark program window showing captured packet data

Know that you will see traffic for all VMs on the specified vmnet. Also, this is a somewhat crude, ‘good enough’ method to get basic network traffic visibility, but a more elegant, real-time monitoring solution (e.g. via named pipes) is probably preferable if heavy use is expected.


Use Case: Memory Forensics


Security research can also require one to look for artifacts in volatile memory. Fortunately, memory acquisition for virtual machines is less complicated than the traditional forensic processes predicated on access to a physical machine’s hardware combined with specialized forensic devices (e.g. sampling via DMA) or software-based memory sampling tools that execute within the running OS and depend entirely on a lack of prior OS tampering.

VMware Fusion saves virtual machine memory as files on the host OS filesystem with the following extensions:


These go by different names online (e.g. paging file, saved state file, etc.), but we only have to look at files with the .vmem extension. Here’s a simple way to analyze this file, using a common memory forensics tool like the Volatility framework, to search for memory-resident artifacts and understand the state of the OS:

Step 1. Give your forensics VM read-only access to the target VM’s folder via a shared folder.

By default, VMware Fusion saves any VM you create to a folder named “Virtual Machines” within your user’s home directory, but if you followed the prior steps in the Planning & Organizing section of this post, then the folder containing your VM(s) will be different. Browsing your host OS filesystem with Finder shows you VMs as files with a .vmwarevm file extension, but they’re really just folders that contain all of the internal files that make up the VM including the memory file we’ll be analyzing.

Finder program window showing internal VMware VM files If you expect to perform lots of memory forensics on more than one VM, to save yourself from repetitive work, you could give your forensics VM read-only access to the folder which contains all of your VMs instead of just one.


Step 2. Stage your target VM for analysis and take a snapshot.


Step 3. Find the newly-generated memory file (e.g. by naming convention).

VMware Fusion saves a new memory file when you take a snapshot or suspend a VM. For a snapshot, the memory file name is the VM name followed by a hyphen character, the word Snapshot, one or more digits, and the .vmem extension. Snapshot memory files are numbered sequentially, and the most recently modified file in the VM’s folder with the above naming scheme will represent the latest snapshot taken. You can find it like this:

$ 
ls -t {FOLDER} | egrep "\-Snapshot[0-9]+.vmem$" | head -n 1

It can take a minute or two for a snapshot’s memory file to completely save, and attempting to read the file before it’s ready will causes errors. You’ll know that the snapshotting process is complete when the VM’s folder no longer contains a file ending in the .vmem.lck file extension.

Alternatively, you can suspend the target VM instead of snapshotting. In this case, the memory file’s name will consist of the VM’s name, a hyphen, eight (lowercase) hexidecimal characters, and the .vmem extension. Here’s a regex to find it:

$ 
ls {FOLDER} | egrep "\-[0-9a-f]{8}.vmem$"

I’d like to thank Jonathon Poling for his feedback and contributions to my understanding here. Jonathon points out that suspending a VM may alter its memory contents such as by killing running processes, and so working with snapshots is preferred. Thanks Jonathon!


Step 4. Run Volatility supplying the memory file, matching OS profile, and desired plugin command.

$ 
python vol.py -f {FILE} --profile={STR} {CMD}

Ubuntu terminal window with Volatility output


Backup & Automation


To maintain productivity, and to avoid the annoyances of boring, routine maintenance tasks, security researchers should ensure that repeatability is a priority in the design on their daily driver workstations and VM libraries. For starters, having a backup solution in place before disaster strikes is a no-brainer. Since VMware Fusion is the topic here it’s relevant to consider backup options for macOS. Fortunately, macOS ships with its own incremental backup utility, Time Machine, and setting it up with an external HDD takes 30 seconds. With your VMs safely copied to a separate storage device, in the event you need to restore a VM or set up a new Mac, recovery is quick and repeatable.

Time Machine program window Not only should infosec practitioners automate backup procedures, but many boilerplate setup and configuration tasks our research depends on can be automated as well. Basically, any command line snippets in this post could be thought of as building blocks in your own scripts or tools. Also, VMware Fusion ships with many more binaries, such as vmrun, vmnet-cli, vmss2core, and others that could be very practical to integrate too. Programs like Packer and Vagrant work with VMware Fusion as well, and they can handle quite a lot of heavy lifting for you.

Regardless of how far you take it, automating VM lifecycle management tasks can save you valuable time, take error-prone CLI procedures off your plate, and allow you to focus more on actually doing meaningful research.


Evolution


This post described how VMware Fusion can be used to work with operating systems via virtual machines to improve one’s personal security and perform a few rudimentary, OS-level research and analysis activities. There are many other types of security research one can study that don’t require or benefit from maintaining a library of VMs. Still, working with the major operating systems one is likely to encounter in a professional context (e.g. as a SOC analyst, penetration tester, malware analyst, security engineer, etc.) and comprehending their internals is a great skill to have.

Infosec is a fast-changing field, and eventually this content will reach its expiration date, but I like to think the advice on workstation setup and the specific forensic methods presented in this post serve as a good starting point for someone looking to develop his or her capabilities today. I’m completely open to critical feedback and alternative approaches, so feel free to tell me your thoughts and whether or not this post helped you. Thanks for reading!




Mike Iacovacci
Mike Iacovacci is an information security professional specializing in endpoint security, intrusion investigation, and security research. His efforts have prevented serious security incidents and continue to disrupt cybercrime operations and sophisticated threat actors.

FOR FUTURE POSTS AND UPDATES FOLLOW @mikeiacovacci