CONFIRMATION BIAS IN ACTION

Every Child is Perfect, Just Ask The Parents

One of the best stories to describe confirmation bias goes as follows. Johnny loved magicians. One day his parents took him to see a famous magician, Phil the Great. At the end of the show the parents told Phil how much their son loved magic. Phil then offered to show them a trick. Johnny eagerly accepted.

The magician proceeded to pull out a coin and move it back and forth between both hands then closed his fists and held out his hands. He asked Johnny to identify the hand containing the coin, which he did correctly. Now guessing correctly one time is not much of a feat, but this game was repeated many times and each time Johnny correctly guessed the hand containing the coin. While this was going on the magician made comments like, “You must have excellent vision to see which hand contains the coin,” and “You must be an expert on reading my facial expressions and that is how you know where the coin is.”

Eventually Johnny had correctly identified the hand with the coin fifty times in a row! His parents were amazed. They called the grandparents and told all of their friends about it on Facebook, Twitter, and other social media sites. When they finally thanked the magician and turned to leave, he shouted, “goodbye,” and waved with both hands. Each hand contained a coin.

It was the parents’ confirmation bias that lead them to believe what they wanted to believe, that Johnny was a savant, and distracted them from the truth, that the magician was indeed tricking them. Remain objective during an investigation. Don’t let what you or your boss want to be true keep you from seeing contrary evidence.

Reconstruction of Events

In my mind trying to reconstruct what happened is the most fun part of an investigation. The explosion in size of storage media might make the searching phase longer than it was in the past, but that only helps to make the reconstruction phase that much more enjoyable. It is very unlikely that you will find all the evidence you need for your event reconstruction in one place. It is much more common to get little pieces of evidence from multiple places which you put together into a larger picture. For example, a suspicious process in a process list stored in a memory image might lead you to look at files in a filesystem image which might lead you back to an open file list in the memory image which in turn points toward files in the filesystem image. Putting all of these bits together might allow you to determine when and by whom a rootkit was downloaded and when and by which user it was subsequently installed.

HIGH-LEVEL PROCESS

While not every Linux forensic investigation is part of an incident response, it will be the focus of this book. The justification for this is that the vast majority of Linux forensic investigations are conducted after a suspected breach. Additionally, many of the items discussed in this book will be relevant to other Linux investigations as well. The high level process for incident response is shown in Figure 1.2.

FIGURE 1.2

High-level Process for Linux Incident Response

As can be seen in Figure 1.2, it all begins with a call. Someone believes that a breach (or something else) has occurred and they have called you to investigate. Your next step is to determine whether or not there was a breach. A small amount of live analysis might be required in order to make this determination. If no breach occurred, you get to document what happened and add this to your knowledge base.

If there was an incident, you would normally start with live analysis before deciding whether or not dead analysis is justified. If you deem it necessary to perform the dead analysis you need to acquire some images and then actually perform the analysis. Whether or not you performed a dead analysis it isn’t over until the reports are written. All of these steps will be discussed in detail in future chapters.

BUILDING A TOOLKIT

In order to do Linux forensics effectively you might want to acquire a few tools. When it comes to software tools you are in luck as all of the Linux forensics tools are free (most are also open source). In addition to the notebook discussed previously, some hardware and software should be in every forensic investigator’s tool kit.

Hardware

You will likely want one or more external hard drives for making images (both RAM and hard disks). External hard drives are preferred as it is much easier to share with other investigators when they can just plug in a drive. USB 3.0 devices are the best as they are significantly faster than their USB 2.0 counterparts.

A write blocker is also helpful whenever an image is to be made of any media. Several hardware write blockers are available. Most of these are limited to one particular interface. If your budget affords only one hardware write blocker, I would recommend a SATA blocker as this is the most common interface in use at this time. Software write blockers are also a possibility. A simple software write blocker is presented later in this book.

Software

Software needs fall into a few categories: forensic tools, system binaries, and live Linux distributions. Ideally these tools are stored on USB 3.0 flash drives and perhaps a few DVDs if you anticipate encountering systems that cannot boot from a USB drive. Given how cheap USB flash drives are today, even investigators with modest budgets can be prepared for most situations.

There are a number of ways to install a set of forensics tools. The easiest method is to install a forensics oriented Linux distribution such as SIFT from SANS (http://digitalforensics.sans.org/community/downloads). Personally, I prefer to to run my favorite Linux and just install the tools rather than be stuck with someone else’s themes and sluggish live system performance. The following script will install all of the tools found in SIFT on most Debian or Ubuntu based systems (unlike the SANS install script that works only on specific versions of Ubuntu).

!/bin/bash # Simple little script to load DFIR tools into Ubuntu and Debian systems

by Dr. Phil Polstra @ppolstra # create repositories echo “deb http://ppa.launchpad.net/sift/stable/ubuntu trusty main” \

/etc/apt/sources.list.d/sift-ubuntu-stable-utopic.list echo “deb http://ppa.launchpad.net/tualatrix/ppa/ubuntu trusty main” \

/etc/apt/sources.list.d/tualatrix-ubuntu-ppa-utopic.list #list of packages pkglist=”aeskeyfind afflib-tools afterglow aircrack-ng arp-scan autopsy binplist

bitpim bitpim-lib bless blt build-essential bulk-extractor cabextract clamav cryptsetup dc3dd dconf-tools dumbpig e2fslibs-dev ent epic5 etherape exif extundelete f-spot fdupes flare flasm flex foremost g++ gcc gdb ghex gthumb graphviz hexedit htop hydra hydra-gtk ipython kdiff3 kpartx libafflib0 libafflib-dev libbde libbde-tools libesedb libesedb-tools libevt libevt-tools libevtx libevtx-tools libewf libewf-dev libewf-python libewf-tools libfuse-dev libfvde libfvde-tools liblightgrep libmsiecf libnet1 libolecf libparse-win32registry-perl libregf libregf-dev libregf-python libregf-tools libssl-dev libtext-csv-perl libvshadow libvshadow-dev libvshadow-python libvshadow-tools libxml2-dev maltegoce md5deep nbd-client netcat netpbm nfdump ngrep ntopng okular openjdk-6-jdk

p7zip-full phonon pv pyew python python-dev python-pip python-flowgrep python-nids python-ntdsxtract python-pefile python-plaso python-qt4 python-tk python-volatility pytsk3 rsakeyfind safecopy sleuthkit ssdeep ssldump stunnel4 tcl tcpflow tcpstat tcptrace tofrodos torsocks transmission unrar upx-ucl vbindiff virtuoso-minimal winbind wine wireshark xmount zenity regripper cmospwd

ophcrack ophcrack-cli bkhive samdump2 cryptcat outguess bcrypt ccrypt readpst ettercap-graphical driftnet tcpreplay tcpxtract tcptrack p0f netwox lft netsed socat knocker nikto nbtscan radare-gtk python-yara gzrt testdisk scalpel qemu qemu-utils gddrescue dcfldd vmfs-tools mantaray python-fuse samba open-iscsi curl git system-config-samba libpff

libpff-dev libpff-tools libpff-python xfsprogs gawk exfat-fuse exfat-utils xpdf feh pyew radare radare2 pev tcpick pdftk sslsniff dsniff rar xdot ubuntu-tweak vim” #actually install # first update apt-get update for pkg in ${pkglist} do if (dpkg —list | awk ‘{print $2}’ | egrep “^${pkg}$” 2>/dev/null) ; then echo “yeah ${pkg} already installed” else

try to install echo -n “Trying to install ${pkg}…”

if (apt-get -y install ${pkg} 2>/dev/null) ; then echo “+++Succeeded+++” else echo “–-FAILED–-”

fi fi done

Briefly, the above script works as described here. First, we run a particular shell (bash) using the special comment construct #!{command to run}. This is often called the “shebang” operator or “pound-bang” or “hash-bang,” Second, the lines with the echo statements add two repositories to our list of software sources. Technically, these repositories are intended to be used with Ubuntu 14.04, but they are likely to work with new versions of Ubuntu and/or Debian as well.

Third, a variable named pkglist is created which contains a list of the tools we wish to install. Fourth, we update our local application cache by issuing the command apt-get update. Finally, we iterate over our list of packages stored in pkglist and install them if they aren’t already installed. The test involves a string of commands, dpkg —list | awk ‘{print $2}’ | egrep “^${pkg}$” 2>/dev/null. The command dpkg —list lists all installed packages and this list is then passed to awk ‘{print $2}’ which causes the second word (the package name) to be printed; this is in turn passed to egrep “^${pkg}$” 2>/dev/null which checks to see if the package name exactly matches one that is installed (the ^ matches the start and $ matches the end). Any errors are sent to the null device because we only care if there were any results.

A set of known good system binaries should be installed to a flash drive in order to facilitate live response. At a minimum you will want the /bin, /sbin, and /lib directories (/lib32 and /lib64 for 64-bit systems) from a known good system. You may also want to grab the /usr directory or at least /usr/local/, /usr/bin, and /usr/sbin. Most Linux systems you are likely to encounter are running 64-bit versions of Linux; after all, 64-bit Linux has been available since before 64-bit processors were commercially available. It might still be worth having a 32-bit system on hand.

On occasion a live Linux system installed on a bootable USB drive could be useful. Either a distribution such as SIFT can be installed by itself on a drive or the live system can be installed on the first partition of a larger USB drive and the system binaries installed on a second partition. If you are using a USB drive with multiple partitions it is important to know that Windows systems will only see the first partition and then only if it is formated as FAT or NTFS. Partitions containing system binaries should be formatted as EXT2, EXT3, or EXT4 in order to mount them with correct permissions. Details of how to mount these system binaries will be provided in future chapters.

SUMMARY

In this chapter we have discussed all the preliminary items that should be taken care of before arriving on the scene after a suspected incident has occurred. We covered the hardware, software, and other tools that should be in your go bag. In the next chapter we will discuss the first job when you arrive, determining if there was an incident.

CHAPTER

2