Category Archives: Psychology

Know Your Bias – Anchoring

In the first part of this series I told a personal story to illustrated the components and effects of bias in a non-technical setting. In each post following I’ll examine a specific type of bias, how it manifests in a non-technical example, and provide real-world examples where I’ve seen this bias negatively affect a security practitioner. In this post, I’ll discuss anchoring.

Anchoring occurs when a person tends to rely too heavily on a single piece of information when making decisions, most often based on information received early in the decision-making process.

Anchoring Outside of Security

Think about the average price of a Ford car. Is it higher or lower than 80,000? This number is clearly too high, so you’d say lower. Now let’s flip this a bit. I want you to think about the average price of a Ford car again. Is it higher or lower than 10,000? Once again, the obvious answer is that it’s higher.

These questions may seem obvious and innocent, but here’s the thing. Let’s say that I ask one group of people the first question, and a separate group of people the second question. After that, I ask both groups to name what they think the average price of a Ford car is. The result is that the first group presented with the 80,000 number would pick a price much higher than the second group presented with the 10,000 number. This has been tested in multiple studies with several variants of this scenario [1].

In this scenario, people are subconsciously fixating on the number they are presented and it is subtly influencing their short term mindset. You might be able to think of a few cases in sales where this is used to influence consumers. Sticking with our car theme, if you’ve been on a car lot you know that every car has a price that is typically higher than what you pay. By pricing cars higher initially, consumers anchor to that price. Therefore, when you negotiate a couple thousand dollars off, it feels like you’re getting a great deal! In truth, you’re paying what the dealership expected, you just perceive the deal because of the anchoring effect.

They key takeaway from these examples is that anchoring to a specific piece of information is not inherently bad. However, making judgements in relation to an anchored data point where too much weight is applied can negatively effect your realistic perception of a scenario. In the first example, this led you to believe the average price of a car is higher or lower than it really is. In the second example, this led you to believe you were getting a better deal than you truly were.

 

Anchoring in Security

Anchoring happens based on the premise that mindsets are quick to form but resistant to change. We quickly process data to make an initial assessment, but our ability to hone that assessment is generally weighed in relation to the initial assessment. Can you think of various points in a security practitioner’s day where there is an opportunity for an initial perception into a problem scenario? This is where we can find opportunities for anchoring to have occurred.

List of Alerts

Let’s consider a scenario where you have a large number of alerts in a queue that you have to work through. This is a common scenario for many analysts, and if you work in a SOC that isn’t open 24×7 then you probably walk in each morning to something similar. Consider this list of the top 5 alerts in a SOC over a twelve hour period:

  • 41 ET CURRENT_EVENTS Neutrino Exploit Kit Redirector To Landing Page
  • 14  ET CURRENT_EVENTS Evil Redirector Leading to EK Apr 27 2016
  • 9 ET TROJAN Generic gate[.].php GET with minimal headers
  • 2 ET TROJAN Generic -POST To gate.php w/Extended ASCII Characters (Likely Zeus Derivative)
  • 2 ET INFO SUSPICIOUS Zeus Java request to UNI.ME Domain

Which alert should be examined first? I polled this question and found that a significant number of inexperienced analysts chose the one at the top of the list. When asked why, most said because of the frequency alone. By making this choice, the analyst assumes that each of these alerts are weighted equally. By occurring more times, the rule at the top of the list represents a greater risk. Is this a good judgement?

In reality, the assumption that each rule should be weighted the same is unfounded. There are a couple of ways to evaluate this list.

Using a threat-centric approach, not only does each rule represent a unique threat that should be considered uniquely, some of these alerts gain more context in the presence of others. For example, the two unique Zeus signatures alerting together could pose some greater significance. In this case, the Neutrino alert might represent a greater significance if it was paired with an alert representing the download of an exploit or communication with another Neutrino EK related page. Merely hitting a redirect to a landing page doesn’t indicate a successful infection, and is a fairly common event.

You could also evaluate this list with a risk-centric approach, but more information is required. Primarily, you would be concerned with the affected hosts for each alert. If you know where your sensitive devices are on the network, you would evaluate the alerts based on which ones are more likely to impact business operations.

This example illustrates the how casual decisions can come with implied assumptions. Those assumptions can be unintentional, but they can still lead you down the wrong path. In this case, the analyst might spend a lot of time pursuing alerts that aren’t very sensitive while delaying the investigation of something that represents a greater risk to the business. This happens because it is easy to look at a statistic like this and anchor to a singular facet of the stat without fully considering the implied assumptions. Statistics are useful for summarizing data, but they can hide important context that will keep you from making uninformed decisions that are the result of an anchoring bias.

 

Visualizations without Appropriate Context

As another example, understand that numbers and visual representations of them have a strong ability to influence an investigation.  Consider a chart like the one in the figure below.

This is a treemap visualization used to show the relative volume of communication based on network ports for a single system. The larger the block, the more communication occurred to that port. Looking at this chart, what is the role of the system whose network communication is represented here? Many analysts I polled decided it was a web server because of the large amount of port 443 and 80 traffic. These ports are commonly used by web servers to receive requests.

This is where we enter the danger zone. An analyst isn’t making a mistake by looking at this and considering that the represented host might be a web server. The mistake occurs when the analyst fully accepts this system as a web server and proceeds in an investigation under that assumption. Given this information alone, do you know for sure this is a web server? Absolutely not.

First, I never specified whether this treemap exclusively represents inbound traffic, and it’s every bit as likely that it represents outbound communication that could just be normal web browsing. Beyond that, this chart only represents a finite time period and might not truly represent reality. Lastly, just because a system is receiving web requests doesn’t necessarily mean its primary role is that of a web server. It might simply have a web interface for managing some other service that is its primary role.

The only way to truly ascertain whether this system is a web server is to probe it to see if there is a listening service on a web server port or to retrieve a list of processes to see if a web server application is running.

There is nothing wrong with using charts like this to help characterize hosts in an investigation.  This treemap isn’t an inherently bad visualization and it can be quite useful in the right context. However, it can lead to investigations that are influenced by unnecessarily anchored data points. Once again, we have an input that leads to an assumption.  This is where it’s important to verify assumptions when possible, and at minimum identify your assumptions in your reporting. If the investigation you are working on does end up utilizing a finding based on this chart and the assumption that it represents a web server, call that out specifically so that it can be weighed appropriately.

 

Diminishing Anchoring Bias

The stories above illustrate common places anchoring can enter the equation during your investigations. Throughout the next week, try to look for places in your daily routine where you form an initial perception and there is an opportunity for anchoring bias to creep in. I think you’ll be surprised at how many you come across.

Here are a few ways you can recognize when anchoring bias might be affecting you or your peers and strategies for diminishing its effects:

Consider what data represents, and not just it’s face value. Most data in security investigations represents something else that it is abstracted from. An IP address is abstracted from a physical host, a username is abstracted from a physical user, a file hash is abstracted from an actual file, and so on.

Limit the value of summary data. A summary is meant to be just that, the critical information you need to quickly triage data to determine its priority or make a quick (but accurate) judgement of the underlying events disposition. If you carry forward input from summary data into a deeper investigation, make sure you fully identify and verify your assumptions.

Don’t let your first impression be your only impression. Rarely is the initial insertion point into an investigation the most important evidence you’ll collect. Allow the strength of conclusions to be based on your evidence collected throughout, not just what you gathered at the onset. This is a hard thing to overcome, as your mind wants to anchor to your first impression, but you have to try and overcome that and try to examine cases holistically.

An alert is not an answer, it’s merely a question. Your job is to prove or disprove the alert, and until you’ve done one of those things the alert is not representative of a certainty. Start looking at alerts as the impetus for asking questions that will drive your investigation.

If you’re interested in learning more about how to help diminish the effects of bias in an investigation, take a look at my Investigation Theory course where I’ve dedicated an entire module to it. This class is only taught periodically, and registration is limited.

[1] Strack, F., & Mussweiler, T. (1997). Explaining the enigmatic anchoring effect: Mechanisms of selective accessibility. Journal of personality and social psychology73(3), 437.

 

 

Know your Bias – Foundations

In this blog series I’m going to dissect cognitive biases and how they relate to information security. Bias is prevalent in any form of investigation, whether you’re threat hunting, reversing malware, responding to an incident, attributing network attacks, or reviewing an IDS alert. In each post, I’ll describe a specific type of bias and how it manifests in various information security specialties. But first, this post will explain some fundamentals about what bias is and why it can negatively influence investigations.

 

What is Bias?

Investigating security threats is a process that occurs within the confines of your mind and centers around bridging the gap between your limited perception and the reality of what has actually occurred. To investigate something is to embrace that perception and reality aren’t the same thing, but you would like for them to be. The mental process that occurs while trying to bridge that perception-reality gap is complex and depends almost entirely on your mindset.

A mindset is how you see and approach the world and is shaped by your genetics and your collective world experience. Everyone you’ve ever met, everything you’ve ever done, and every sense you’ve perceived molds your mindset. At a high level, a mindset is neither a good or bad thing, but it can lead to positive or negative results. This is where bias comes in.

Bias is a predisposition towards a certain way of thinking, and it can be the difference in a successful or failed investigation. In some ways, bias is good when it allows us to learn from our previous mistakes and create mental shortcuts to solving problems. In other ways its bad, and can lead us to waste time pursuing bad leads or jump to conclusions without adequate evidence. Let’s consider a non-technical example first.

 

The (very) Personal Effects of Bias

On a night like any other, I laid my head down on my pillow at about 11 PM. However, I was unexpectedly awoken around 2 AM with wrenching stomach pain unlike anything I’d ever felt before. I tossed and turned for an hour or so without relief, and finally woke my wife up. A medical professional herself, she realized this might not be a typical stomach ache and suggested we head to the emergency room.

About an hour later I was in the ER being seen by the doctor on shift that night. Based on the location of the pain, he indicated the issue was likely gal bladder related, and that I probably had one or more gall stones causing my discomfort. I was administered some pain medication and setup with an appointment with a primary care physician the next day for further evaluation.

The primary care physician also believed that a gal bladder was the likely cause of the previous night’s discomfort, so she scheduled me for an ultrasound the same day. I walked over to the ultrasound lab where they spent about twenty minutes trying to get good images of the area. Shortly thereafter, the ultrasound technician came in and looked at the image and shared his thoughts. He had identified my gal bladder and concluded that it was full of gal stones, which had caused my stomach pain. My next stop was a referral to a surgeon, where my appointment took no more than ten minutes as he quickly recommended I switch to a low fat diet for a few weeks until he could perform surgery to remove the malfunctioning organ.

Fast forward a couple of weeks later and I’m waking up from an early morning cholecystectomy operation. The doctor is there as soon as I wake up, which strikes me as odd even in my groggy state.

“Hey Chris, everything went fine, but…”

Let me take this opportunity to tell you that you never want to hear “but…” seconds after waking up from surgery.

“But, we couldn’t remove your gal bladder. It turns you don’t have one. It’s a really rare thing and less than .1% of the population is born without one, but you’re one of them.”

At first I didn’t believe him. As a matter of fact, I was convinced that my wife had put him up to it and they were messing with me while I wasn’t completely with it. It wasn’t until I got home that same day and woke up again several hours later that I grasped what had happened. I really wasn’t born with a gal bladder, and the surgery had been completely unnecessary.

 

Figure 1: This is a picture of where my gal bladder should be

 

Dissecting the Situation

Let’s dissect what happened here.

  1. ER Doctor: Believed I was having gal bladder issues based on previous patients presenting with similar symptoms. Could not confirm this in the ER, so referred me to a primary care physician.
  2. Primary Care Doctor: Also believed the gal bladder issue was likely, but couldn’t confirm this in her office, so she referred me to a radiologist.
  3. Radiologist: Reviewed my scans to attempt to confirm the diagnosis of the previous two doctors. Found what appeared to be confirming evidence and concluded my gal bladder was malfunctioning.
  4. Surgeon: Agreed with the conclusion of the radiologist (without looking at the source data himself) and proceeded to recommend surgery, which I went through.

So where and why did things go wrong?

For the most part, the first two doctors were doing everything within their power to diagnose me. The appropriate steps of a differential diagnosis instruct physicians to identify the most likely affliction and perform the test that can rule that out. In this case, that’s the ultrasound that was performed by the radiologist, which is where things started to go awry.

The first two doctors presented a case for a specific diagnosis, and the radiologist was predisposed towards confirming this bias before even looking at the scans. It’s a classic case of finding something weird when you go looking for it, because everything looks a little weird when you want to find something. In truth, the radiologist wasn’t confident in his finding, but he let the weight of the other two doctor’s opinions bear down on him such that he was actually seeking to confirm their diagnosis more than trying to independently come to an accurate conclusion. This is an example of confirmation bias, which his perhaps the most common type of bias encountered. It also represents characteristics of belief bias and anchoring.

The issue here was compounded when I met the surgeon. Rather than critically assessing the collective findings of all the professionals that were involved to this point, he assumed a positive diagnosis and merely glanced at the ultrasound results in passing. All of the same biases are repeated here again.

In my case bias led to an incorrect conclusion, but understand that even if my gal bladder had been the issue and the surgery went as expected, the bias was still there. In many (and sometimes most) cases, bias might exist and you will still reach the correct conclusion. That doesn’t mean that the same bias won’t cause you to stumble later on.

 

Consequences of Bias

In my story, you see that bias resulted in some fairly serious consequences. Surgery is a serious matter, and I could have suffered some type of complication while on the table, or a post op infection that could have resulted in extreme sickness or death.

In any field, bias influences conclusions and those conclusions have consequences when they result in action being taken. In my case, it was a few missed days of work and a somewhat painful recovery. In security investigations, bad decisions can result in wasted time pursuing leads or missing something important. The latter could have drastic effects if you work for a government, military, ICS environment, or any other industry where lives depend on system integrity and uptime. Even if normal commercial environments, bad decisions resulting from the influence of bias can lead to millions of lost dollars.

Let’s examine one other effect of this scenario. I’m a lot less likely to trust the conclusions of doctors now, and I’m a lot less likely to agree to surgery without a plethora of supporting evidence indicating the need for it. These things in themselves are a form of bias. This is because bias breeds additional bias. We saw this with the relationship between the radiologist and the surgeon as well.

The effects of bias are highly situational. It’s best not to think of bias as something that dramatically changes your entire outlook on a situation. Think of bias like a pair of tinted glass that subtly influences how you perceive certain scenarios. When the right bias meets the right set of circumstances, things can go bad quickly.

 

Countering Bias

Bias is insanely hard to detect because of how it develops in either an individual or group setting.

In an individual setting, bias is inherent to your own decision making. Since humans inherently stink at detecting their own bias, it is unlikely you will become aware of it unless your analysis is reviewed by someone else who points it out. Even then, bias usually exists in small amounts and is unlikely to be noticed unless it meets a certain threshold that is noticeable by the reviewer.

In a group setting, things get complicated because bias enters from multiple people in small amounts. This creates a snowball effect in which group bias exists outside the context of any specific individual. Therefore, if hand offs occur in a linear manner such that each person only interacts one degree downstream or upstream, it is only possible to detect overwhelming bias at the upstream levels. Unfortunately, in real life these upstream levels are usually where the people are less capable of detecting the bias because they have lesser subject matter expertise in the field. In security, think managers and executives trying to catch this bias instead of analysts.

Let me be clear – you can’t fully eliminate bias in an investigation or in most other walks of your life. The way humans are programmed and the way our mindsets work prohibit that, and honestly, you wouldn’t want to eliminate all bias because it can be useful at times too. However, you can minimize the effects of negative bias in a couple of ways.

Evidence-Based Conclusions

A conclusion without supporting evidence is simply a hypothesis or a belief. In both medicine and information security, belief isn’t good enough without data to support it. This is challenging in many environments because visibility isn’t always in all the places we need it, and retention might not be long enough to gather the information you need when an event occurred far enough in the past. Use these instances to drive your collection strategy and justify appropriate budget for making sound conclusions.

In my case, two doctors made hypotheses without evidence. Another doctor gathered weak evidence and made a bad decisions, and another doctor confirmed that bad decision because the evidence was presented as a certainty when it actually wasn’t. A review of the support facts would have led the surgeon to catch the error before deciding to operate.

Peer Review

By definition, you aren’t aggressively aware of your own bias. Your mindset works the way it works and that is “normal” to you, so your ability to spot biased decisions when you make them is limited. After all, nobody comes to a conclusion they know to be incorrect. As Sheldon from the Big Bang Theory would say, “If I was wrong, don’t you think I’d know it?”

This is where peers come in. The surgeon might have caught the radiologist’s error if he had thoroughly reviewed the ultrasound results. Furthermore, if there was a peer review system in place in the radiology department, another person might have caught the error before it even got that far. Other people are going to be better at identifying your biases than you are, so that is an opportunity to embrace your peers and pull them in to review your conclusions.

Knowledge of Bias

What little hope you have of identifying your own biases doesn’t manifest during the decision-making process, but instead during the review of your final products and conclusions. When you write reports or make recommendations, be sure to identify the assumptions you’ve made. Then, you can review your conclusions as weighed against supporting evidence and assumptions to look for places bias might creep in. This requires a knowledge of common types of bias and how they manifest, which is precisely the purpose of this series of blog posts.

Most physicians are trained to understand and recognize certain types of bias, but that simply failed in my case until after the major mistakes had been made.

 

Conclusion

The investigation process provides an opportunity for bias to affect conclusions, decisions, and outcomes.  In this post I described a non-technical example of how bias can creep in while attempting to bridge the gap from perception to reality. In the rest of this series I’ll focus on specific types of bias and provide technical and non-technical examples of the bias in action, along with strategies for recognizing the bias in yourself and others.

As it turns out my stomach aches eventually stopped on their own. I never spoke to the radiologist again, but I did speak to the surgeon at length and he readily admitted his mistake and felt horrible for it. Some people might be furious at this outcome, but in truth, I empathized with his plight. Complex investigations, be it medical or technical, present opportunity for bias and we all fall victim to it from time to time. We’re all only human.

 

 

If you’re interested in learning more about how to help diminish the effects of bias in an investigation, take a look at my Investigation Theory course where I’ve dedicated an entire module to it. This class is only taught periodically, and registration is limited.

The Effects of Opening Move Selection on Investigation Speed

What follows is a shortened version of a longer paper that will be released at a later time. You can also learn more about this research by watching my recent Security Onion Conference 2016 video where I discuss these results and other similar experiments.

Background

The core construct of computer network defense is the investigation. It’s here where human analysts receive anomalous signals and pursue evidence that might prove the existence of an information system breach. While little formalized methodology related to the investigation process exists, research in this area is beginning to emerge.

Existing research in human thought and knowledge acquisition is applicable to information security and computer network defense. Daniel Kahneman provided modern research in support of a dual-process theory of thinking that defines two separate processes governing human thought. The first process, called intuitive or system 1 thinking, is automatic and usually kicks in without directly acknowledging it. You don’t have to think about brushing your teeth or starting your car, you just do it. The second process, called reflective or system 2 thinking, is deliberate and requires attentive focus. You have to think about math problems and your next move when playing checkers. Both of these systems are in play during the investigation process.

In an investigation, analysts use intuitive thought when pursuing data related to an alert. This is most often the case when the analyst makes their opening move. The analyst’s opening move is the first data query they execute after receiving an alert. By virtue of being the first move, the analyst doesn’t apply a lot of explicit reflective thought on the issue and they simply jump to the place they believe will provide the quickest and most definitive answer. It’s assumed that in these cases, the analyst’s first move is probably the data source they perceive as being the most valuable.

The goal of the present research is to determine which common data source analysts were more likely to use as their opening move, and to assess the impact of that first move on the speed of the investigation.

Methods

The foundation of this research was a purpose built investigation simulator. The investigation simulator was built to recreate the investigation environment in a tool agnostic manner, such that individual scenarios could be loaded for a sample population and the variables could be tightly controlled.

A pool of security analysts was selected based on their employment history. Every analysts selected was currently or recently in a role were they were responsible for investigating security alerts to determine if a security compromise had occurred. Demographic information was collected, and analysts were placed into three skills groups based on their qualifications and level of experience: novice, intermediate, or expert.

 

Group A – Exploit Kit Infection

The primary experiment group was asked to connect to the investigation simulator remotely and work through the investigation scenario provided to arrive at a conclusion whether an infection or compromise had successful occurred.

The scenario presented the user with a Suricata IDS alert indicating that an internal host visited an exploit kit landing page.

 

openingmove-figure1

Figure 1: The Suricata IDS alert that initiated the simulation

The following data sources were provided for investigation purposes:

Data Source Query Format Output Style
Full packet capture (PCAP) Search by IP or port TCPDump
Network flow/session data Search by IP or port SiLK IPFIX
Host file system Search by filename File path location
Windows Logs Option: User authentication/process create logs Windows event log text
Windows Registry Option: Autoruns/System restore/application executions (MUI cache) Registry keys and values
Antivirus Logs Search by IP Generic AV log text
Memory Option: Running process list/Shim cache Volatility
Open Source Intelligence Option: IP or domain reputation/file hash reputation/Google search Text output similar to popular intelligence providers

Table 1: Data source provided for Group A

Subjects were instructed that they should work towards a final disposition of true positive (an infection occurred), or false positive (no infection occurred). Whenever they had enough information to reach a conclusion, they were to indicate their final disposition in the tool, at which point the simulation exited.

The simulator logged every query the analysts made during this experiment, along with a timestamp and the start and end time. This produced a timeline of the analysts enter investigation, which was used to evaluate the research questions.

 

Group B – PCAP Data Replaced with Bro Data

Based on results achieved with group A, a second non-overlapping sample group of analysts were selected to participate in another experiment. Since group indicated a preference for higher context PCAP data, the second scenario removed the PCAP data option and replaced it with Bro data, another high context data source that is more structured and organized. The complete list of data sources provided for this group were:

Data Source Query Format Output Style
Bro Search by IP Bro
Network flow/session data Search by IP or port SiLK IPFIX
Host file system Search by filename File path location
Windows Logs Option: User authentication/process create logs Windows event log text
Windows Registry Option: Autoruns/System restore/application executions (MUI cache) Registry keys and values
Antivirus Logs Search by IP Generic AV log text
Memory Option: Running process list/Shim cache Volatility
Open Source Intelligence Option: IP or domain reputation/file hash reputation/Google search Text output similar to popular intelligence providers

Table 2: Data source provided for Group B

All experiment procedures and investigation logging measures remained in place, consistent with group A.

Group C – Survey Group

A third semi-overlapping group was selected at random to collect self-reported statistics to assess what opening move analysts self reported they would be more likely to make given a generic investigation scenario.

Using a combination of manually polling analysts and collecting responses from Twitter polling, analysts were asked the following question:

In a normal investigation scenario, what data source would you look at first?

The multiple-choice options presented were:

  1. PCAP
  2. Flow
  3. Open Source Intelligence
  4. Other

Results

The first item evaluated was the distribution of opening moves. Simply put, what data source did analysts look at first?

In Group A, an 72% of analysts chose PCAP as their first move, 16% chose flow data, and the remaining 12% chose OSINT. The observed numbers differ significantly from the numbers analysts reported during information polling. In the Group C polling, 49% of analysts reported PCAP would be their first move, 28% chose flow data, and 23% chose OSINT.

openingmove-chart1

Chart 1: Opening move selection observed for Group A

The mean time to disposition (MTTD) metric was calculated for each first move group by determining the difference between start and end investigation time for each analysts and averaging the results of all analysts within the group together. Analyst’s who chose PCAP had a MTTD of 16 minutes, those who chose flow had a MTTD of 10 minutes, and those who chose OSINT had a MTTD of 9 minutes.

openingmove-chart2

Chart 2: Time to disposition for Group A

 

In Group B where PCAP data was replaced with Bro data, 46% of analysts chose Bro data as their first move, 29% chose OSINT, and 25% chose flow.

openingmove-chart3

Chart 3: Comparison of group A and B opening moves

Analysts who chose Bro had a MTTD of 10 minutes, while those who chose flow and OSINT and MTTDs of 10 minutes and 11 minutes, respectively.

openingmove-chart4

Chart 4: Comparison of group A and B average time to close

Discussion

While not entirely conclusive, the data gained from this research does provide several suggestions. First, given an overwhelming 72% of people chose to begin their investigation with PCAP data, it’s clear that analysts prefer a higher context data source when its available, even if other lower context data sources available. In these simulations there were multiple ways to come to the correct conclusion, and PCAP data did not have to be examined at all to reach it.

The data also suggests that an opening move to a high context but relatively unorganized data source can negatively affect the speed an analyst reaches an appropriate conclusion. The MTTD for analysts whose opening move was PCAP in Group A was significantly higher than those who started with lower context data sources flow and OSINT. This is likely because PCAP data contains extraneous data that isn’t beneficial to the investigator, and it takes much longer to visually parse and interpret. Examining the results of the group B experiment further supports this finding. PCAP was replaced with Bro log data, which generally contains most of the same useful information that PCAP provides, but organizes it in a much more intuitive way that makes it easier to sift through. Analysts who chose Bro data for their opening move had a MTTD that was much lower than PCAP and comparable to flow and OSINT data sources.

The comparison between observed and reported opening moves highlights another finding that analysts often don’t understand their own tendencies during an investigation. There was a significant difference between the number of people who reported they would choose to investigate an anomaly with PCAP, and those who actually did. Opening move selection is somewhat situational however, so the present study did not introduce enough unique simulations to truly validate the statistics supporting that finding.

Possible limitations for this study mostly center on a limited number of trials, as only one simulation (albeit modified for one group) was used. More trials would serve to strengthen the findings. In addition, there is some selection bias towards analysts who are more specialized in network forensics than host forensics. This likely accounts for no first moves being to host-based data. Additionally, in the simulations conducted here access to all data sources took an equal amount of time. In a real world scenario, some data sources take longer to access. However, since PCAP and other higher context data sources are usually larger in size on disk, the added time to retrieve this data would only strengthen these findings that PCAP data negatively affects investigation speed.

Conclusion

Overall, this research provides insight into the value of better organized higher context data sources. While PCAP data contains an immense level of context, it is also unorganized, hard to filter and sift through compared to other data types, and has extraneous data not useful for furthering the investigation. To improve investigation efficiency, it may be better to make opening moves that start with lower context data sources so that a smaller net can be cast when it comes time to query higher context sources. Furthermore, when more organized higher context data sources are available, they should be used.

While the present research isn’t fully conclusive due to its sample size and a limited number of simulation trials, it does provide unique insight into the investigation process. The methods and strategy used here should be applicable for additional research to further confirm the things this data suggests.

 

Interested in learning more about the investigation process, choosing the right data sources to look at, and becoming a better analyst? Sign up here to be the first to hear about my new analyst training course being released later this year. Mailing list subscribers will get the first opportunity to sign up for the exclusive web-based course, and space is limited. Proceeds from this course will go to benefit charity.

Accelerating Experience with Investigation Heuristics

ifthenelseWhy is someone who has been investigating security incidents for ten years so much better than someone who has only been doing it for a year?

That’s a simple question, and the simple answer is experience. As an analyst learns the fundamentals, develops a larger tool chest, and encounters more diverse scenarios they will naturally become better at their craft.

That’s straightforward, but consider these alternate scenarios. There are analysts who have been involved with security investigations for three years who are better than analysts who have been involved for ten years. Why is that? Furthermore, if there are two analysts with the same amount of experience, why would one analyst be better at investigating things than the other?

While we like to measure experience in units of time that is rarely an effective way to relate why an analyst is good at their job. Experience is related to expertise, but they don’t always directly correlate.

Today, I want to focus two elements particularly relevant to how expertise can be quantified between novice and expert analysts. These are rule-based reasoning and investigation heuristics.

Rule-Based Reasoning

I recently conducted a series of case studies where I brought in several security analysts of varying experience levels and asked them to describe a case they had worked. Through a technique known as the stimulated recall interview, I had them describe the process from beginning to end, focusing on why they took certain actions as the investigation progressed.

Once I collected a reasonable sample of these case studies, I reviewed each of them and performed a key phrase mapping exercise. I identified a list of categories based on a dual process theory model and mapped relevant statements made by the analyst to those categories. I was left with a distribution of how many responses existed in each category that I could divide based on various analyst demographics, like experience.

One category where there was a significant difference between the number of responses given be novice and expert analysts was rule-based reasoning. The expert analyst had nearly three times as many instances where rule-based reasoning was responsible for their actions.

Rule-based reasoning can be best thought of as an if-then-else statement. It’s a way that many believe humans store, retrieve, and manipulate knowledge, often leading to an action. Of course, as with several matters of the mind there are other theories too.

Regardless, it should come as no surprise that computers were designed to work using if-then-else statements, because computers are in some ways mankind’s attempt to recreate itself. It represents some of our most fundamental understanding of how we think and process information, and it can be demonstrated in all walks of life. Investigations are no different.

Consider the domain google.com. When you see that domain appear in an alert you immediately assume the alert is a false positive. This is because you’ve applied a rule like this:

  • If: Domain belongs to a well-known public company
  • Then: It’s probably not hosting malicious content
  • Else: It might have been victim of a strategic web compromise

Now consider the domain jr2jk2ndiskd.oje2kje.ru. When you see this domain in an alert you immediately assume its evil. This could be the result of a rule like this:

  • If: Domain appears to be mostly random alphanumeric characters
  • Then: It might be generated by a domain name generation algorithm and/or owned by an attacker
  • Else: It could be a coincidence, and should be documented in case I run across it again

These are simple rules that can be articulated easily. Of course, not all rules are that cut and dry.

Even if you don’t realize it, any time you review evidence in an investigation you’re evaluating a set of rules to make decisions. Some of these are very deliberate (reflective thinking) and some of them are very automatic (intuitive thinking). These two types of thinking and how they relate define dual process theory.

With that said, a rule-based system is a simplification of something that is insanely more complex. We aren’t just dealing with a linear approach to information processing, but more likely with the activation of millions of neurons in a semantic network or some other form of connectionist model. That goes well beyond the scope of this article and some levels of the current state of human understanding. Although a simplification, a rule-based system is a reasonable one for how humans might take inputs, compare them against existing knowledge (see: top-down processing), and produce outputs.

Accelerating Experience

Given this perspective on rule-based reasoning, it should come as no surprise that expert analysts have a much larger library of rules than novice analysts. These rules can be gained through experience, but as I stated earlier, experience doesn’t correlate perfectly with expertise. Gaining expertise is more about optimizing the analyst’s ability to build mental rules than arbitrarily waiting for the passage of time.

Certainly experience provides more of an opportunity to learn things, but if we can identify those things then there is little reason they can’t be taught in a more direct manner. Practically, this means that it’s possible to accelerate the rate at which an analyst gains experience by subjecting them to an environment that is more suitable for the development of rules.

That’s one reason we get analysts with the same amount of experience but varying levels of expertise (ignoring natural disposition towards the work). One environment might support the development of rules better than another. Experience is accelerated in these environments.

Investigation Heuristics

A simple way to help analysts develop a bigger library of rules is to write them down. The infosec industry has done a poor job of this, as it’s not something you’ll find publicly available. Some organizations have invested in the creation of investigation playbooks, which are a step in the right direction.

To document investigation-focused mental rules, the same if-then-else framework discussed earlier can be applied. If it ain’t broke, don’t fix it. These are more appropriately called heuristics, which are rules used to make decisions, solve problems, or draw conclusions. Better said, heuristics are mental shortcuts to finding answers to questions.

A more formalized heuristic format looks like this:

Heuristic Name
Input: $evidence_type
If:
     $evidence is/has/contains $observation
Then:
     $conclusion
Else:
     $conclusion

 

Each heuristic is given a name for quick reference. It also includes an input evidence type, because in general any investigative conclusion is drawn from some type of observation or analysis on evidence. In many cases, a heuristic could be relevant for input of multiple types of evidence, or may require multiple types.

From there, the if-then-else statement makes up the meat of the heuristic. Similar to normal if-then-else statements, these scenarios can be made infinitely more complex. Of course, the simpler they can be made the better. Humans are processing these, so they don’t have to be perfect or follow all the same guidelines as though we’d expect a computer to be able to interpret them. Here are a few examples.

Domain Fast Flux Heuristic

Input: Domain

  • If: Domain resolves to a large number of IP addresses with diverse registration ownership or geography in a very short period
  • Then: It is likely that the domain is attacker owned and exhibiting fast-flux characteristics.
  • Else: The domain could be owned by a hosting company.
  • Else: The observation could be a coincidence.

 

File Type Mismatch Heuristic

Input: File

  • If: A file received in an e-mail is identified as a specific type based on its extension, but static analysis identifies a different file type.
  • Then: It is probable that the file is malicious in nature.
  • Else: The observation could be a coincidence.

 

Isolated POST Heuristic 

Input: IP, URL

  • If: An external IP sends an HTTP POST to one of your web servers, but doesn’t send any HTTP GET requests during the same period.
  • Then: There is a possibility that the internal host has become infected with a web shell, and the communication represents malicious traffic.
  • Else: This could be normal behavior for the system.

 

These heuristics all share the fact that they probably aren’t strong enough indicators on their own to warrant detection alerts; at least, not as scale grows beyond the small business. They do make useful investigation heuristics given the appropriate input in another investigation, whether alert-driven or human-driven (as in hunting).

This is a simplified example of a structured heuristic, but there is room to add a lot of interesting metadata to this format. For example, adding reference points to specific techniques used to retrieve evidence. Another example would be adding confidence ratings to the conclusions. This is a great place to make use of words of estimative probability so analysts can approach the heuristic with the appropriate weight and scrutiny.

Ultimately, the format doesn’t matter too much as long as this fits into the investigative workflow seamlessly. If you are embracing the investigation method, this should fit well with the question-hypothesis-answer format. These heuristics serve the role of helping develop questions and hypotheses to existing questions. They can also be used to drive initial observations when the investigation takes the form of hunting.

As a Teaching Tool

In an ideal world, the industry rallies around a format for investigation heuristics that can be explained in both a narrative and programmatic form, a standard is developed, and large common bodies of knowledge could exist that teach people how to investigate things.

In reality, the information security industry isn’t great at standards, so it’s probably a bit of a pipe dream; but it’s okay to have goals. In the interim, just maintaining a simple wiki with these types of investigation shortcuts can provide a tremendous benefit to analysts in your environment attempting to gain expertise. Even in environments where you might be a one-man-army network administrator and security analysts, having the reference available and reviewing it within the context of an active investigation is a helpful. It’s a worthwhile up front time investment.

They goal of this article isn’t to give you a format for creating and storing investigation heuristics. Instead, it’s to introduce rule-based reasoning and how the familiar construct of the if-then-else statement can be used to represent investigation shortcuts. It’s up to you to find the best way you can capture and represent this information for your own development, and the nurturing of analysts on your team.

 

 

How Analysts Approach Investigations

A  challenge facing information security is our inability to effectively train new analysts. The majority of security knowledge is tacit. We have plenty of practitioners who are good at catching bad guys, but most of them can’t articulate how they do it. I believe that overcoming this issue requires a focus on fundamental thought processes underlying security investigations, which is the foundation of my doctoral research.

Every major thought-based profession has a core construct through which everything is framed. For doctors, it’s the patient case. From this stems the diagnostic process, testing frameworks, and treatment plans. For lawyers, it’s the legal case. From this stems the discovery exercise, the trial, and sentencing. These core constructs are defined as an entities whose whole is greater than the sum of their parts. Each one is a story all its own.

In information security, our core construct is the investigation case. Everything we do is based on determining if malicious activity has happened, and to what extent. I don’t think many would argue this point, but surprisingly, there is very little formal writing out there about the investigation process itself. Many texts gloss over it and merely consider in the sum of its parts, a basic container for related evidence.

I propose that the investigation is so much more.

The Investigation Method

The investigation is at the heart of information security. It is a living, beating thing through which all of our actions are motivated and framed. It is our lens. To understand the investigation you must understand how humans think.

  1. Perception is not reality. What we perceive as reality and what actually exists are two separate things separated by our ability to interpret sensory input and using higher order reasoning. The process of getting from an initial perception to an accurate depiction of reality is the basis for learning and cognition.
  2. Learning comes from questioning. Straight from the womb, humans learn by questioning their environment, themselves, and their limits. By asking questions and employing various techniques to find answers we learn to move, walk, talk, and think. These techniques range from simple experimentation to complex reasoning, and can be motivated by primal needs like food and water, or higher order needs like achievement or respect.
  3. Our biases are always present. There are countless barriers that limit our ability to get from perception to reality. The most dangerous of these is our own mindset and the biases that are inherent to it. Humans are opinionated, and the same questions that drive us toward the pursuit of reality also drive opinions. When those opinions are educated and conscious they are hypotheses, and when none of those conditions are met they are guesses, and more subject to limiting bias.

If you consider this knowledge of human psychology, it begins to paint a picture of an investigation. Instead of trying to create a framework that dictates how investigations should be done, I wanted to take an approach the uncovers how you approach investigations as a form of learning. After all, that’s basically what an investigation is. It’s all about bridging the gap between perception and reality by learning facts. This yields the following definition and method.

“An investigation is the systematic inquiry and examination of evidence and observations in an effort to gain an accurate perception of whether an incident has occurred, and to what extent.”

The Investigation Process

If this looks familiar to you, that’s because it’s not too different from the scientific method. In a similar manner, the scientific manner wasn’t thought up as some way that scientific discovery should be done; it is an identification how most scientific discovery is done based on how humans learn. Even if scientists don’t intentionally set out to use the scientific method, their subconscious mind is doing it. The scientific method is responsible for the vast majority of scientific discovery. The investigation method is similarly responsible for the discovery of network intruders.

The investigation method contains five parts. I’ll briefly cover them here, although each one is worthy of its own article which will come later.

Observation

Every investigation begins with some observation that arouses suspicion. This is often machine generated in the form of an IDS alert, but could also be human driven in the form of an observation made while hunting. It doesn’t have to be an internal observation, and may come from a third-party notification. The tactics of the investigation are often shaped by the source of the initial observation, but the general process remains the same.

  • An observation is usually based on some form of initial evidence.
  • An observation can come from anywhere, but should be supportable. Even hunches or gut feelings are supportable when framed appropriately.
  • The first goal of the investigation is usually to validate or invalidate the initial observation as the premise of the investigation. If that observation isn’t valid, the investigation may not need to progress.

Question

An investigation consists of a series of questions for which the analyst must seek answers. Based on the initial observation, the overarching questions will likely be some version of “Did a breach occur?” or “Is this malicious?” To answer those questions, more questions must be asked. Answers to one question will usually generate more questions. At any given point, an analyst should be able to articulate what question they’re trying to answer.

  • The ability to define good questions increases with experience because expert analysts have a larger pool of heuristics (rules) to draw from.
  • Most questions are centered around uncovering relationships, because ultimately it’s the relationships between devices and users that define an attack or breach.
  • Newer analysts will frequently begin answer seeking activities without clearly identifying the question they are attempting to answer. This can lead to wasted effort, but usually diminishes with experience.

Hypothesis

You’re usually already slanted towards a specific answer from the moment you define your question, even if you don’t realize it. Your opinion forms based on your mindset, and is shaped by the entirety of your experience, both personal and professional. This is also where bias lives in the investigation process. The ability to articulate a hypothesis is an ideal way to expose bias so that your assumptions can be challenged if necessary. It also provides a clear path to additional questions that can validate or invalidate your hypothesis. Collectively, this leads to better, stronger conclusions.

  • Most hypothesis generation is passive and occurs subconsciously. A trick to making this an active process is to form an “I believe” statement for a hypothesis in response to each question. I believe ______ because _______.
  • Ideally a hypothesis is an educated guess. If you cannot complete the last half of the because statement, your assumptions may be from a place of bias, inexperience, or an inability to articulate well.
  • Every question should provide opportunity for a hypothesis, even if it’s a null hypothesis stating that a scenario isn’t probable.

Answer

The area of investigation most analysts are familiar with is answer seeking. It involves familiar tasks like retrieving, manipulating, and reviewing data. Any time you analytically review data or perform research it’s because you’re seeking an answer to your questions, usually to prove or disprove a hypothesis. Traditionally, newer analysts usually learn answer seeking before anything else which explains why the learning curve is so steep. They are trying to find answers for questions they don’t fully understand.

  • The goal of every answer isn’t to solve the investigation, it’s often to provide an opportunity for more questions. The answers you find will only be as good as the question they’re trying to resolve.
  • While it may seem logical to seek answers that prove a hypothesis, seeking to disprove a hypothesis is usually a much faster route to better questions.
  • Some questions won’t be answerable due to a lack of visibility or not enough data retention. Inability to answer a question is notable, because it might have impact on the investigation later. An unanswered question does not equal an invalid hypothesis.

Conclusion

The conclusion of an investigation is its terminal point. The investigation can terminate as a false positive alert, an acceptable risk, a simple malware infection, or a large breach requiring coordinated incident response. When a terminal disposition has been made, the investigation will contain a series of questions, hypotheses, and answers that uncover a (hopefully) accurate representation of events as they have occurred.

  • The strength of conclusions should always be accurately depicted by using estimative language. Certainties should be cited as such and backed up with evidence. Analytic opinions should be weighted based on their estimated certitude and available evidence.
  • If the steps that led you to a conclusion are considered carefully and documented well throughout the process, it should ease the burden of citing supporting information when documenting conclusions.

Framing an Investigation

Let’s look at example of what an investigation looks like through the lens of the investigation method. In this case, our fictional analyst has received an alert from an intrusion detection system.

Initial Observation: IDS Alert – User account was added to a domain admin group

This alert represents activity that might be legitimate, but could be malicious if it was unauthorized. The first question that generally follows an alert of this nature is whether it is malicious or normal activity.

Question 1: Does this alert represent malicious activity?

If the analyst were in a small organization they might be aware of any changes like this that should be occurring. Our analyst works in a very large enterprise, so it’s entirely possible that someone made this change for a legitimate reason without the analyst knowing. Because of this, the analyst believes its legitimate activity.

Hypothesis 1: I believe this is legitimate activity because this is something that happens frequently within the organization. 

To answer the initial question, the analyst must prove or disprove the hypothesis. To do this, more questions must be asked. There are a number of routes the analyst could go here, but one many analysts would pursue relates to follow-up actions taken by the user account.

Question 2: What actions did the user account take after being added to the admin group?

Based on the earlier hypothesis that this is normal behavior, it’s likely the hypothesis to Q2 will be similar.

Hypothesis 2: I believe the account participated in legitimate admin activity because it supports hypothesis 1. 

Seeking an answer to Q2 should be fairly easily with adequate visibility into your system and network logs. The analyst is able to search through logs fed into his SIEM and determine that the user account in question logged into a workstation, opened Outlook, and mounted several C-level executives mailboxes from the Exchange mail server.

Answer 2: The user account logged into a workstation, opened Outlook, and mounted several C-level executives mailboxes from the Exchange mail server.

The answer to Q2 appears to disprove our hypothesis 2, which in turn disproves hypothesis 1. The activity exhibited by the user account is definitely malicious, and answers our first question.

Answer 1: The actions taken by the user account after being added to the domain admin group are malicious in nature due to unauthorized access to multiple sensitive mailboxes.

At this point, the analyst is confident a breach has occurred, and the investigation can continue with that in mind. This should bring up more questions as the investigation evolves, including:

  • Was the user account an existing user account whose credentials were compromised?
  • Are there any indicators of compromise on the workstation normally used by the user who owns this account?
  • How did the potential attacker gain enough access to be able to promote the compromised account into an admin group?
  • How did the user account gain access to the workstation used to mount the Exchange mailboxes?
  • Is there any malware installed on the workstation the mailboxes were mounted from?
  • Were any other accounts accessed from the system belonging to the owner of the compromised account?

As you can see, what I’ve articulated here is only a fraction of what could be a much larger investigation. The key takeaway is that it provides a very structured, easy to follow timeline of the investigation and how it progressed. This makes it much easier to review the investigation process from beginning to end, and to use this investigation as a teaching tool for novice analysts.

InvestgationScenario

As a Universal Method

The investigation method is a universal construct within information security. While the industry often glamorizes unique subspecialties like hunting and malware analysis, they all fit within the same scope of activities. The method still applies.

For example, consider threat hunting. It follows the same process to bridge the gap from perception to reality. The only difference is that the initial observation is usually human-driven. Instead of receiving an IDS alert or an external notification, the analyst asks broad questions based on their library of experience-derived heuristics. The goal of this questioning is for the answers to generate more questions, or lead to the discovery of evidence that represents malicious activity.

This isn’t to say that subspecialties don’t require unique skill sets. They most certainly do. A hunter is usually someone more experienced because they have a larger library of investigative heuristics to work from, which allows them to be more effective at coming up with questions that can drive the discovery of interesting observations. A novice analyst wouldn’t have nearly as many heuristics to rely on, and their efforts would be less fruitful.

The characteristics of a good analyst will vary based on specialization, but the method is universal.

Why It Matters

The investigation method isn’t provided as a framework. The truth is that this is the method you likely already use to investigate security events, even if you aren’t aware of it. That awareness is key, because it gives practitioners a language to express their knowledge. From this comes more insightful analysis, more clearly identified methods that lead to conclusions, and an ability to teach novice analysts how investigations can be performed through the lens of an expert.

If you walk into a hundred SOCs you will find a hundred ways of documenting investigations. There is no standard, and worse yet, most end up adopting whatever format their tooling provides. What happens is that ticketing systems and wikis end up defining how analysts perform investigations. This is tragic.

If you walk into those same hundreds SOC’s, you’ll also typically only find one way of teaching people how investigations should be done — through on the job observation. While observation-based training is a key component of any training program, an education that is founded entirely on observation is sure to fail. I wouldn’t want a surgeon who skipped medical school and went straight to residency to be operating on me. Sure, they might be able to get the job done, but they’ll be missing the fundamentals that make them flexible and prepared for the inevitable unknown.

This is one significant reason why defenders are so badly outpaced by attackers in information security. Our profession hasn’t gone through its cognitive revolution where we seek to understand how we approach the investigation and it’s components. If we want to get there, understanding human thought and the methods that form the investigation are key. This article seeks to shed light in some of those areas, and certainly the articles to follow will as well.

I’d encourage you to consider the method shown here and think through it as you perform your investigations. What questions are you asking? How are your hypotheses swaying your analysis? How strong are your conclusions? How do you express how you approach investigations? These are all useful questions and are pivotal in your own understanding of the craft, as well as those who will come after you.