Category Archives: Psychology

The Cult of Passion

Would you perform your job if you weren’t paid? That’s the question people are often asked to measure their passion for a profession. But, it’s not that simple.

Go one step further and really consider that statement. It’s not asking you if you would work for nothing, it’s asking you if you would pay to work in your job. By choosing to work without compensation you are incurring a direct cost for equipment, commuting expenses, education, and more. You’re also incurring an indirect expense because the time spent working prevents you from earning a salary elsewhere.

So, would you pay to work in information security? You probably wouldn’t. Does that mean you aren’t passionate about infosec? I would wager that most practitioners are not. You would pay to garden, play soccer, barbecue, or play the guitar…but you wouldn’t take a financial loss to install patches, look at packets, and change firewall rules.

But, if that’s the case then why does our industry seem to revolve around passion? Nearly every blog post you read about hiring or job-seeking discusses the importance of passion and they often provide advice for how to demonstrate it. Some advice goes so far as to highlight passion as the most important characteristic you can exhibit. Infosec is described less of a job and more as a lifestyle. This sounds a lot less like job advice and more like recruitment for a cult.

In this post, I’m going to talk about passion, myths commonly associated with it, and how the cult of passion harms the practice of information security.

Passion as a Currency

Passion is commonly equated with extreme motivation surrounding a specific topic. In its simplest form passion manifests through hard work and time spent. These are both traits that are viewed admirably, especially in the US. Working from sunrise to sunset harkens back to memories of farmers earning an honest living while providing food for the masses, or to middle-class factory workers going the extra mile to provide for their families. These images are pervasive and are the backbone of society.

Of course, hard work isn’t truly a measure of passion. The farmer isn’t always passionate about farming. He’s passionate about providing a living for his family. The factory worker doesn’t love stamping car frames for 12 hours a day, but it enables the things he or she is truly passionate about.

In truth, passion isn’t reliably measurable either, because it can only be measured relative to others. In infosec hiring, an interviewer may only see someone else as passionate if they appear to exhibit passion in the same way as them and to a greater degree. Jim speaks at 12 security conferences a year, contributes to 5 open source projects, and works 16 hours a day. These are things he finds value in and how he would quantify his own passion. He is interviewing Terry, who only speaks at one or two conferences a year, contributes to one open source project, and works about 10 hours a day. Jim is likely to see Jerry as someone who isn’t very passionate. However, this is a purely relative viewpoint. It might not also consider things that Jerry does that Jim doesn’t value as a form of passion such as mentoring less experienced practitioners or doing tech-related community service.

When you attempt to evaluate people via traits that are difficult to objectively measure (like passion), you present an opportunity for undesirable results. This is something often seen with faith in religion. A false prophet commits to lead followers to the promised land if only they demonstrate appropriate faith. That faith might be prayer, tithing 10% of your income, tithing 100% of your income, or violently killing people of opposing faith. I highlight the wide range here because it shows the extremes that can arise when your currency isn’t objectively measurable.

In information security, we use passion as an unquantifiable currency to measure the potential success of someone in our field. A common piece of advice given to someone who wants to work in information security is that it isn’t simply enough for infosec to be your job. If you want to be successful in infosec, it must be the thing that gets you up in the morning. There must be more to this.

Do You Really Mean Passion?

Psychologically, passion is either harmonious or obsessive. Vallerand describes this better than I can:

Harmonious passion originates from an autonomous internalization of the activity into one’s identity while obsessive passion emanates from a controlled internalization and comes to control the person. Through the experience of positive emotions during activity engagement that takes place on a regular and repeated basis, it is posited that harmonious passion contributes to sustained psychological well-being while preventing the experience of negative affect, psychological conflict, and ill-being. Obsessive passion is not expected to produce such positive effects and may even facilitate negative affect, conflict with other life activities, and psychological ill-being.

What do people mean when they talk about passion in infosec? Rarely is it ever defined through any other mechanism but example. If you ask most to describe someone who is passionate about information security they’ll say that these people spend copious amounts of time outside of work on infosec projects, contribute to open source, go to a lot of security conferences, are actively involved in the security community, or have a blog.

Assuming you’ve found someone who does all of those things, can you guarantee that means they are passionate about infosec? How would you be able to differentiate them from someone who is passionate about being successful, or making money, or being recognized for being an expert? Finally, how do you differentiate harmonious and obsessive passion? That is a very challenging proposition.

Passion is very difficult to attribute to a source. In fact, most people aren’t good at identifying the things they are passionate about themselves. The vast majority of security practitioners are not passionate about information security itself. Instead, they’re passionate about problem-solving, being an agent of justice, being intelligent, being seen as intelligent, actually being intelligence, solving mysteries, making a lot of money, or simply providing for their families.

In most cases, I don’t think the trait people are looking for is actually passion. Instead, they’re looking for curiosity. Curiosity has a motivational component and is often described simply as “the desire to know.” It is rooted in our ability to recognize a gap between our knowledge on a topic and the amount of available knowledge out there. When we recognize that gap, we make a subconscious gamble about the risk/reward of pursuing the knowledge and eventually decide to try and close the gap or not. This is called information gap theory, and through this theory we can gain a better understanding of trait and applied curiosity that can improve our ability to teach and hire people.

Diverging from Cult Mentality

Passion has its place. I know some people who truly are passionate about the practice of security, and they are among the top practitioners in our field.  However, it is unwise to constantly compare yourselves to these people. I offer the following:

For information security practitioners…

Hard work matters, but you can work hard and not allow this industry to pull you into the cult of passion. Choose where and how you spend your time so that your work enriches your personal life, and enjoy a personal life that enriches your work. If you fall victim to the thought that information security must be your life, you will eventually burn out. You will suffer, and if there is anybody left around you, they will suffer too.

Here are professions of people who work 8-10 hours a day and go home and don’t think about work: doctors, lawyers, engineers, scientists, researchers. Do the top 5% of practitioners in these fields think about work all the time? Probably. But you also probably aren’t one of those people. Not everyone is extraordinary and that’s okay. There is this myth that we all must be the best. As Ricky Bobby famously said, “If you ain’t first, your last!”. But, by constantly trying to be the best it breeds things like imposter syndrome, self-doubt, and depression. In an industry where so many have substance abuse problems and we’ve lost far too many friends, these are feelings we should actively avoid promoting.

For hiring managers…

It isn’t just limiting to only hire people who make infosec their life, it’s exclusionary. You’re missing out on people with diversity of interests that will enrich your security program. You’re also preventing people who have more important personal life issues from finding gainful employment.

To pursue the knowledge that exists in the curiosity information gap I discussed earlier, a person should be aware the gap exists. Otherwise, they don’t know what they don’t know. This implies that a job candidate needs to know a little about a topic to be strongly motivated to pursue knowledge in it and sustain that pursuit. The last part is important. Sure, the journey of a thousand miles begins with a single step, but that first step is also usually the easiest. It’s quite a few miles in where we normally lose people. This is one reason why the notion of trying to hire TRULY entry level people based on passion in infosec is a fool’s errand. Someone with no experience in this field does not have a proper footing to be passionate about it. If they are passionate about infosec, then that passion can’t be trusted to be sustained. You’re hiring based on a mirage.

A key to maintaining interest is a constant stream of novel information. For a novice, most things within a field can be novel because the key is to building passion is exploration. To transition to expertise, an individual must find novelty in the nuance of specific topics. Someone who enjoys nuance is best set up to be an expert. Most people will never truly be world-class experts in something, but again, that’s perfectly fine.

For job seekers…

Much to my dismay, most people will never read this article, truly understand passion, and cultivate an ability to notice genuine curiosity. That means you have to play the game that is hiring. People will keep asking about passion, but reframe the question under your own terms. Tell them you see passion as a term used to describe curiosity and motivation. Try to identify what really motivates you and how your curiosity pushes to toward goals. Relate to people at a personal, human level. A lot of candidates talk about how they eat/breathe/sleep infosec. You don’t have to do that. Instead, talk about how you critically think about important problems and optimize your time so that you don’t have to be work 16 hour days to be successful. Hard work is important, but working smart is much more important, and is actually sustainable.

Conclusion

Along my pursuit to understand passion I’ve learned that it’s a highly contentious topic. People hate to have their passion questioned, and I’m sure this article will stoke that fire. I wonder why that is? I would wager that many quantify their own ability and maybe even their own self-worth in their subjective self-evaluation of their own passion. Once again, passion is a good thing and measuring yourself based on some degree of it is probably fine. It’s when we choose to measure others based on our subjective views of their passion that we get into trouble and create cult-like scenarios. We can do better.

My goal with this article was to share my understanding of passion, how it’s often misinterpreted, and how that can negatively affect our industry. Once of the most liberating moments of my life was when I figured out that I wasn’t passionate about information security, it was just infosec that allowed me to achieve other things I was passionate about. If others can relate then I hope they can feel the same liberation someday through a better understand of passion. If you are truly passionate about infosec itself, then that’s great too, we need you!

 

Further reading:

 

Know Your Bias – Availability Heuristic

This is part three in the Know your Bias series where I examine a specific type of bias, how it manifests in a non-technical example, and provide real-world examples where I’ve seen this bias negatively affect a security practitioner. You can view part one here, and part two here. In this post, I’ll discuss the availability heuristic.

The availability heuristic is a mental shortcut that relies on recalling the most recent or prevalent example that comes to mind when evaluating data to make a decision.

For the security practitioner, this type of bias is primarily an attack on your time more so than your accuracy. Let’s go through a few examples both inside and outside of security before discussion ways to mitigate the negative effects availability heuristic can have.

Availability Heuristic Outside of Security

Are you more likely to be killed working as a police officer or as a fisherman? Most people select police officer. However, statistics show that you are as much as 10x more likely to meet your end while working on a fishing boat [1]. People get this wrong because of the availability heuristic. Whenever a police officer is killed in the line of duty, it is often a major news event. Police officers are often killed in the pursuit of criminals and this is typically viewed as a heroic act, which means it becomes a human interest story and news outlets are more likely to cover it.

Try this yourself. Go to Google News and search for “officer killed”. You will almost certainly find multiple recent stories and multiple outlets covering the same story. Next, search for “fisherman killed”, and you’ll find a lot fewer results returned. When there are results, they are typically only covered by the locale the death happened in and not picked up by national outlets. The news disproportionately covers the death of police officers over fishermen. To be clear, I’m not questioning that practice at all. However, this does explain why most tend to think that the police work is more deadly than being a fisherman. We are more likely to trust the information we can recall more quickly, and by virtue of seeing more news stories about police deaths, the availability heuristic tricks us into thinking that police work is more deadly. I’d hypothesize that if we posed the same question to individuals who were regular viewers of the Discovery Channel show “The Deadliest Catch”, they might recognize the danger associated with commercial fishing and select the correct answer to the question.

One thing we know about human memory and recall is that it is efficient. We often go with the first piece of information that can be recalled from our memory. Not only does the availability of information drive our thoughts, it also shapes our behavior.  It’s why advertisers spend so much money to ensure that their product is the first thing we associate with specific inputs. When many Americans think of cheeseburgers they think of McDonalds. When you think of coffee you think of Starbucks. When you think of APT you think of Mandiant. These aren’t accidental associations — a lot of money has been spent to ensure those bonds were formed.

Availability Heuristic in Security

Availability is all about the things you observe the most and the things you observe most recently. Consider these scenarios that highlight examples of how availability can affect decisions in security practice.

Returning from a Security Conference

I recently attended a security conference where multiple presenters showed examples that included *.top domains that were involved with malicious activity. These sites were often hosting malware or being used to facilitate command and control channels with infected machines. One presenter even said that any time he saw a *.top domain, he assumed it was probably malicious.

I spoke with a colleague who had really latched on to that example. He started treating every *.top domain he found as inherently malicious and driving his investigations with that in mind. He even spent time actively searching out *.top domains as a function of threat hunting to proactively find evil. How do you think that turned out for him? Sure, he did find some evil. However, he also found out that the majority of *.top domains he encountered on his network were actually legitimate. It took him several weeks to realize that he had fallen victim to the availability heuristic. He put too much stock in the information he had received because of the recency and frequency of it. It wasn’t until he had gathered a lot of data that he was able to recognize that the assumption he was making wasn’t entirely correct. It wasn’t something that warranted this much of his time.

In another recent example, I saw a colleague purchase a lot of suspected APT owned domains with the expectation that sinkholing them would result in capturing a lot of interesting C2 traffic. He saw someone speak on this topic and thought that his success rate would be higher than it was because they speaker didn’t cover that topic in depth. My colleague had to purchase a LOT of domain names before he got any interesting data, and by that point, he had pretty much decided to give up after spending both a lot of time and money on the task.

It is very hard for someone giving a 30-minute talk to fully support every claim they make. It also isn’t easy to stop and cite additional sources in the middle of a verbal presentation. Because our industry isn’t strict about providing papers to support talks, we end up with a lot of opinions and not much fact. Those opinions get wheels and they may be taken much farther than the original presenter ever intended. This tricks people who are less metacognitively aware into accepting opinions as fact. 

Data Source Context Availability

If you work in a SOC, you have access to a variety of data sources. Some of those are much lower context like flow data or DNS logs, and some are much higher context like PCAP data or memory. In my research, I’ve found that analysts are much more likely to pursue high-context data sources when working an investigation, even when lower context data sources contain the information they need to answer their questions.

On one hand, you might say that this doesn’t matter because if you are arriving at the correct answer, why does it matter how you got there? Analytically speaking, we know that the path you take to an answer matters. It isn’t important just to be accurate in an investigation, you also need to be expedient. Security is an economic problem wherein the cost to defend a network needs to be low and the cost to attack it needs to be high. I’ve seen that users who start with higher context data sources when it is not entirely necessary often spend much more time in an investigation. By using higher context data sources when it isn’t necessary, it introduces an opportunity for distractions in the investigation process. The more opportunity for distracting information, the more opportunity that availability bias can creep in as a result of the new information being given too much priority in your decision making. That isn’t to say that all new information should be pushed aside, but you also have to carefully control what information you allow to hold your attention.

Structured Adversary Targeting

In the past five years, the security industry has become increasingly dominated by fear-based marketing. A few years ago it was the notion that sophisticated nation-state adversaries were going to compromise your network no matter who you were. These stories made national news and most security vendors began to shift their marketing towards guaranteeing protection against these threats.

The simple truth is that most businesses are unlikely to be targeted by nation-state level threat actors. But, because the news and vendor marketing have made this idea so prevalent, the availability of it has led an overwhelming number of people to believe that this could happen to them. When I go into small businesses to talk about security I generally want to talk about things like opportunistic attacks, drive-by malware, and ransomware. These are the things small businesses are mostly likely to be impacted by. However, most of these conversations now involve a discussion about structured threat actors because of the availability of that information. I don’t want to talk about these things, but people ask about them. While this helps vendors sell products, it takes some organizations’ eye off the things they should really be concerned about. I’m certain Billy Ray’s Bait Shop will never get attacked by the Chinese PLA, but a ransomware infection has the ability to destroy the entire business. In this example, the abundance of information associated with structured threat actors clouds perspective and takes time away from more important discussions. 

Diminishing Availability Heuristic

The stories above illustrate common places availability heuristic manifests in security. Above all else, the availability of information is most impactful to you in how you spend your time and where you focus your attention. Attention is a limited resource, as we can only focus on one or two things at a time. Consider where you place your attention and what is causing you to place it there.

Over the course of the next week, start thinking about the places you focus your attention and actively question why information led you to do that. Is that information based on fact or opinion? Are you putting too much or too little time into your effort? Is your decision making slanted in the wrong direction?

Here are a few ways you can recognize when availability heuristic might be affecting you or your peers and strategies for diminishing its effects:

Carefully consider the difference between fact and opinion. In security, most of the publicly available information you’ll find is a mix of opinions and facts and the distinction isn’t always so clear. Whenever you make a judgment or decision based on something elsewhere, spend a few minutes considering the source of the information and doing some manual research to see if you can validate it elsewhere.

Use patience as a shield. Since your attention is a limited resource, you should protect at accordingly. Just because new information has been introduced doesn’t mean it is worthy of shifting your attention to it. Pump the breaks on making quick decisions. Take a walk or sleep on new information before acting to see if something still matters as much tomorrow as it does today. Patience is a valuable tool in the fight to diminish the effects of many biases.

Practice question-driven investigating. A good investigator is able to clearly articulate the questions they are trying to answer, and only seeks out data that will provide those answers. If you go randomly searching through packet capture data, you’re going to see things that will distract you. By only seeking answers to questions you can articulate clearly, you’ll diminish the opportunity for availability bias to distract your attention.

Utilize a peer group for validation. By definition, we aren’t good at recognizing our own biases. When you are pursuing a new lead or deciding whether to focus your attention on a new task or goal, considering bouncing that idea off of a peer. They are likely to have had differing experiences than you, so their decision making could be less clouded by the recency or availability of information that is affecting you. A question to that group can be as simple as “Is ____ as big of a concern as I think it is?”

If you’re interested in learning more about how to help diminish the effects of bias in an investigation, take a look at my Investigation Theory course where I’ve dedicated an entire module to it. This class is only taught periodically, and registration is limited.

[1] http://www.huffingtonpost.com/blake-fleetwood/how-dangerous-is-police-w_b_6373798.html

Know Your Bias – Anchoring

In the first part of this series I told a personal story to illustrated the components and effects of bias in a non-technical setting. In each post following I’ll examine a specific type of bias, how it manifests in a non-technical example, and provide real-world examples where I’ve seen this bias negatively affect a security practitioner. In this post, I’ll discuss anchoring.

Anchoring occurs when a person tends to rely too heavily on a single piece of information when making decisions, most often based on information received early in the decision-making process.

Anchoring Outside of Security

Think about the average price of a Ford car. Is it higher or lower than 80,000? This number is clearly too high, so you’d say lower. Now let’s flip this a bit. I want you to think about the average price of a Ford car again. Is it higher or lower than 10,000? Once again, the obvious answer is that it’s higher.

These questions may seem obvious and innocent, but here’s the thing. Let’s say that I ask one group of people the first question, and a separate group of people the second question. After that, I ask both groups to name what they think the average price of a Ford car is. The result is that the first group presented with the 80,000 number would pick a price much higher than the second group presented with the 10,000 number. This has been tested in multiple studies with several variants of this scenario [1].

In this scenario, people are subconsciously fixating on the number they are presented and it is subtly influencing their short term mindset. You might be able to think of a few cases in sales where this is used to influence consumers. Sticking with our car theme, if you’ve been on a car lot you know that every car has a price that is typically higher than what you pay. By pricing cars higher initially, consumers anchor to that price. Therefore, when you negotiate a couple thousand dollars off, it feels like you’re getting a great deal! In truth, you’re paying what the dealership expected, you just perceive the deal because of the anchoring effect.

They key takeaway from these examples is that anchoring to a specific piece of information is not inherently bad. However, making judgements in relation to an anchored data point where too much weight is applied can negatively effect your realistic perception of a scenario. In the first example, this led you to believe the average price of a car is higher or lower than it really is. In the second example, this led you to believe you were getting a better deal than you truly were.

 

Anchoring in Security

Anchoring happens based on the premise that mindsets are quick to form but resistant to change. We quickly process data to make an initial assessment, but our ability to hone that assessment is generally weighed in relation to the initial assessment. Can you think of various points in a security practitioner’s day where there is an opportunity for an initial perception into a problem scenario? This is where we can find opportunities for anchoring to have occurred.

List of Alerts

Let’s consider a scenario where you have a large number of alerts in a queue that you have to work through. This is a common scenario for many analysts, and if you work in a SOC that isn’t open 24×7 then you probably walk in each morning to something similar. Consider this list of the top 5 alerts in a SOC over a twelve hour period:

  • 41 ET CURRENT_EVENTS Neutrino Exploit Kit Redirector To Landing Page
  • 14  ET CURRENT_EVENTS Evil Redirector Leading to EK Apr 27 2016
  • 9 ET TROJAN Generic gate[.].php GET with minimal headers
  • 2 ET TROJAN Generic -POST To gate.php w/Extended ASCII Characters (Likely Zeus Derivative)
  • 2 ET INFO SUSPICIOUS Zeus Java request to UNI.ME Domain

Which alert should be examined first? I polled this question and found that a significant number of inexperienced analysts chose the one at the top of the list. When asked why, most said because of the frequency alone. By making this choice, the analyst assumes that each of these alerts are weighted equally. By occurring more times, the rule at the top of the list represents a greater risk. Is this a good judgement?

In reality, the assumption that each rule should be weighted the same is unfounded. There are a couple of ways to evaluate this list.

Using a threat-centric approach, not only does each rule represent a unique threat that should be considered uniquely, some of these alerts gain more context in the presence of others. For example, the two unique Zeus signatures alerting together could pose some greater significance. In this case, the Neutrino alert might represent a greater significance if it was paired with an alert representing the download of an exploit or communication with another Neutrino EK related page. Merely hitting a redirect to a landing page doesn’t indicate a successful infection, and is a fairly common event.

You could also evaluate this list with a risk-centric approach, but more information is required. Primarily, you would be concerned with the affected hosts for each alert. If you know where your sensitive devices are on the network, you would evaluate the alerts based on which ones are more likely to impact business operations.

This example illustrates the how casual decisions can come with implied assumptions. Those assumptions can be unintentional, but they can still lead you down the wrong path. In this case, the analyst might spend a lot of time pursuing alerts that aren’t very sensitive while delaying the investigation of something that represents a greater risk to the business. This happens because it is easy to look at a statistic like this and anchor to a singular facet of the stat without fully considering the implied assumptions. Statistics are useful for summarizing data, but they can hide important context that will keep you from making uninformed decisions that are the result of an anchoring bias.

 

Visualizations without Appropriate Context

As another example, understand that numbers and visual representations of them have a strong ability to influence an investigation.  Consider a chart like the one in the figure below.

This is a treemap visualization used to show the relative volume of communication based on network ports for a single system. The larger the block, the more communication occurred to that port. Looking at this chart, what is the role of the system whose network communication is represented here? Many analysts I polled decided it was a web server because of the large amount of port 443 and 80 traffic. These ports are commonly used by web servers to receive requests.

This is where we enter the danger zone. An analyst isn’t making a mistake by looking at this and considering that the represented host might be a web server. The mistake occurs when the analyst fully accepts this system as a web server and proceeds in an investigation under that assumption. Given this information alone, do you know for sure this is a web server? Absolutely not.

First, I never specified whether this treemap exclusively represents inbound traffic, and it’s every bit as likely that it represents outbound communication that could just be normal web browsing. Beyond that, this chart only represents a finite time period and might not truly represent reality. Lastly, just because a system is receiving web requests doesn’t necessarily mean its primary role is that of a web server. It might simply have a web interface for managing some other service that is its primary role.

The only way to truly ascertain whether this system is a web server is to probe it to see if there is a listening service on a web server port or to retrieve a list of processes to see if a web server application is running.

There is nothing wrong with using charts like this to help characterize hosts in an investigation.  This treemap isn’t an inherently bad visualization and it can be quite useful in the right context. However, it can lead to investigations that are influenced by unnecessarily anchored data points. Once again, we have an input that leads to an assumption.  This is where it’s important to verify assumptions when possible, and at minimum identify your assumptions in your reporting. If the investigation you are working on does end up utilizing a finding based on this chart and the assumption that it represents a web server, call that out specifically so that it can be weighed appropriately.

 

Diminishing Anchoring Bias

The stories above illustrate common places anchoring can enter the equation during your investigations. Throughout the next week, try to look for places in your daily routine where you form an initial perception and there is an opportunity for anchoring bias to creep in. I think you’ll be surprised at how many you come across.

Here are a few ways you can recognize when anchoring bias might be affecting you or your peers and strategies for diminishing its effects:

Consider what data represents, and not just it’s face value. Most data in security investigations represents something else that it is abstracted from. An IP address is abstracted from a physical host, a username is abstracted from a physical user, a file hash is abstracted from an actual file, and so on.

Limit the value of summary data. A summary is meant to be just that, the critical information you need to quickly triage data to determine its priority or make a quick (but accurate) judgement of the underlying events disposition. If you carry forward input from summary data into a deeper investigation, make sure you fully identify and verify your assumptions.

Don’t let your first impression be your only impression. Rarely is the initial insertion point into an investigation the most important evidence you’ll collect. Allow the strength of conclusions to be based on your evidence collected throughout, not just what you gathered at the onset. This is a hard thing to overcome, as your mind wants to anchor to your first impression, but you have to try and overcome that and try to examine cases holistically.

An alert is not an answer, it’s merely a question. Your job is to prove or disprove the alert, and until you’ve done one of those things the alert is not representative of a certainty. Start looking at alerts as the impetus for asking questions that will drive your investigation.

If you’re interested in learning more about how to help diminish the effects of bias in an investigation, take a look at my Investigation Theory course where I’ve dedicated an entire module to it. This class is only taught periodically, and registration is limited.

[1] Strack, F., & Mussweiler, T. (1997). Explaining the enigmatic anchoring effect: Mechanisms of selective accessibility. Journal of personality and social psychology73(3), 437.

 

 

Know your Bias – Foundations

In this blog series I’m going to dissect cognitive biases and how they relate to information security. Bias is prevalent in any form of investigation, whether you’re threat hunting, reversing malware, responding to an incident, attributing network attacks, or reviewing an IDS alert. In each post, I’ll describe a specific type of bias and how it manifests in various information security specialties. But first, this post will explain some fundamentals about what bias is and why it can negatively influence investigations.

 

What is Bias?

Investigating security threats is a process that occurs within the confines of your mind and centers around bridging the gap between your limited perception and the reality of what has actually occurred. To investigate something is to embrace that perception and reality aren’t the same thing, but you would like for them to be. The mental process that occurs while trying to bridge that perception-reality gap is complex and depends almost entirely on your mindset.

A mindset is how you see and approach the world and is shaped by your genetics and your collective world experience. Everyone you’ve ever met, everything you’ve ever done, and every sense you’ve perceived molds your mindset. At a high level, a mindset is neither a good or bad thing, but it can lead to positive or negative results. This is where bias comes in.

Bias is a predisposition towards a certain way of thinking, and it can be the difference in a successful or failed investigation. In some ways, bias is good when it allows us to learn from our previous mistakes and create mental shortcuts to solving problems. In other ways its bad, and can lead us to waste time pursuing bad leads or jump to conclusions without adequate evidence. Let’s consider a non-technical example first.

 

The (very) Personal Effects of Bias

On a night like any other, I laid my head down on my pillow at about 11 PM. However, I was unexpectedly awoken around 2 AM with wrenching stomach pain unlike anything I’d ever felt before. I tossed and turned for an hour or so without relief, and finally woke my wife up. A medical professional herself, she realized this might not be a typical stomach ache and suggested we head to the emergency room.

About an hour later I was in the ER being seen by the doctor on shift that night. Based on the location of the pain, he indicated the issue was likely gal bladder related, and that I probably had one or more gall stones causing my discomfort. I was administered some pain medication and setup with an appointment with a primary care physician the next day for further evaluation.

The primary care physician also believed that a gal bladder was the likely cause of the previous night’s discomfort, so she scheduled me for an ultrasound the same day. I walked over to the ultrasound lab where they spent about twenty minutes trying to get good images of the area. Shortly thereafter, the ultrasound technician came in and looked at the image and shared his thoughts. He had identified my gal bladder and concluded that it was full of gal stones, which had caused my stomach pain. My next stop was a referral to a surgeon, where my appointment took no more than ten minutes as he quickly recommended I switch to a low fat diet for a few weeks until he could perform surgery to remove the malfunctioning organ.

Fast forward a couple of weeks later and I’m waking up from an early morning cholecystectomy operation. The doctor is there as soon as I wake up, which strikes me as odd even in my groggy state.

“Hey Chris, everything went fine, but…”

Let me take this opportunity to tell you that you never want to hear “but…” seconds after waking up from surgery.

“But, we couldn’t remove your gal bladder. It turns you don’t have one. It’s a really rare thing and less than .1% of the population is born without one, but you’re one of them.”

At first I didn’t believe him. As a matter of fact, I was convinced that my wife had put him up to it and they were messing with me while I wasn’t completely with it. It wasn’t until I got home that same day and woke up again several hours later that I grasped what had happened. I really wasn’t born with a gal bladder, and the surgery had been completely unnecessary.

 

Figure 1: This is a picture of where my gal bladder should be

 

Dissecting the Situation

Let’s dissect what happened here.

  1. ER Doctor: Believed I was having gal bladder issues based on previous patients presenting with similar symptoms. Could not confirm this in the ER, so referred me to a primary care physician.
  2. Primary Care Doctor: Also believed the gal bladder issue was likely, but couldn’t confirm this in her office, so she referred me to a radiologist.
  3. Radiologist: Reviewed my scans to attempt to confirm the diagnosis of the previous two doctors. Found what appeared to be confirming evidence and concluded my gal bladder was malfunctioning.
  4. Surgeon: Agreed with the conclusion of the radiologist (without looking at the source data himself) and proceeded to recommend surgery, which I went through.

So where and why did things go wrong?

For the most part, the first two doctors were doing everything within their power to diagnose me. The appropriate steps of a differential diagnosis instruct physicians to identify the most likely affliction and perform the test that can rule that out. In this case, that’s the ultrasound that was performed by the radiologist, which is where things started to go awry.

The first two doctors presented a case for a specific diagnosis, and the radiologist was predisposed towards confirming this bias before even looking at the scans. It’s a classic case of finding something weird when you go looking for it, because everything looks a little weird when you want to find something. In truth, the radiologist wasn’t confident in his finding, but he let the weight of the other two doctor’s opinions bear down on him such that he was actually seeking to confirm their diagnosis more than trying to independently come to an accurate conclusion. This is an example of confirmation bias, which his perhaps the most common type of bias encountered. It also represents characteristics of belief bias and anchoring.

The issue here was compounded when I met the surgeon. Rather than critically assessing the collective findings of all the professionals that were involved to this point, he assumed a positive diagnosis and merely glanced at the ultrasound results in passing. All of the same biases are repeated here again.

In my case bias led to an incorrect conclusion, but understand that even if my gal bladder had been the issue and the surgery went as expected, the bias was still there. In many (and sometimes most) cases, bias might exist and you will still reach the correct conclusion. That doesn’t mean that the same bias won’t cause you to stumble later on.

 

Consequences of Bias

In my story, you see that bias resulted in some fairly serious consequences. Surgery is a serious matter, and I could have suffered some type of complication while on the table, or a post op infection that could have resulted in extreme sickness or death.

In any field, bias influences conclusions and those conclusions have consequences when they result in action being taken. In my case, it was a few missed days of work and a somewhat painful recovery. In security investigations, bad decisions can result in wasted time pursuing leads or missing something important. The latter could have drastic effects if you work for a government, military, ICS environment, or any other industry where lives depend on system integrity and uptime. Even if normal commercial environments, bad decisions resulting from the influence of bias can lead to millions of lost dollars.

Let’s examine one other effect of this scenario. I’m a lot less likely to trust the conclusions of doctors now, and I’m a lot less likely to agree to surgery without a plethora of supporting evidence indicating the need for it. These things in themselves are a form of bias. This is because bias breeds additional bias. We saw this with the relationship between the radiologist and the surgeon as well.

The effects of bias are highly situational. It’s best not to think of bias as something that dramatically changes your entire outlook on a situation. Think of bias like a pair of tinted glass that subtly influences how you perceive certain scenarios. When the right bias meets the right set of circumstances, things can go bad quickly.

 

Countering Bias

Bias is insanely hard to detect because of how it develops in either an individual or group setting.

In an individual setting, bias is inherent to your own decision making. Since humans inherently stink at detecting their own bias, it is unlikely you will become aware of it unless your analysis is reviewed by someone else who points it out. Even then, bias usually exists in small amounts and is unlikely to be noticed unless it meets a certain threshold that is noticeable by the reviewer.

In a group setting, things get complicated because bias enters from multiple people in small amounts. This creates a snowball effect in which group bias exists outside the context of any specific individual. Therefore, if hand offs occur in a linear manner such that each person only interacts one degree downstream or upstream, it is only possible to detect overwhelming bias at the upstream levels. Unfortunately, in real life these upstream levels are usually where the people are less capable of detecting the bias because they have lesser subject matter expertise in the field. In security, think managers and executives trying to catch this bias instead of analysts.

Let me be clear – you can’t fully eliminate bias in an investigation or in most other walks of your life. The way humans are programmed and the way our mindsets work prohibit that, and honestly, you wouldn’t want to eliminate all bias because it can be useful at times too. However, you can minimize the effects of negative bias in a couple of ways.

Evidence-Based Conclusions

A conclusion without supporting evidence is simply a hypothesis or a belief. In both medicine and information security, belief isn’t good enough without data to support it. This is challenging in many environments because visibility isn’t always in all the places we need it, and retention might not be long enough to gather the information you need when an event occurred far enough in the past. Use these instances to drive your collection strategy and justify appropriate budget for making sound conclusions.

In my case, two doctors made hypotheses without evidence. Another doctor gathered weak evidence and made a bad decisions, and another doctor confirmed that bad decision because the evidence was presented as a certainty when it actually wasn’t. A review of the support facts would have led the surgeon to catch the error before deciding to operate.

Peer Review

By definition, you aren’t aggressively aware of your own bias. Your mindset works the way it works and that is “normal” to you, so your ability to spot biased decisions when you make them is limited. After all, nobody comes to a conclusion they know to be incorrect. As Sheldon from the Big Bang Theory would say, “If I was wrong, don’t you think I’d know it?”

This is where peers come in. The surgeon might have caught the radiologist’s error if he had thoroughly reviewed the ultrasound results. Furthermore, if there was a peer review system in place in the radiology department, another person might have caught the error before it even got that far. Other people are going to be better at identifying your biases than you are, so that is an opportunity to embrace your peers and pull them in to review your conclusions.

Knowledge of Bias

What little hope you have of identifying your own biases doesn’t manifest during the decision-making process, but instead during the review of your final products and conclusions. When you write reports or make recommendations, be sure to identify the assumptions you’ve made. Then, you can review your conclusions as weighed against supporting evidence and assumptions to look for places bias might creep in. This requires a knowledge of common types of bias and how they manifest, which is precisely the purpose of this series of blog posts.

Most physicians are trained to understand and recognize certain types of bias, but that simply failed in my case until after the major mistakes had been made.

 

Conclusion

The investigation process provides an opportunity for bias to affect conclusions, decisions, and outcomes.  In this post I described a non-technical example of how bias can creep in while attempting to bridge the gap from perception to reality. In the rest of this series I’ll focus on specific types of bias and provide technical and non-technical examples of the bias in action, along with strategies for recognizing the bias in yourself and others.

As it turns out my stomach aches eventually stopped on their own. I never spoke to the radiologist again, but I did speak to the surgeon at length and he readily admitted his mistake and felt horrible for it. Some people might be furious at this outcome, but in truth, I empathized with his plight. Complex investigations, be it medical or technical, present opportunity for bias and we all fall victim to it from time to time. We’re all only human.

 

 

If you’re interested in learning more about how to help diminish the effects of bias in an investigation, take a look at my Investigation Theory course where I’ve dedicated an entire module to it. This class is only taught periodically, and registration is limited.

The Effects of Opening Move Selection on Investigation Speed

What follows is a shortened version of a longer paper that will be released at a later time. You can also learn more about this research by watching my recent Security Onion Conference 2016 video where I discuss these results and other similar experiments.

Background

The core construct of computer network defense is the investigation. It’s here where human analysts receive anomalous signals and pursue evidence that might prove the existence of an information system breach. While little formalized methodology related to the investigation process exists, research in this area is beginning to emerge.

Existing research in human thought and knowledge acquisition is applicable to information security and computer network defense. Daniel Kahneman provided modern research in support of a dual-process theory of thinking that defines two separate processes governing human thought. The first process, called intuitive or system 1 thinking, is automatic and usually kicks in without directly acknowledging it. You don’t have to think about brushing your teeth or starting your car, you just do it. The second process, called reflective or system 2 thinking, is deliberate and requires attentive focus. You have to think about math problems and your next move when playing checkers. Both of these systems are in play during the investigation process.

In an investigation, analysts use intuitive thought when pursuing data related to an alert. This is most often the case when the analyst makes their opening move. The analyst’s opening move is the first data query they execute after receiving an alert. By virtue of being the first move, the analyst doesn’t apply a lot of explicit reflective thought on the issue and they simply jump to the place they believe will provide the quickest and most definitive answer. It’s assumed that in these cases, the analyst’s first move is probably the data source they perceive as being the most valuable.

The goal of the present research is to determine which common data source analysts were more likely to use as their opening move, and to assess the impact of that first move on the speed of the investigation.

Methods

The foundation of this research was a purpose built investigation simulator. The investigation simulator was built to recreate the investigation environment in a tool agnostic manner, such that individual scenarios could be loaded for a sample population and the variables could be tightly controlled.

A pool of security analysts was selected based on their employment history. Every analysts selected was currently or recently in a role were they were responsible for investigating security alerts to determine if a security compromise had occurred. Demographic information was collected, and analysts were placed into three skills groups based on their qualifications and level of experience: novice, intermediate, or expert.

 

Group A – Exploit Kit Infection

The primary experiment group was asked to connect to the investigation simulator remotely and work through the investigation scenario provided to arrive at a conclusion whether an infection or compromise had successful occurred.

The scenario presented the user with a Suricata IDS alert indicating that an internal host visited an exploit kit landing page.

 

openingmove-figure1

Figure 1: The Suricata IDS alert that initiated the simulation

The following data sources were provided for investigation purposes:

Data Source Query Format Output Style
Full packet capture (PCAP) Search by IP or port TCPDump
Network flow/session data Search by IP or port SiLK IPFIX
Host file system Search by filename File path location
Windows Logs Option: User authentication/process create logs Windows event log text
Windows Registry Option: Autoruns/System restore/application executions (MUI cache) Registry keys and values
Antivirus Logs Search by IP Generic AV log text
Memory Option: Running process list/Shim cache Volatility
Open Source Intelligence Option: IP or domain reputation/file hash reputation/Google search Text output similar to popular intelligence providers

Table 1: Data source provided for Group A

Subjects were instructed that they should work towards a final disposition of true positive (an infection occurred), or false positive (no infection occurred). Whenever they had enough information to reach a conclusion, they were to indicate their final disposition in the tool, at which point the simulation exited.

The simulator logged every query the analysts made during this experiment, along with a timestamp and the start and end time. This produced a timeline of the analysts enter investigation, which was used to evaluate the research questions.

 

Group B – PCAP Data Replaced with Bro Data

Based on results achieved with group A, a second non-overlapping sample group of analysts were selected to participate in another experiment. Since group indicated a preference for higher context PCAP data, the second scenario removed the PCAP data option and replaced it with Bro data, another high context data source that is more structured and organized. The complete list of data sources provided for this group were:

Data Source Query Format Output Style
Bro Search by IP Bro
Network flow/session data Search by IP or port SiLK IPFIX
Host file system Search by filename File path location
Windows Logs Option: User authentication/process create logs Windows event log text
Windows Registry Option: Autoruns/System restore/application executions (MUI cache) Registry keys and values
Antivirus Logs Search by IP Generic AV log text
Memory Option: Running process list/Shim cache Volatility
Open Source Intelligence Option: IP or domain reputation/file hash reputation/Google search Text output similar to popular intelligence providers

Table 2: Data source provided for Group B

All experiment procedures and investigation logging measures remained in place, consistent with group A.

Group C – Survey Group

A third semi-overlapping group was selected at random to collect self-reported statistics to assess what opening move analysts self reported they would be more likely to make given a generic investigation scenario.

Using a combination of manually polling analysts and collecting responses from Twitter polling, analysts were asked the following question:

In a normal investigation scenario, what data source would you look at first?

The multiple-choice options presented were:

  1. PCAP
  2. Flow
  3. Open Source Intelligence
  4. Other

Results

The first item evaluated was the distribution of opening moves. Simply put, what data source did analysts look at first?

In Group A, an 72% of analysts chose PCAP as their first move, 16% chose flow data, and the remaining 12% chose OSINT. The observed numbers differ significantly from the numbers analysts reported during information polling. In the Group C polling, 49% of analysts reported PCAP would be their first move, 28% chose flow data, and 23% chose OSINT.

openingmove-chart1

Chart 1: Opening move selection observed for Group A

The mean time to disposition (MTTD) metric was calculated for each first move group by determining the difference between start and end investigation time for each analysts and averaging the results of all analysts within the group together. Analyst’s who chose PCAP had a MTTD of 16 minutes, those who chose flow had a MTTD of 10 minutes, and those who chose OSINT had a MTTD of 9 minutes.

openingmove-chart2

Chart 2: Time to disposition for Group A

 

In Group B where PCAP data was replaced with Bro data, 46% of analysts chose Bro data as their first move, 29% chose OSINT, and 25% chose flow.

openingmove-chart3

Chart 3: Comparison of group A and B opening moves

Analysts who chose Bro had a MTTD of 10 minutes, while those who chose flow and OSINT and MTTDs of 10 minutes and 11 minutes, respectively.

openingmove-chart4

Chart 4: Comparison of group A and B average time to close

Discussion

While not entirely conclusive, the data gained from this research does provide several suggestions. First, given an overwhelming 72% of people chose to begin their investigation with PCAP data, it’s clear that analysts prefer a higher context data source when its available, even if other lower context data sources available. In these simulations there were multiple ways to come to the correct conclusion, and PCAP data did not have to be examined at all to reach it.

The data also suggests that an opening move to a high context but relatively unorganized data source can negatively affect the speed an analyst reaches an appropriate conclusion. The MTTD for analysts whose opening move was PCAP in Group A was significantly higher than those who started with lower context data sources flow and OSINT. This is likely because PCAP data contains extraneous data that isn’t beneficial to the investigator, and it takes much longer to visually parse and interpret. Examining the results of the group B experiment further supports this finding. PCAP was replaced with Bro log data, which generally contains most of the same useful information that PCAP provides, but organizes it in a much more intuitive way that makes it easier to sift through. Analysts who chose Bro data for their opening move had a MTTD that was much lower than PCAP and comparable to flow and OSINT data sources.

The comparison between observed and reported opening moves highlights another finding that analysts often don’t understand their own tendencies during an investigation. There was a significant difference between the number of people who reported they would choose to investigate an anomaly with PCAP, and those who actually did. Opening move selection is somewhat situational however, so the present study did not introduce enough unique simulations to truly validate the statistics supporting that finding.

Possible limitations for this study mostly center on a limited number of trials, as only one simulation (albeit modified for one group) was used. More trials would serve to strengthen the findings. In addition, there is some selection bias towards analysts who are more specialized in network forensics than host forensics. This likely accounts for no first moves being to host-based data. Additionally, in the simulations conducted here access to all data sources took an equal amount of time. In a real world scenario, some data sources take longer to access. However, since PCAP and other higher context data sources are usually larger in size on disk, the added time to retrieve this data would only strengthen these findings that PCAP data negatively affects investigation speed.

Conclusion

Overall, this research provides insight into the value of better organized higher context data sources. While PCAP data contains an immense level of context, it is also unorganized, hard to filter and sift through compared to other data types, and has extraneous data not useful for furthering the investigation. To improve investigation efficiency, it may be better to make opening moves that start with lower context data sources so that a smaller net can be cast when it comes time to query higher context sources. Furthermore, when more organized higher context data sources are available, they should be used.

While the present research isn’t fully conclusive due to its sample size and a limited number of simulation trials, it does provide unique insight into the investigation process. The methods and strategy used here should be applicable for additional research to further confirm the things this data suggests.

 

Interested in learning more about the investigation process, choosing the right data sources to look at, and becoming a better analyst? Sign up here to be the first to hear about my new analyst training course being released later this year. Mailing list subscribers will get the first opportunity to sign up for the exclusive web-based course, and space is limited. Proceeds from this course will go to benefit charity.