This is part three in the Know your Bias series where I examine a specific type of bias, how it manifests in a non-technical example, and provide real-world examples where I’ve seen this bias negatively affect a security practitioner. You can view part one here, and part two here. In this post, I’ll discuss the availability heuristic.
The availability heuristic is a mental shortcut that relies on recalling the most recent or prevalent example that comes to mind when evaluating data to make a decision.
For the security practitioner, this type of bias is primarily an attack on your time more so than your accuracy. Let’s go through a few examples both inside and outside of security before discussion ways to mitigate the negative effects availability heuristic can have.
Availability Heuristic Outside of Security
Are you more likely to be killed working as a police officer or as a fisherman? Most people select police officer. However, statistics show that you are as much as 10x more likely to meet your end while working on a fishing boat [1]. People get this wrong because of the availability heuristic. Whenever a police officer is killed in the line of duty, it is often a major news event. Police officers are often killed in the pursuit of criminals and this is typically viewed as a heroic act, which means it becomes a human interest story and news outlets are more likely to cover it.
Try this yourself. Go to Google News and search for “officer killed”. You will almost certainly find multiple recent stories and multiple outlets covering the same story. Next, search for “fisherman killed”, and you’ll find a lot fewer results returned. When there are results, they are typically only covered by the locale the death happened in and not picked up by national outlets. The news disproportionately covers the death of police officers over fishermen. To be clear, I’m not questioning that practice at all. However, this does explain why most tend to think that the police work is more deadly than being a fisherman. We are more likely to trust the information we can recall more quickly, and by virtue of seeing more news stories about police deaths, the availability heuristic tricks us into thinking that police work is more deadly. I’d hypothesize that if we posed the same question to individuals who were regular viewers of the Discovery Channel show “The Deadliest Catch”, they might recognize the danger associated with commercial fishing and select the correct answer to the question.
One thing we know about human memory and recall is that it is efficient. We often go with the first piece of information that can be recalled from our memory. Not only does the availability of information drive our thoughts, it also shapes our behavior. It’s why advertisers spend so much money to ensure that their product is the first thing we associate with specific inputs. When many Americans think of cheeseburgers they think of McDonalds. When you think of coffee you think of Starbucks. When you think of APT you think of Mandiant. These aren’t accidental associations — a lot of money has been spent to ensure those bonds were formed.
Availability Heuristic in Security
Availability is all about the things you observe the most and the things you observe most recently. Consider these scenarios that highlight examples of how availability can affect decisions in security practice.
Returning from a Security Conference
I recently attended a security conference where multiple presenters showed examples that included *.top domains that were involved with malicious activity. These sites were often hosting malware or being used to facilitate command and control channels with infected machines. One presenter even said that any time he saw a *.top domain, he assumed it was probably malicious.
I spoke with a colleague who had really latched on to that example. He started treating every *.top domain he found as inherently malicious and driving his investigations with that in mind. He even spent time actively searching out *.top domains as a function of threat hunting to proactively find evil. How do you think that turned out for him? Sure, he did find some evil. However, he also found out that the majority of *.top domains he encountered on his network were actually legitimate. It took him several weeks to realize that he had fallen victim to the availability heuristic. He put too much stock in the information he had received because of the recency and frequency of it. It wasn’t until he had gathered a lot of data that he was able to recognize that the assumption he was making wasn’t entirely correct. It wasn’t something that warranted this much of his time.
In another recent example, I saw a colleague purchase a lot of suspected APT owned domains with the expectation that sinkholing them would result in capturing a lot of interesting C2 traffic. He saw someone speak on this topic and thought that his success rate would be higher than it was because they speaker didn’t cover that topic in depth. My colleague had to purchase a LOT of domain names before he got any interesting data, and by that point, he had pretty much decided to give up after spending both a lot of time and money on the task.
It is very hard for someone giving a 30-minute talk to fully support every claim they make. It also isn’t easy to stop and cite additional sources in the middle of a verbal presentation. Because our industry isn’t strict about providing papers to support talks, we end up with a lot of opinions and not much fact. Those opinions get wheels and they may be taken much farther than the original presenter ever intended. This tricks people who are less metacognitively aware into accepting opinions as fact.
Data Source Context Availability
If you work in a SOC, you have access to a variety of data sources. Some of those are much lower context like flow data or DNS logs, and some are much higher context like PCAP data or memory. In my research, I’ve found that analysts are much more likely to pursue high-context data sources when working an investigation, even when lower context data sources contain the information they need to answer their questions.
On one hand, you might say that this doesn’t matter because if you are arriving at the correct answer, why does it matter how you got there? Analytically speaking, we know that the path you take to an answer matters. It isn’t important just to be accurate in an investigation, you also need to be expedient. Security is an economic problem wherein the cost to defend a network needs to be low and the cost to attack it needs to be high. I’ve seen that users who start with higher context data sources when it is not entirely necessary often spend much more time in an investigation. By using higher context data sources when it isn’t necessary, it introduces an opportunity for distractions in the investigation process. The more opportunity for distracting information, the more opportunity that availability bias can creep in as a result of the new information being given too much priority in your decision making. That isn’t to say that all new information should be pushed aside, but you also have to carefully control what information you allow to hold your attention.
Structured Adversary Targeting
In the past five years, the security industry has become increasingly dominated by fear-based marketing. A few years ago it was the notion that sophisticated nation-state adversaries were going to compromise your network no matter who you were. These stories made national news and most security vendors began to shift their marketing towards guaranteeing protection against these threats.
The simple truth is that most businesses are unlikely to be targeted by nation-state level threat actors. But, because the news and vendor marketing have made this idea so prevalent, the availability of it has led an overwhelming number of people to believe that this could happen to them. When I go into small businesses to talk about security I generally want to talk about things like opportunistic attacks, drive-by malware, and ransomware. These are the things small businesses are mostly likely to be impacted by. However, most of these conversations now involve a discussion about structured threat actors because of the availability of that information. I don’t want to talk about these things, but people ask about them. While this helps vendors sell products, it takes some organizations’ eye off the things they should really be concerned about. I’m certain Billy Ray’s Bait Shop will never get attacked by the Chinese PLA, but a ransomware infection has the ability to destroy the entire business. In this example, the abundance of information associated with structured threat actors clouds perspective and takes time away from more important discussions.
Diminishing Availability Heuristic
The stories above illustrate common places availability heuristic manifests in security. Above all else, the availability of information is most impactful to you in how you spend your time and where you focus your attention. Attention is a limited resource, as we can only focus on one or two things at a time. Consider where you place your attention and what is causing you to place it there.
Over the course of the next week, start thinking about the places you focus your attention and actively question why information led you to do that. Is that information based on fact or opinion? Are you putting too much or too little time into your effort? Is your decision making slanted in the wrong direction?
Here are a few ways you can recognize when availability heuristic might be affecting you or your peers and strategies for diminishing its effects:
Carefully consider the difference between fact and opinion. In security, most of the publicly available information you’ll find is a mix of opinions and facts and the distinction isn’t always so clear. Whenever you make a judgment or decision based on something elsewhere, spend a few minutes considering the source of the information and doing some manual research to see if you can validate it elsewhere.
Use patience as a shield. Since your attention is a limited resource, you should protect at accordingly. Just because new information has been introduced doesn’t mean it is worthy of shifting your attention to it. Pump the breaks on making quick decisions. Take a walk or sleep on new information before acting to see if something still matters as much tomorrow as it does today. Patience is a valuable tool in the fight to diminish the effects of many biases.
Practice question-driven investigating. A good investigator is able to clearly articulate the questions they are trying to answer, and only seeks out data that will provide those answers. If you go randomly searching through packet capture data, you’re going to see things that will distract you. By only seeking answers to questions you can articulate clearly, you’ll diminish the opportunity for availability bias to distract your attention.
Utilize a peer group for validation. By definition, we aren’t good at recognizing our own biases. When you are pursuing a new lead or deciding whether to focus your attention on a new task or goal, considering bouncing that idea off of a peer. They are likely to have had differing experiences than you, so their decision making could be less clouded by the recency or availability of information that is affecting you. A question to that group can be as simple as “Is ____ as big of a concern as I think it is?”
—
If you’re interested in learning more about how to help diminish the effects of bias in an investigation, take a look at my Investigation Theory course where I’ve dedicated an entire module to it. This class is only taught periodically, and registration is limited.
[1] http://www.huffingtonpost.com/blake-fleetwood/how-dangerous-is-police-w_b_6373798.html