Know Your Bias – Anchoring

In the first part of this series I told a personal story to illustrated the components and effects of bias in a non-technical setting. In each post following I’ll examine a specific type of bias, how it manifests in a non-technical example, and provide real-world examples where I’ve seen this bias negatively affect a security practitioner. In this post, I’ll discuss anchoring.

Anchoring occurs when a person tends to rely too heavily on a single piece of information when making decisions, most often based on information received early in the decision-making process.

Anchoring Outside of Security

Think about the average price of a Ford car. Is it higher or lower than 80,000? This number is clearly too high, so you’d say lower. Now let’s flip this a bit. I want you to think about the average price of a Ford car again. Is it higher or lower than 10,000? Once again, the obvious answer is that it’s higher.

These questions may seem obvious and innocent, but here’s the thing. Let’s say that I ask one group of people the first question, and a separate group of people the second question. After that, I ask both groups to name what they think the average price of a Ford car is. The result is that the first group presented with the 80,000 number would pick a price much higher than the second group presented with the 10,000 number. This has been tested in multiple studies with several variants of this scenario [1].

In this scenario, people are subconsciously fixating on the number they are presented and it is subtly influencing their short term mindset. You might be able to think of a few cases in sales where this is used to influence consumers. Sticking with our car theme, if you’ve been on a car lot you know that every car has a price that is typically higher than what you pay. By pricing cars higher initially, consumers anchor to that price. Therefore, when you negotiate a couple thousand dollars off, it feels like you’re getting a great deal! In truth, you’re paying what the dealership expected, you just perceive the deal because of the anchoring effect.

They key takeaway from these examples is that anchoring to a specific piece of information is not inherently bad. However, making judgements in relation to an anchored data point where too much weight is applied can negatively effect your realistic perception of a scenario. In the first example, this led you to believe the average price of a car is higher or lower than it really is. In the second example, this led you to believe you were getting a better deal than you truly were.

 

Anchoring in Security

Anchoring happens based on the premise that mindsets are quick to form but resistant to change. We quickly process data to make an initial assessment, but our ability to hone that assessment is generally weighed in relation to the initial assessment. Can you think of various points in a security practitioner’s day where there is an opportunity for an initial perception into a problem scenario? This is where we can find opportunities for anchoring to have occurred.

List of Alerts

Let’s consider a scenario where you have a large number of alerts in a queue that you have to work through. This is a common scenario for many analysts, and if you work in a SOC that isn’t open 24×7 then you probably walk in each morning to something similar. Consider this list of the top 5 alerts in a SOC over a twelve hour period:

  • 41 ET CURRENT_EVENTS Neutrino Exploit Kit Redirector To Landing Page
  • 14  ET CURRENT_EVENTS Evil Redirector Leading to EK Apr 27 2016
  • 9 ET TROJAN Generic gate[.].php GET with minimal headers
  • 2 ET TROJAN Generic -POST To gate.php w/Extended ASCII Characters (Likely Zeus Derivative)
  • 2 ET INFO SUSPICIOUS Zeus Java request to UNI.ME Domain

Which alert should be examined first? I polled this question and found that a significant number of inexperienced analysts chose the one at the top of the list. When asked why, most said because of the frequency alone. By making this choice, the analyst assumes that each of these alerts are weighted equally. By occurring more times, the rule at the top of the list represents a greater risk. Is this a good judgement?

In reality, the assumption that each rule should be weighted the same is unfounded. There are a couple of ways to evaluate this list.

Using a threat-centric approach, not only does each rule represent a unique threat that should be considered uniquely, some of these alerts gain more context in the presence of others. For example, the two unique Zeus signatures alerting together could pose some greater significance. In this case, the Neutrino alert might represent a greater significance if it was paired with an alert representing the download of an exploit or communication with another Neutrino EK related page. Merely hitting a redirect to a landing page doesn’t indicate a successful infection, and is a fairly common event.

You could also evaluate this list with a risk-centric approach, but more information is required. Primarily, you would be concerned with the affected hosts for each alert. If you know where your sensitive devices are on the network, you would evaluate the alerts based on which ones are more likely to impact business operations.

This example illustrates the how casual decisions can come with implied assumptions. Those assumptions can be unintentional, but they can still lead you down the wrong path. In this case, the analyst might spend a lot of time pursuing alerts that aren’t very sensitive while delaying the investigation of something that represents a greater risk to the business. This happens because it is easy to look at a statistic like this and anchor to a singular facet of the stat without fully considering the implied assumptions. Statistics are useful for summarizing data, but they can hide important context that will keep you from making uninformed decisions that are the result of an anchoring bias.

 

Visualizations without Appropriate Context

As another example, understand that numbers and visual representations of them have a strong ability to influence an investigation.  Consider a chart like the one in the figure below.

This is a treemap visualization used to show the relative volume of communication based on network ports for a single system. The larger the block, the more communication occurred to that port. Looking at this chart, what is the role of the system whose network communication is represented here? Many analysts I polled decided it was a web server because of the large amount of port 443 and 80 traffic. These ports are commonly used by web servers to receive requests.

This is where we enter the danger zone. An analyst isn’t making a mistake by looking at this and considering that the represented host might be a web server. The mistake occurs when the analyst fully accepts this system as a web server and proceeds in an investigation under that assumption. Given this information alone, do you know for sure this is a web server? Absolutely not.

First, I never specified whether this treemap exclusively represents inbound traffic, and it’s every bit as likely that it represents outbound communication that could just be normal web browsing. Beyond that, this chart only represents a finite time period and might not truly represent reality. Lastly, just because a system is receiving web requests doesn’t necessarily mean its primary role is that of a web server. It might simply have a web interface for managing some other service that is its primary role.

The only way to truly ascertain whether this system is a web server is to probe it to see if there is a listening service on a web server port or to retrieve a list of processes to see if a web server application is running.

There is nothing wrong with using charts like this to help characterize hosts in an investigation.  This treemap isn’t an inherently bad visualization and it can be quite useful in the right context. However, it can lead to investigations that are influenced by unnecessarily anchored data points. Once again, we have an input that leads to an assumption.  This is where it’s important to verify assumptions when possible, and at minimum identify your assumptions in your reporting. If the investigation you are working on does end up utilizing a finding based on this chart and the assumption that it represents a web server, call that out specifically so that it can be weighed appropriately.

 

Diminishing Anchoring Bias

The stories above illustrate common places anchoring can enter the equation during your investigations. Throughout the next week, try to look for places in your daily routine where you form an initial perception and there is an opportunity for anchoring bias to creep in. I think you’ll be surprised at how many you come across.

Here are a few ways you can recognize when anchoring bias might be affecting you or your peers and strategies for diminishing its effects:

Consider what data represents, and not just it’s face value. Most data in security investigations represents something else that it is abstracted from. An IP address is abstracted from a physical host, a username is abstracted from a physical user, a file hash is abstracted from an actual file, and so on.

Limit the value of summary data. A summary is meant to be just that, the critical information you need to quickly triage data to determine its priority or make a quick (but accurate) judgement of the underlying events disposition. If you carry forward input from summary data into a deeper investigation, make sure you fully identify and verify your assumptions.

Don’t let your first impression be your only impression. Rarely is the initial insertion point into an investigation the most important evidence you’ll collect. Allow the strength of conclusions to be based on your evidence collected throughout, not just what you gathered at the onset. This is a hard thing to overcome, as your mind wants to anchor to your first impression, but you have to try and overcome that and try to examine cases holistically.

An alert is not an answer, it’s merely a question. Your job is to prove or disprove the alert, and until you’ve done one of those things the alert is not representative of a certainty. Start looking at alerts as the impetus for asking questions that will drive your investigation.

If you’re interested in learning more about how to help diminish the effects of bias in an investigation, take a look at my Investigation Theory course where I’ve dedicated an entire module to it. This class is only taught periodically, and registration is limited.

[1] Strack, F., & Mussweiler, T. (1997). Explaining the enigmatic anchoring effect: Mechanisms of selective accessibility. Journal of personality and social psychology73(3), 437.

 

 

Know your Bias – Foundations

In this blog series I’m going to dissect cognitive biases and how they relate to information security. Bias is prevalent in any form of investigation, whether you’re threat hunting, reversing malware, responding to an incident, attributing network attacks, or reviewing an IDS alert. In each post, I’ll describe a specific type of bias and how it manifests in various information security specialties. But first, this post will explain some fundamentals about what bias is and why it can negatively influence investigations.

 

What is Bias?

Investigating security threats is a process that occurs within the confines of your mind and centers around bridging the gap between your limited perception and the reality of what has actually occurred. To investigate something is to embrace that perception and reality aren’t the same thing, but you would like for them to be. The mental process that occurs while trying to bridge that perception-reality gap is complex and depends almost entirely on your mindset.

A mindset is how you see and approach the world and is shaped by your genetics and your collective world experience. Everyone you’ve ever met, everything you’ve ever done, and every sense you’ve perceived molds your mindset. At a high level, a mindset is neither a good or bad thing, but it can lead to positive or negative results. This is where bias comes in.

Bias is a predisposition towards a certain way of thinking, and it can be the difference in a successful or failed investigation. In some ways, bias is good when it allows us to learn from our previous mistakes and create mental shortcuts to solving problems. In other ways its bad, and can lead us to waste time pursuing bad leads or jump to conclusions without adequate evidence. Let’s consider a non-technical example first.

 

The (very) Personal Effects of Bias

On a night like any other, I laid my head down on my pillow at about 11 PM. However, I was unexpectedly awoken around 2 AM with wrenching stomach pain unlike anything I’d ever felt before. I tossed and turned for an hour or so without relief, and finally woke my wife up. A medical professional herself, she realized this might not be a typical stomach ache and suggested we head to the emergency room.

About an hour later I was in the ER being seen by the doctor on shift that night. Based on the location of the pain, he indicated the issue was likely gal bladder related, and that I probably had one or more gall stones causing my discomfort. I was administered some pain medication and setup with an appointment with a primary care physician the next day for further evaluation.

The primary care physician also believed that a gal bladder was the likely cause of the previous night’s discomfort, so she scheduled me for an ultrasound the same day. I walked over to the ultrasound lab where they spent about twenty minutes trying to get good images of the area. Shortly thereafter, the ultrasound technician came in and looked at the image and shared his thoughts. He had identified my gal bladder and concluded that it was full of gal stones, which had caused my stomach pain. My next stop was a referral to a surgeon, where my appointment took no more than ten minutes as he quickly recommended I switch to a low fat diet for a few weeks until he could perform surgery to remove the malfunctioning organ.

Fast forward a couple of weeks later and I’m waking up from an early morning cholecystectomy operation. The doctor is there as soon as I wake up, which strikes me as odd even in my groggy state.

“Hey Chris, everything went fine, but…”

Let me take this opportunity to tell you that you never want to hear “but…” seconds after waking up from surgery.

“But, we couldn’t remove your gal bladder. It turns you don’t have one. It’s a really rare thing and less than .1% of the population is born without one, but you’re one of them.”

At first I didn’t believe him. As a matter of fact, I was convinced that my wife had put him up to it and they were messing with me while I wasn’t completely with it. It wasn’t until I got home that same day and woke up again several hours later that I grasped what had happened. I really wasn’t born with a gal bladder, and the surgery had been completely unnecessary.

 

Figure 1: This is a picture of where my gal bladder should be

 

Dissecting the Situation

Let’s dissect what happened here.

  1. ER Doctor: Believed I was having gal bladder issues based on previous patients presenting with similar symptoms. Could not confirm this in the ER, so referred me to a primary care physician.
  2. Primary Care Doctor: Also believed the gal bladder issue was likely, but couldn’t confirm this in her office, so she referred me to a radiologist.
  3. Radiologist: Reviewed my scans to attempt to confirm the diagnosis of the previous two doctors. Found what appeared to be confirming evidence and concluded my gal bladder was malfunctioning.
  4. Surgeon: Agreed with the conclusion of the radiologist (without looking at the source data himself) and proceeded to recommend surgery, which I went through.

So where and why did things go wrong?

For the most part, the first two doctors were doing everything within their power to diagnose me. The appropriate steps of a differential diagnosis instruct physicians to identify the most likely affliction and perform the test that can rule that out. In this case, that’s the ultrasound that was performed by the radiologist, which is where things started to go awry.

The first two doctors presented a case for a specific diagnosis, and the radiologist was predisposed towards confirming this bias before even looking at the scans. It’s a classic case of finding something weird when you go looking for it, because everything looks a little weird when you want to find something. In truth, the radiologist wasn’t confident in his finding, but he let the weight of the other two doctor’s opinions bear down on him such that he was actually seeking to confirm their diagnosis more than trying to independently come to an accurate conclusion. This is an example of confirmation bias, which his perhaps the most common type of bias encountered. It also represents characteristics of belief bias and anchoring.

The issue here was compounded when I met the surgeon. Rather than critically assessing the collective findings of all the professionals that were involved to this point, he assumed a positive diagnosis and merely glanced at the ultrasound results in passing. All of the same biases are repeated here again.

In my case bias led to an incorrect conclusion, but understand that even if my gal bladder had been the issue and the surgery went as expected, the bias was still there. In many (and sometimes most) cases, bias might exist and you will still reach the correct conclusion. That doesn’t mean that the same bias won’t cause you to stumble later on.

 

Consequences of Bias

In my story, you see that bias resulted in some fairly serious consequences. Surgery is a serious matter, and I could have suffered some type of complication while on the table, or a post op infection that could have resulted in extreme sickness or death.

In any field, bias influences conclusions and those conclusions have consequences when they result in action being taken. In my case, it was a few missed days of work and a somewhat painful recovery. In security investigations, bad decisions can result in wasted time pursuing leads or missing something important. The latter could have drastic effects if you work for a government, military, ICS environment, or any other industry where lives depend on system integrity and uptime. Even if normal commercial environments, bad decisions resulting from the influence of bias can lead to millions of lost dollars.

Let’s examine one other effect of this scenario. I’m a lot less likely to trust the conclusions of doctors now, and I’m a lot less likely to agree to surgery without a plethora of supporting evidence indicating the need for it. These things in themselves are a form of bias. This is because bias breeds additional bias. We saw this with the relationship between the radiologist and the surgeon as well.

The effects of bias are highly situational. It’s best not to think of bias as something that dramatically changes your entire outlook on a situation. Think of bias like a pair of tinted glass that subtly influences how you perceive certain scenarios. When the right bias meets the right set of circumstances, things can go bad quickly.

 

Countering Bias

Bias is insanely hard to detect because of how it develops in either an individual or group setting.

In an individual setting, bias is inherent to your own decision making. Since humans inherently stink at detecting their own bias, it is unlikely you will become aware of it unless your analysis is reviewed by someone else who points it out. Even then, bias usually exists in small amounts and is unlikely to be noticed unless it meets a certain threshold that is noticeable by the reviewer.

In a group setting, things get complicated because bias enters from multiple people in small amounts. This creates a snowball effect in which group bias exists outside the context of any specific individual. Therefore, if hand offs occur in a linear manner such that each person only interacts one degree downstream or upstream, it is only possible to detect overwhelming bias at the upstream levels. Unfortunately, in real life these upstream levels are usually where the people are less capable of detecting the bias because they have lesser subject matter expertise in the field. In security, think managers and executives trying to catch this bias instead of analysts.

Let me be clear – you can’t fully eliminate bias in an investigation or in most other walks of your life. The way humans are programmed and the way our mindsets work prohibit that, and honestly, you wouldn’t want to eliminate all bias because it can be useful at times too. However, you can minimize the effects of negative bias in a couple of ways.

Evidence-Based Conclusions

A conclusion without supporting evidence is simply a hypothesis or a belief. In both medicine and information security, belief isn’t good enough without data to support it. This is challenging in many environments because visibility isn’t always in all the places we need it, and retention might not be long enough to gather the information you need when an event occurred far enough in the past. Use these instances to drive your collection strategy and justify appropriate budget for making sound conclusions.

In my case, two doctors made hypotheses without evidence. Another doctor gathered weak evidence and made a bad decisions, and another doctor confirmed that bad decision because the evidence was presented as a certainty when it actually wasn’t. A review of the support facts would have led the surgeon to catch the error before deciding to operate.

Peer Review

By definition, you aren’t aggressively aware of your own bias. Your mindset works the way it works and that is “normal” to you, so your ability to spot biased decisions when you make them is limited. After all, nobody comes to a conclusion they know to be incorrect. As Sheldon from the Big Bang Theory would say, “If I was wrong, don’t you think I’d know it?”

This is where peers come in. The surgeon might have caught the radiologist’s error if he had thoroughly reviewed the ultrasound results. Furthermore, if there was a peer review system in place in the radiology department, another person might have caught the error before it even got that far. Other people are going to be better at identifying your biases than you are, so that is an opportunity to embrace your peers and pull them in to review your conclusions.

Knowledge of Bias

What little hope you have of identifying your own biases doesn’t manifest during the decision-making process, but instead during the review of your final products and conclusions. When you write reports or make recommendations, be sure to identify the assumptions you’ve made. Then, you can review your conclusions as weighed against supporting evidence and assumptions to look for places bias might creep in. This requires a knowledge of common types of bias and how they manifest, which is precisely the purpose of this series of blog posts.

Most physicians are trained to understand and recognize certain types of bias, but that simply failed in my case until after the major mistakes had been made.

 

Conclusion

The investigation process provides an opportunity for bias to affect conclusions, decisions, and outcomes.  In this post I described a non-technical example of how bias can creep in while attempting to bridge the gap from perception to reality. In the rest of this series I’ll focus on specific types of bias and provide technical and non-technical examples of the bias in action, along with strategies for recognizing the bias in yourself and others.

As it turns out my stomach aches eventually stopped on their own. I never spoke to the radiologist again, but I did speak to the surgeon at length and he readily admitted his mistake and felt horrible for it. Some people might be furious at this outcome, but in truth, I empathized with his plight. Complex investigations, be it medical or technical, present opportunity for bias and we all fall victim to it from time to time. We’re all only human.

 

 

If you’re interested in learning more about how to help diminish the effects of bias in an investigation, take a look at my Investigation Theory course where I’ve dedicated an entire module to it. This class is only taught periodically, and registration is limited.

Announcing the Investigation Theory Online Course

Investigation Theory LogoI’m excited to announce my newest training course, with a portion of the proceeds supporting multiple charities.

Register Here

When I first started out, learning how to investigate threats was challenging because there was no formal training available. Even in modern SOCs today, most training is centered around specific tools and centers too much around on the job training. There has never been a course dedicated exclusively to the fundamental art and science of the investigation process…until now.

If you’re a security analyst responsible for investigating alerts, performing forensics, or responding to incidents then this is the course that will help you gain a deep understanding how to most effectively catch bad guys and kick them out of your network. Investigation Theory is designed to help you overcome the challenges commonly associated finding and catching bad guys.

  • I’ve got so many alerts to investigate and I’m not sure how to get through them quickly.
  • I keep getting overwhelmed by the amount of information I have to work with an investigation.
  • I’m constantly running into dead ends and getting stuck. I’m afraid I’m missing something.
  • I want to get started threat hunting, but I’m not sure how.
  • I’m having trouble getting my management chain to understand why I need the tools I’m requesting to do my job better.
  • Some people just seem to “get” security, but it just doesn’t seem to click for me.

Course Format

Investigation Theory is not like any online security training you’ve taken. It is modeled like a college course and consists of two parts: lecture and lab.  The course is delivered on-demand so you can proceed through it at your convenience. However, it’s recommended that you take a standard 10-week completion path, or an accelerated 5-week path. Either way, there are ten modules in total, and each module typically consists of the following components:

  • 1 Core Lecture: Theory and strategy is discussed in a series of video lectures. Each lecture builds on the previous one.
  • 1 Bonus Lecture: Standalone content to address specific topics is provided in every other module.
  • 1 Reading Recommendation: While not meant to be read on pace with the course, I’ve provided a curated reading list along with critical questions to consider to help develop your analyst mindset.
  • 1 Quiz: The quiz isn’t meant to test your knowledge, but rather, to give you an opportunity to apply it to reinforce learning through critical thinking and knowledge retrieval.
  • 1 Lab Exercise: The Investigation Ninja system is used to provide labs that simulate real investigations for you to practice your skills.

Investigation Ninja Lab Environment

This course utilizes the Investigation Ninja web application to simulate real investigation scenarios. By taking a vendor agnostic approach, Investigation Ninja provides real world inputs and allows you to query various data sources to uncover evil and decide if an incident has occurred, and what happened. You’ll look through real data and solve unique challenges that will test your newly learned investigation skills. A custom set of labs have been developed specifically for this course. No matter what toolset you work with in your SOC, Investigation Ninja will prepare you to excel in investigations using a data-driven approach.

This slideshow requires JavaScript.

Get stuck in a lab? I’m just an e-mail away and can help point you in the right direction. Enjoy the labs and want to go farther? You can purchase additional access to more labs, including our upcoming “Story Mode” where you create a character and progress through eight levels of investigation scenarios while trying to attain the rank of Investigation Ninja!

Instructor Q&A

This isn’t a typical online course where we just give you a bunch of videos and you’re own your own. The results of your progress, quizzes, and labs are reviewed by me and I provide real time feedback as you progress. I’m available as a resource to answer questions throughout the course.

Syllabus

  1. Metacognition: How to Approach an Investigation
  2. Evidence: Planning Visibility with a Compromise in Mind
  3. Investigation Playbooks: How to Analyze IPs, Domains, and Files
  4. Open Source Intel: Understanding the Unknown
  5. Mise en Place: Mastering Your Environment with Any Toolset
  6. The Timeline: Tracking the Investigation Process
  7. The Curious Hunter: Finding Investigation Leads without Alerts
  8. Your Own Worst Enemy: Recognizing and Limiting Bias
  9. Reporting: Effective Communication of Breaches and False Alarms
  10. Case Studies in Thinking Like an Analyst

Plus, several bonus lectures!

Cost

The course and lab access are $497 for a single user license. Discounts are available for multiple user licenses where at least 10 seats are purchased (please contact me to discuss payment). A significant portion of the purchase price will go to support multiple charities including the Rural Technology Fund, the Against Malaria Foundation, and others.

You’ll receive:

  • 1-yr Access to Course Videos and Content
  • 1-yr Access to Investigation Ninja
  • A Certification of Course Completion
  • Continuing Education Credits (CPEs/CEUs)

Sign Up Now!

This course is only taught periodically and space is limited.

Spring 2017 Session 1 – Beings January 9th SOLD OUT

Spring 2017 Session 2 – Begins March 20

  • Register now for the March session, as pricing will increase by $100 after January 1st

Making an Impact with Local Security Conferences

Student's Using OSMO Coding Kits

Student’s Using Donated OSMO Coding Kits

Running a non-profit is really tough sledding. It requires a complex balance of spending just enough to raise awareness, while ensuring that the donations you are bringing in are substantial enough to make a positive impact on the world. The absolute best way to ensure success is to partner with other people who are like minded and willing to help.

I’m excited to announce a recent effort resulting from a partnership between the Rural Tech Fund and the great folks who run the Archcon security conference. One of the organizers, Paul, contacted me a month or so ago and asked if the RTF could use the funds generated from the conference to do some positive things in rural and low income areas in Missouri. We made the commitment (as I do with all donations to the RTF) that we would use 100% of the donation to donate equipment to school districts in the area. With that money, we were able to do the following:

  • Robotics and Ardunio kits in the St. Louis / Mehlville area. 
  • Programming kits in Gladstone, MO
  • Robotics and Ardunio kits in Essex, MO. 
  • Electronics kits in Saint Charles, MO
  • Chromebooks for programming classes in the St. Louis / Jennings area
  • Circuitry and robotics kits in the St. Louis / Mason area
  • Raspberry Pi kits in Independence, MO
  • Robotics and coding kit to El Dorado Springs, MO

With a relatively small amount of money, we were able to make donations that will directly impact around 600 students across Missouri. By utilizing giving networks like Donors Choose and matching funds from organizations like the Ewing Marion Kauffman Foundation, the value of the money was maximized to ensure reach to the most number of students.

Don’t just hear about the impact from me though, take it from a couple teachers in these classrooms:

“I am humbled and grateful for the generous donation from the Rural Technology Fund. It will be thrilling to watch the students interact with their new technology and enhance their creative potential. Expressing my thanks does not relay the full measure of emotions at this moment. I am incredibly appreciative…to the point of tears.” – Dr. Flynn (St. Louis)

“Thank you so much for seeing my vision for my students. Your contribution to my class will forever impact the students. To know that one person’s generosity can change the lives of others is the greatest gift ever. Your contribution will bring to STEM to life in my class”. – Ms. Jefferson (Jennings, MO)

While Archcon already had a tangible impact in the security community, this ensured that the conference will have a lasting impact that pays dividends for underprivileged students in the state, as well as for the overall economy of the state.

It’s sometimes hard to find massive wins like this, but this is one I’m very proud to be a part of. I want to thank Paul Jaramillo and the folks who organized and participated in Archcon. It’s a fine conference and I plan to attend myself next year.

If you run a security conference and want to help connect your conference to your community and make a similar impact, please reach out to me. Your donation is tax deductible and I’ll commit to using 100% of it to support technology education. The RTF is a volunteer led organization, so nothing will be eaten up by administrative costs. 

Three Useful SOC Dashboards

I worked in security operation centers for a long time, and I really grew to hate dashboards. Most of them were specially designed pages by vendors meant to impress folks who don’t know any better when they stroll through the SOC and glance at the wall of low-end plasmas. They didn’t really help me catch bad guys any better, and worse yet, my bosses made my ensure they were always functional. Fast forward a few years, and I end up working for a vendor who builds security products. Much to my dismay, while planning for features we end up having to build these same dashboards because, despite my best efforts to persuade otherwise, CISO’s consistently ask for eye candy, even while admitting that it doesn’t have anything to do with the goal of the product. Some of them even tell us, straight up, that they won’t purchase our product if it doesn’t have eye catching visuals.

I provide that backstory to provide some insight into my long, tortuous relationship with useless dashboards. I talk about this enough at work that I feel like I’ve almost created a support group for people who have stress triggers associated with dashboards. If you’ve ever attended a conference talk from my good friend Martin Holste, you may know he hates dashboards even more than me. Alas, I’m not here just to rant. I actually believe that dashboards can be useful if they focus less on looking like video games and they help analysts do their job better. So, in this post I’m going to talk about three dashboard metrics you can collect right now that are actually useful. They won’t look pretty, but they will be effective.

Data Availability

The foundation of any investigation is rooted in asking questions, making hypotheses, and seeking answers that either disprove or prove your educated guesses. Your questioning and answer seeking with both be driven, in part, based on the data you have available. If you have PCAP data then you know you can seek answers about the context within network communication, and if you have Sysmon configured on your Windows infrastructure, you know you can look for file hashes in process execution logs.

While the existence of a data source is half the battle, the other half is retention. Some sources might have a specific time window. You might store PCAP for 3 day and flow data for 90 days, for example. Other data sources will probably use a rolling window, like most logs on Windows endpoints that are given a disk quota and roll over when that quota is met. In both cases, the ability to quickly ascertain the availability of data you have to work with is critical for an analyst. In short, if the data isn’t there, you don’t want to waste time trying to look for it. I contend that any time spent gathering data is wasted time, because the analyst should spend most of their time in the question and answer process or drawing conclusions based on data they’ve already retrieved.

A data availability section on a live dashboard helps optimize this part of the analyst workflow by providing a list of every data source and the earliest available data.

dashboard-dataavailability

In the example above I’ve created a series of tiles representing five different data types common to a lot of SOCs. Each tile boldly displays the name of the data source, and the earliest available date and time of data for it. In this example, I’ve also chosen to color code certain tiles. Data sources with a fixed retention period are green, sources with a rolling retention period based on a disk quota are yellow and red. I’ve chosen to highlight endpoint logs in red because those are not centralized and are more susceptible to a security event causing the logs to roll faster. The idea here is to relay some form of urgency into the analyst if they need to gather data from a particular source. While PCAP, flow, and firewall logs are likely to be there a few hours later, things can happen that will purge domain auth and Windows endpoint logs.

Ideally, this dashboard component is updated quickly and in an automated fashion. At minimum, someone updating this manually once a day will still save a lot of time for the individual analyst or collective group.

Open Case Status

Most SOCs use some form of case tracking or management system. While there aren’t a lot of really great options that are designed with the SOC in mind, there are things people find a way to make work like RTIR, Remedy, Archer, JIRA, and more. If integrated properly, the case management system can be a powerful tool for facilitating workflow when you assign users to cases and track states properly. This can be a tremendous tool for helping analysts organized, either through self organization or peer accountability.

 

dashboard-casestatus

In this example, I’ve gone with a simple table displaying the open cases. They are sorted and color coded by alive time, which is the time since the case was opened. As you might expect, things that have been pending for quite some time are given the more severe color as they require action. This could, of course, be built around an SLAs or internal guidelines you use for required response and closure times.

The important thing here is that this dashboard component shows the information the analysts needs to know. This provides the ability to determine what is open (case number), who they can talk to about it (owner), how serious it is (status), what it’s waiting on (pending), and how long have we known about the issue (alive).

Unsolved Mysteries

On any given day an analyst will run into things that appear to be suspicious, but for which there is no evidence to confirm that suspicion. These unsolved mysteries are usually tied to a weird external IP address or domain name, or perhaps an internal user or system. In a single analyst SOC this is easily manageable because if that analyst runs across the suspicious thing again it is likely to draw attention. That is a tougher proposition in the larger SOC however, because there is a chance that a completely different analyst is the one who runs across the suspicious entity the second time. In truth, you could have half a dozen analysts who encounter the same suspicious thing in different contexts without any of them knowing about the other persons finding. Each encounter could hold a clue that will unravel the mystery of what’s going on, but without the right way to facilitate that knowledge transfers something could be missed.

As a dashboard component,  using watch lists to spread awareness of suspicious entities is an effective strategy. To use it, analysts must  have a mechanism for adding things to a watch list, which is displayed on a screen for reference. Any time an analyst runs across something that looks suspicious but they can’t quite pin down, they first check the screen and if it’s not on there, they add it. Everything that shows up on this list is auto cycled off of it every 24-48 hours unless someone else puts it back on there.

dashboard-weirdthings

In this component, I’ve once again chosen a simple table. This provides the thing that is weird (item), who to talk to about it (observer), when it was observed in the data (date), and where you can go to find out the context of the scenario in which it was found (case) if there is any.

Conclusion

A Dashboard doesn’t have to use a fancy chart type or have lasers to be useful. In this post I described three types of information that are useful in a SOC when displayed on a shared dashboard. The goal is to use group dashboards to help analysts save time or be more efficient in their investigations. If you have the capacity to display this information, you’ll be well on your way to doing both of those things.

 

Do you have a really useful dashboard idea that you think is relevant in most SOCs? Let me know and I might blog about it down the road in a follow up.

Interested in learning more about the investigation process and how these dashboards fit in? Sign up for my mailing list to get first shot at my upcoming course focused entirely on the human aspect of security investigations.