SANS recently released the results of their SOC survey that was put together by Chris Crowley. The report has a lot of useful data points and is worth your time to go through whether you’re in a SOC and wondering how you stack up against others, or if you’re thinking about establishing a SOC and need to see where the goal posts currently are.

In this post, I want to focus on five takeaways I garnered from the report*. These takeaways will revolve around the human analyst, just as all investigations do.

Heavily Regulated Industries (and vendors) are Leading the Way

Figure 1 illustrates the distribution of SOCs across specific industries. Taking dedicated cyber security and technology companies aside, the industries that appear to have a greater number of SOCs share a commonality of being heavily regulated. This includes government, finance, manufacturing, and healthcare. This seems consistent with the notion that many organizations develop their security operations by first embracing required compliance.

SOC Survey Industries Represented

By virtue of being the first and most prolific adopters of SOCs, these industries will naturally dictate best practices across the field as they mature. The common traits and mindsets predominant in these industries will influence the direction of the SOC as we know it. This will matriculate to cyber security vendors who will inevitably swap staff with practitioners in these SOCs. When combined with vendor’s focusing sales goals towards these industries ensure that vendors are also more likely to build products and produce educational materials that also promote the mindsets predominant in these fields.

A mindset is neither good or bad, and bias can be both helpful and harmful. It’s important we identify common trait distributions and mindset biases associated with these fields so that the evolution of the SOC concept benefits from a diversity of opinion.

 

SIEM as the Investigative Centerpiece

Figure 13 shows how SOC analysts correlate and analyze event data, IOCs, and other security and threat-related data. This chart essentially identifies the tool at the center of the investigative process. 77% cited the use of a SIEM for facilitating the investigation process.

In my experience, many SOCs tend to let the workflow inherent to their SIEM dictate their analyst investigation workflow. New analysts learn primarily via on the job training and through the lens of the workflow dictated by the SIEM. Given there are only a handful of widely popular and accepted SIEMs and investigative theory training isn’t widespread, most analysts currently practicing likely learned their craft via a few popular SIEMs like Arcsight, QRadar, or others. I would posit that a test could be developed wherein you could present an analyst with investigation scenarios and monitor how they solve them to arrive at an accurate assessment of which SIEM they cut their teeth on.

If the SOC doesn’t provide training in fundamental investigation concepts then an additional concern moving forward is that analysts are more likely to be “SIEM-locked” wherein they don’t know how to perform investigations without the use of a specific SIEM.  SOC managers must be certain that their SIEM supports their human-centered workflow rather than developing a workflow solely because it aligns with a SIEM. Currently, my assessment of the SIEM market is that most tools that exist don’t adequately consider or deliver workflow features that focus enough on the needs of the human analyst. It’s likely that many of the 23% of organizations identified as having built their own SIEM like tool share this opinion.

 

Investigation Metrics are Non-Existent

Collecting actionable metrics has been a pain point for most SOCs I’ve worked in or consulted with. Figure 18 describes metrics that are used, enforced, and consistently met. There are very few metrics associated with the investigation experience itself, except for the time from detection to containment and eradication. As the investigation function is the central workflow of the SOC, this continues to be an area where improvement is desired. Instead, most metrics that are considered are focused on SOC output, and not the efficiency of the SOC itself. While this is helpful for justifying the existence of the SOC (why an org spends money on the function), it isn’t as helpful for improving the SOC (reducing the cost of the function).

SOC Metrics Collected

Investigation-centric metrics might include tracking the usage of specific data sources during investigations (assists), the number of times a data source would have been helpful but was unavailable (turnovers), the most commonly aggregated fields, and average time spent viewing specific data sources. An investigation-centric metric is one that seeks to better understand how the human analyst spends their time while attempting to connect the dots in pursuit of greater speed and accuracy.

 

Internationalization of SOCs

A significant number of SOC practitioners exist outside the United States, as shown in Figure 2. However, a much smaller percentage of organizations who responded to the survey are headquartered outside the US. The disproportionate number of international analysts is likely attributed to organizations attempting to cut costs by hiring in lower income regions, and organizations that seek to staff 24×7 operations by staffing in different time zones (thereby avoiding having to hire a night shift in the US, which is notoriously difficult).

Locale of Security Operations

There are significant differences in how people think based on the culture they hail from. By nature, most Americans tend to be less sensitive to these variances and project their way of thinking onto others. As the number of international practitioners grows, it’s critical to consider the biases inherent to how Americans think so that we can identify where they may not hold up for international practitioners. As an example, people from Asian cultures tend to require more certainty about a conclusion than their American counterparts before feeling confident in it. Put more simply, someone from Kansas may view the investigation process completely differently than someone from Kazakhstan. By understanding how differing cultural mindsets impact how people approach investigations, we can draw useful data and conclusions towards a more universal investigation process.

It’s worth noting that SANS is a US-based company with a larger market penetration in the US (I don’t know this for a fact, it is an assumption). Therefore, the respondents for this survey question probably under-represent the number of international practitioners and the survey may not represent an adequate global sample. Without access to the source data, I’m unable to assert confidence regarding sample distribution. None the less, this only seeks to strengthen the points mentioned here.

 

Distributed Environments Require Unique Communication Skills

The increased number of international SOC practitioners in remote SOCs (Figure 2) and the significant number of distributed SOCS (Figure 3) stress the importance of communication in which analysts are not in the same room.

SOC Architectural Approaches

Communication is a critical function of the SOC, and it must be facilitated with appropriate tools. This stresses the importance of investigation tools that provide built-in collaboration features such as the assignment of cases, shared notes, and context tagging. It also stresses the importance of data access and information sharing via tools like wikis or knowledgebases.

Not lost here is the ability to identify and hire staff who excel at non-present communication. As someone who has managed remote teams, I quickly learned that some people simply aren’t effective communicators via text-based mechanisms like chatrooms. Managers should strive to develop strategies for identifying analysts who can excel specifically in these environments. Furthermore, if we can identify these traits and qualities, we should strive to enhance our ability to teach improvement in this communication skillset.

Conclusion

I enjoyed parsing through the SOC survey and want to thank SANS and Chris Crowley for putting it together. In the future, I’d love to see more questions relating to how human analysts perform their jobs and what pain points they have beyond just the tools they use. I’d also love to see this survey repeated on a periodic basis so that trends could be highlighted.

Finally, I’d love to see the methodology used for data collection described here and why they chose the questions they did. I appreciate SANS identifying that the research is sponsored, but citing the methodological approach would shed light on how much influence the vendors had on the questions and the interpretation of their output. A positive step would be making the raw source data publicly available for additional analysis.

I’d love to hear your thoughts on my analysis of the report, including both things you agree and disagree with. You can reach me via Twitter @chrissanders88 or you can e-mail me.

 

*Note: I was not asked to do this by SANS. This post only reflects my analysis and opinions.