I’ve written a lot about the metacognitive gap in digital forensics and incident response. As an industry, we aren’t very effective at identifying the skills that make our expert practitioners so good at what they do, and we are even worse at teaching them. While there are a myriad of skills that make someone good at finding evil and eradicating adversaries, what’s the most important? What is the “X Factor” that makes an investigator great?

That’s a highly subjective question and everyone has an opinion on it biased towards his or her own experience. Regardless, I recently posed this question to some of my coworkers and Twitter followers. The most common answer I received was centered on curiosity.

Based on these results, I conducted a semi-formal survey where I asked 12 experienced analysts to rate the importance of possessing a highly curious mind while attempting to solve an investigation.

In the first survey item, I asked respondents to address the statement “A curious mind is important for an investigator to arrive at a state of resolution in an investigation with accurate and thorough results.”

All 12 investigators responded Strongly Agree using a 5-point Likert scale.

In a second question, I asked respondents to address the statement “A curious mind is important for an investigator to arrive at a state of resolution in an investigation in a timely manner.”

Using the same rating sale, 10 investigators responded Strongly Agree and 2 responded Agree.

Finally, I asked respondents to address the statement “A curious mind is a primary factor in determining whether an investigator will be successful in resolving/remediating an investigation.”.

Using the same rating sale, all 12 analysts responded Strongly Agree.

Clearly, expert practitioners believe that a curious mind is important in terms of accuracy, thoroughness, and speed at which an investigation is conducted. While curiosity isn’t the only thing makes an investigator successful in their craft, it certainly warrants attention as a key player. In this post I will talk about curiosity as a trait, how it manifests in the investigative process, how it’s measured, and whether it’s a teachable skill.

What is Curiosity?

There are many definitions of curiosity scattered across psychology research text, but the one I think most accurately depicts the construct from an applied perspective comes from Litman and Spielberger (2003). They state that curiosity can be broadly defined as a desire to acquire new information, knowledge, and sensory experience that motivates exploratory behavior.

Lowenstein (1994) also provides relevant insight by defining curiosity as “the desire to know.” In this sense, he describes that a desire to know more can arise when a person encounters stimuli that are inconsistent with an idea he or she holds. When this is experienced, the person may feel some kind of deprivation that can only be alleviated by resolving this inconsistency and closing the knowledge gap that has been identified. This jives well with the thoughts of other great psychology thinkers like Kant and Freud.

Curiosity takes form early in life when infants start exploring the world around them to test the limitations of their own body. Many developmental psychologists agree that this curiosity and simple but constant experimentation is the foundation of early learning and normal development. As we grow older, our curiosity continues to spark experimentation.

While curiosity has been considered a research-worthy construct from a theoretical perspective, there has been little effort put into pinning down the neural substrates that underlie it. This is unfortunate, but something to look forward to as neurology and brain imaging techniques continue to rapidly advance.

As it relates to computer security investigations, curiosity manifests practically in a number of ways that most of us can easily recognize. A few of those include the following:

 

Dead End Scenarios

The most dreaded scenario in an investigation occurs when an investigator reaches a point where there are still unanswered questions, but there are no leads left to pursue answers. This is common, especially when things like data retention and availability often limit us. In these scenarios a required data source might not be available, a lead from other evidence could run dry, or the data might not point to an obvious next step.

A limited amount of curiosity can be correlated with an increased number of dead end experiences encountered by an investigator. Without adequate motivation to explore additional avenues for answering questions, the investigation might miss logical paths to the answers they are seeking. They may also fail to adequately ask the appropriate questions.

 

Hypothesis Generation

The investigative process provides many opportunities for open-ended questions, such as “What is responsible for the network traffic?” or “Why would this internal host talk to that external host?” The process of reasoning through these questions is usually characterized initially by divergent thinking to generate ideas to be explored in route to a possible solution. This manifests as an internal dialog when conducted by a single analyst, but can be expressed verbally when a group is involved.

When presented with an open-ended question, curiosity is responsible for motivating the internal evaluation of hypothetical situations. Without curiosity, an individual won’t conduct mind-wandering exercises and may only consider a small number of potential hypotheses when there is potential for many other valid ones. In this scenario an investigator might not be pursuing the correct answers because they haven’t considered all of the potential questions that should be asked.

Note: It’s probably worth noting here that creativity plays a role in this process too, and is linked to curiosity depending on which model you subscribe to. That, however, is a little beyond the scope of what I want to talk about here.

 

Data Manipulation

Looking at the same data in different ways can yield interesting results. This can include using sed, grep, and awk to pull specific columns out of a data stream for comparison, using uniq and sort to aggregate field values, or reducing PCAP data into flows for comparison of multiple data streams.

While having the skills to manipulate data is a separate discussion, having the desire to find out if the manipulation of existing data into a new format will yield useful results is a product of curiosity. Investigators who lack curiosity to find out if such an exercise would be fruitful end up in more dead end scenarios and may take longer routes towards resolving investigations.

 

Pivoting to Tangential Evidence

The goal of collecting and reviewing evidence is to yield answers relevant to the question(s) an investigator has asked. However, it’s common for the review of evidence to introduce tangential questions or spawn completely new investigations. Within an investigation, you might review network connections between a friendly internal host and a potentially hostile external host only to find that other friendly devices have communicated with the hostile device and warrant examination. In another example, while examining web server logs for exploitation of a specific vulnerability, you might find unrelated evidence of successful SQL injection that warrants a completely separate investigation.

Curiosity is a key determinant in whether an investigator chooses to pivot to these tangential data points and pursue their investigation. Without the motivation that curiosity provides, an investigator may neglect to provide more than a cursory glance to these data points, or fail to note them down for later review. This can result in missed intrusions or improper scoping.

Relating Curiosity and Experience

Our cognitive processes don’t operate in a vacuum. Any decision we make is influenced by a symphony of different traits and emotions working in concert together. Some work in perfect harmony while others operate as opposing forces, and curiosity is not exempt. When we talk about curiosities role in an investigation, we also have to talk about experience.

Earlier, I mentioned that curiosity is pivotal to human development, and that our experimentation at an early age is motivated by curiosity to learn more about ourselves and the world around us. This isn’t a characteristic that goes away with time; we just become more aware of it. As we get older, we gain more experience and become more selective of what experiments we conduct. This manifests in many forms of our lives and in every day decisions. For example, a person who has never slept in on a Tuesday might hit the snooze button a few times because curiosity motivates them to explore the benefits and/or consequences of that action.

Experience serves as both a motivating and regulating force for curiosity. In an investigation, I believe this is best illustrated by assessing curiosity and experience as they relate to each other. Consider the following scenarios where we assess the level of curiosity (C) and experience (E) possessed by an individual investigator.

High C / Low E:

With a lot of curiosity but little experience, an investigator is jumpy. This person’s curiosity drives them to dig into everything that seems new, and without experience to regulate it, this persons ends up chasing a lot of ghosts. They will encounter dead end scenarios frequently because they will choose to pursue inconsequential leads within the evidence they are reviewing. They will rarely admit to encountering a dead-end scenario because their lack of experience doesn’t permit them to realize they’ve encountered one. This person will generate many ideas when hypothesis generation is required, but many of those ideas will be unrealistic because of a lack of experience to weed out the less useful ones. They will seek alternate views of data constantly, but will spend a considerable amount of time pursuing alternate views that don’t necessarily help them. Instead of learning to use tools that get them close to the views they want, they’ll spend time attempting to do more manual work to get the data precisely how they desire even if going that extra 20% doesn’t provide a discernable benefit to their investigation. Even though this person will spend a lot of time failing, they will fail fast and gain experience quickly.

Low C / High E

An investigator possessing a lot of experience but little curiosity could be described as apathetic. This doesn’t necessarily mean they aren’t effective at all, but it does make them less likely to investigate tangential leads that might be indicative of a larger compromise scope or a secondary compromise. In many cases, a person in this state may have started with a high degree of curiosity, but it may have waned over time as their experience increased. This can result in the investigator using their experience as a crutch to make up for their lack of curiosity. They won’t encounter too many dead end scenarios because of this, but may be more prone to them in new and unfamiliar situations. This person will manipulate data, but will rely on preexisting tools and scripts to do so when possible. They will carefully evaluate the time/reward benefit of their actions and will trust their gut instinct more than anything else. This person’s success in resolving investigations will be defined by the nature of their experience, because they will be significantly less successful in scenarios that don’t relate to that experience. These individuals won’t be as highly motivated in terms of out-of-the-box thinking and may be limited in hypothesis generation.

High C / High E

Because this person has a high level of curiosity they will be more motivated to investigate tangential leads. Because they also possess a high level of experience, they will be more efficient in choosing which leads they follow because they will have a wealth of knowledge to reflect upon. When encountering a dead-end scenario, this person should be able to move past it quickly, or if they claim they’ve hit a true dead end, it’s more likely to be an accurate representation of the truth. This person will excel in hypothesis generation and will provide valuable input to lesser experienced investigators relating to how their time could be best spent. They will seek to perform data manipulation when possible, but will be adept at realizing when to use already available tools and when to create their own. They will realize when they’ve found a data manipulation solution that is good enough, and won’t let perfect be the enemy of good enough. This presents an ideal scenario where the investigator is highly capable of resolving an investigation and doing so in a timely manner. These individuals are ideal candidates for being senior leaders, because they can often effectively guide less experienced investigators regarding what leads are worth pursuing and what the right questions to ask are. This person is always learning and growing, and may have several side projects designed to make your organization better.

Low C / Low E

This presents an undesirable scenario. Not only does this person not have the experience to know what they are looking at, they don’t have enough curiosity to motivate them to perform the necessary research and experimentation needed to learn more. This will handicap their professional growth and have them getting outpaced by their peers with a similar amount of experience.

 

If you are an investigator or have spent time around a lot of them then the descriptions you read in each of these scenarios might remind you of someone you know, or even yourself at different points in your career. It’s important to also consider progression, because the level of curiosity and experience of a person changes throughout their career. In these scenarios, a person always starts with no experience but their level of curiosity may affect how quickly that experience is gained.

 

High Curiosity – Sustained

In this ideal scenario, an investigator learns very quickly, and the rate at which they learn also grows. As they realize there is more to learn, they begin to consume more information in more efficient ways.

 

High Curiosity – Waning

While many start very curious, some experience a waning level of curiosity as their experience grows. When this happens, these investigators will rely more on their experience and their rate of learning will slow.

 

Low Curiosity – Sustained

An investigator with a sustained level of low curiosity will continually learn, but at a very slow rate through their career. Peers with a similar number of years experience will outpace them quickly.

 

Low Curiosity – Growing

If an investigator is able to develop an increased level of curiosity over time, their rate of learning will increase. This can result in dramatic mid to late career growth.

 

Each of these scenarios represents a bit of an extreme case. In truth, the progression of an investigators career is affected by many other factors, and curiosity can often take a back seat to other prevailing forces. Most of us who have served in an investigative capacity also know that curiosity often comes in peaks and valleys as new ideas or technologies are discovered. For instance, new tools like Bro have sparked renewed interest for many in the field of network forensics, while the maturity of memory analysis tools like Volatility have sparked curiosity for many in host-based forensics. A new job or changes in someone’s personal life can also positively or negatively affect curiosity.

Recognizing and Assessing Curiosity

We’ve established that curiosity is a desirable trait, and we’ve reviewed examples of what an investigator possessing varying degrees of curiosity and experience might look like. It’s only logical to consider whether curiosity is a testable characteristic. Several researchers have tackled this problem, and as a result there are different tests that can be used to measure varying degrees of this construct.

Available tests include, but are not limited to, the State-Trait Personality Inventory (Spielberger et al, 1980), the Academic Curiosity Scale (Vidler & Rawan, 1974), and the Melbourne Curiosity Inventory (Naylor, 1981). All of these tests are simple self-reported pencil and paper inventories designed to ask somewhat abstract questions in order to assess different facets of curiosity. Some use likert scales to evaluate whether statements describe them, where as others use agreement/disagreement choices in response to whether specific activities sound interesting. These tests all use different models for curiosity, spanning three, four, and five-factor models. They also all require someone with an understanding of administering personality tests to deliver and interpret the results.

A paper published by Reio, et al (2016) completed a factor analysis study of eleven different test designed to measure facets of curiosity. Their findings confirmed research done other psychologists that supports a three-factor model for curiosity delineated by cognitive, physical thrill seeking, and social thrill seeking components. Of course, the former of those is most interesting in our pursuits.

Psychometrics and personality testing is a very unique field of research. While many tests exist that can measure curiosity to some degree, their delivery, administration, and interpretation isn’t entirely feasible by those outside of the field. Simply choosing which test to administer requires a detailed understanding of test reliability and validity beyond what would be expected in a normal SOC. Of course, there is room for more investigation and research here that might yield simplified versions of these personality inventories that are approachable by technical leaders. This is yet another gap that can be bridged where psychology and information security intersect.

Teaching and Promoting Curiosity

Many believe that there is an aspect of curiosity that is a product of nature, and one that is a product of nurture. That is to say some people are born innately with a higher level of curiosity than others. The nature/nurture debate is one of the most prolific arguments in human history, and it goes well beyond the scope of my this article. However, I think we can stipulate that all humans are born with an innate ability to be curious.

If we know curiosity is important, that humans are born with a capacity for it, and we have models that can assess it, the practical question is whether we can teach it. As the field of cognitive psychology has grown, academics have sought to increase the practical application of research in this manner, incorporating the last hundred years of research on reasoning, memory, learning, and other relevant topics.

Nichols (1963) provides useful insight about scenarios that can inhibit and foster curiosity. He identifies three themes.

 

Theme 1: Temperance

A state of temperance is a state of moderation or restraint. While we usually think that it’s in our best interest to absorb all the information we can in an investigation, this can actually serve to limit curiosity. In short, a hungry man is much more curious than a well-fed one.

I think Nichols says it best, “Intemperance in a work situation is largely a condition we bring upon ourselves by limiting our mental exercise to a narrow span of interest. This is most commonly manifested in an over-attention paid to the details of what we are doing. Once our mind becomes satiated by an abundance of minor facts, we cannot, merely by definition, provide it with new and fresh ideas that will allow us to expand our intellectual perception. Our capacity to do so is restricted by our inability to cram more into a mind that is already overburdened by minutiae. Instead, if we recognize that our responsibility is to focus on the vital few points rather than the trivial many, we will have released ourselves so that we may—as the juggler does—examine these areas from several vantage points and mentally manipulate them in a way that will be both more productive and give greater self-satisfaction (Nichols, 1963, p.4). “

 

Theme 2: Success and Failure

When know this from basic principles of conditioning that humans will use avoidance techniques to prevent experiencing a stimulus that is perceived as negative. Because of this, an investigator who repeatedly attempts to perform the same activity and fails will be dissuaded from pursuing that activity. As we’ve established curiosity as a motivation to fill knowledge gaps, it’s clear to see the correlation between repeated failure and decreased curiosity.

For example, an investigator who has little scripting ability might decide that they would like to write a script to output the contents packet capture file and print all of the DNS queries and responses. If they attempt this multiple times and fail, they will eventually just move on to other methods of inquiry. At this point they are much less likely to pursue this same task again, and worse, are much less likely to attempt to solve similar problems using scripting techniques.

 

Theme 3: Culture

Whenever someone is surrounded by a group of others without any sense of curiosity, it’s likely that their level of curiosity will slow or cease growing at all. Fortunately, the opposite of the previous case is also true, as Nichols noted, “Just as association with a group methodically killing curiosity soon serves to stifle that precious commodity within us, becoming part of a group concerned with intellectual growth stimulates our personal curiosity and growth. This does not mean that each of us must throw over our present job, don a white lab coat, and head for the research and development department. It does mean that we should be discriminating in our choice of attitudinal surroundings both on and off the job. Specifically, it requires that we surround ourselves with doers, with competition that will give us incentive to exercise the creative abilities that grow out of intellectual curiosity. We all have the opportunity to find and benefit from an environment that stimulates our curiosity if we only seek it (Nichols, 1963, p.4).”

 

I’ve written extensively about creating a culture of learning, but there is something to be said for creating a culture of curiosity as a part of that. In a more recent study (Koranda & Sheehan, 2014), a group of researchers concerned with organizational psychology in the advertising field built upon that practical implications of Nichols’ work and designed a course with the goal of promoting curiosity in advertising professionals. This, of course, is another field highly dependent on curiosity for success. While this study stopped short of using one of the aforementioned inventories to measure curiosity before and after the course, the researchers did use less formal surveys to ascertain a distinguishable difference in curiosity for those who had participated in the course.

Based on all these things we can identify impactful techniques that can be employed in the education of computer security investigators encompassing formal education, shorter-term focused training, and on-the-job training. I’ve broken those into three areas:

Environment

• When possible, encourage group interaction and thinking as much as possible. It exposes investigators to others with unique experience and ways of thinking.
• Provide an environment that is rich in learning opportunities. It isn’t enough to expect an investigator to wade through false positive alerts all day and hope they maintain their curiosity. You have to foster it when scenario-based learning that is easily accessible.

Tone

• Encourage challenging the status quo and solving old problems in new ways. This relates directly to data manipulation, writing custom code, and trying new tools.
• Stimulate a hunger for knowledge by creating scenarios that allow investigators to fail fast and without negative repercussions. When an investigator is met with success, make sure they know it. Remember that experience is the thing we get when we don’t get what we wanted.
• Pair lesser experienced investigators with mentors. This reduces the change of repetitive failure and increases positive feedback.

Content

• Tie learning as much as possible to real world scenarios that can be solved in multiple ways. If every scenario is solved in the same way or only provides one option, it limits the benefits of being curious, which will stifle it.
• Create scenarios that are intriguing or mysterious. Just like reading a book, if there isn’t some desire to find out what happens next then the investigator won’t invest time it and won’t be motivated towards curiosity. The best example I can think of here is the great work being done by Counter Hack with Cyber City and the SANS Holiday Hacking Challenges.
• Present exercises that aren’t completely beyond comprehension. This means that scenario difficulty should be appropriately established and paired correctly with the individual skill sets of investigators participating in them.

 

Of course, each of these thoughts presents a unique opportunity for more research, both of a practical and scientific manner. You can’t tell someone to “be more curious” and expect them to just do it any more than you can tell someone “be smarter” and expect that to happen. Curiosity is regulated by a complex array of traits and emotions that aren’t fully understood. Above all else, conditioning applies. If someone is encouraged to be curious and provided with opportunities for it, they will probably trend in that direction. If a person is discouraged or punished for being curious or isn’t provided opportunities to exhibit that characteristic, they will probably shy away from it.

Conclusion

Is curiosity the “X factor” that makes someone good at investigating security incidents? It certainly isn’t the only one, but most would agree that it’s in that conversation and it’s importance can’t be understated.

In this article I discussed the construct of curiosity, why it’s important, how it manifests, and what can be done to measure and promote it. Of course, beyond the literature review and application to our field, many of the things presented here are merely launching points for more research. I look forward to furthering this research myself, and hearing from those who have their own thoughts.

 

References:

Koranda, D., & Sheehan, K. B. (2014). Teaching Curiosity: An Essential Advertising Skill?. Journal Of Advertising Education, 18(1), 14-23

Litman, J. A., & Spielberger, C. D. (2003). Measuring epistemic curiosity and its diversive and specific components. Journal of personality assessment,80(1), 75-86.

Lowenstein, G. (1994). `The Psychology of Curiosity: A Review and Reinterpretation.

Naylor, F. D. (1981). A state-trait curiosity inventory. Australian Psychologist,16(2), 172-183.

Nichols, R. G. (1963). Curiosity – The Key to Concentration. Management Of Personnel Quarterly, 2(1), 23-26.

Reio, T. J., Petrosko, J. M., Wiswell, A. K., & Thongsukmag, J. (2006). The Measurement and Conceptualization of Curiosity. The Journal Of Genetic Psychology: Research And Theory On Human Development, 167(2), 117-135. doi:10.3200/GNTP.167.2.117-135

Vidler, D. C., & Rawan, H. R. (1974). Construct validation of a scale of academic curiosity. Psychological Reports, 35(1), 263-266.