Gaining Technical Experience with Deliberate Practice

When I moved to Georgia I started riding my mountain bike several times a week. Almost every other day I’d pop out of the office before lunch and ride three one-mile circuits on a trail near my house.  Six months after starting, I realized I’d never looked at the data collected by the trip logger on my bike. I thought it’d be really interesting to see how my speed had improved as I biked more.

I was pretty disappointed when I saw the data.

My performance hadn’t increased a bit.

After months of biking the same three miles, I had no noticeable gains in my cycling ability. I wasn’t finishing the ride any faster, and I wasn’t any less tired. What gives?

 

Me failing at life (dramatization).

 

I got angry and decided to spend some time focusing on performance. Hell hath no fury like a geek with the ability to collect data and manipulate independent variables. I looked up YouTube videos about cycling posture and breathing techniques. I also started reviewing my trip log after every ride and setting goals. I wanted the average of my ride times to improve by at least a few seconds every week.

After sticking with this regimen for just another month, the results were in. I had improved my performance by nearly 30%.

Why was I able to accomplish so much in a month after not making any progress in six months?

The answer is that I stopped riding mindlessly and began deliberately practicing.

 

Practice vs. Performance

We mostly think of practice vs. performance in terms of athletes, so let’s stick with that example. In a given basketball game lasting an hour, an active player might get up a dozen two-pointers, a few three-pointers, and a couple of layups. They might make about forty passes and jump for twenty rebounds. They might also run three offensive and defensive schemes twenty times each.

Now let’s compare those stats with a week of hour-long daily practice sessions as I’ve done in the table below.

 

	1 Game	1 Week of Practice
Two pt shots	12	500
Three pt shots	3	500
Layups	2	200
Passes	40	1000
Rebound attempts	20	200
Offensive schemes	3 x 20	5 x 100
Defensive schemes	3 x 20	5 x 100

 

Clearly, what happens in a game performance is a much smaller subset of what is practiced. The purpose of practice is to develop individual skills in preparation for a performance. Practice is a planned, mindful exercise. An entire practice might be devoted to a single skill like shooting, or mastering a skill within a specific scenario, like rebounding in a man-to-man defense.

A performance combines every facet of your practice. Performance is full of surprise and unpredictable. While practice is actively thoughtful, performance generally involves acting out of muscle memory and getting into a zone that many psychologists called “flow”. Experts experience a much stronger state of flow because they’ve developed more muscle memory (both mentally, and physically).

 

The Secret of Practice

In the example I just described, you might notice that the amount of time allotted to practice is much greater than performance. Be careful though — the development of skill isn’t purely a function of time. That’s why my six months of riding trails showed no improvement in my cycling skill. It’s also why you drive a car every day but are probably ill-equipped to steer a race car around Daytona at 200mph.

The secret of building expertise through practice isn’t that experts log more hours of practice. Experts log higher quality practice.

Whether you’re an athlete or an analyst, the characteristics of high-quality practice are the same.

Requirement 1: A clearly defined long-term goal

The goal of practice is to perform well. What does peak performance look like? In sports, this usually means putting up good stats or winning a game. In intellectual pursuits, it might mean arriving at an accurate answer or completing a task quickly. High-quality practice works toward long-term goals.

Requirement 2: An understanding of the component parts of that goal

Performance is made up of multiple skills used in a variety of scenarios. High-quality practice requires that you understand your long-term goal well enough to break it down into these component parts so you can focus on them individually.

Requirement 3: 100% concentration and effort

Performance is all about muscle memory and flow, but those things are established in practice. You practice a skill several times so that when you really need to use it, you can do so quickly. Performance isn’t just about completing a task, it’s about doing it efficiently and effortlessly. To do this, you must be mindful of the task you’re performing and apply all your attention to it. This way, simpler tasks can become automatic and you can devote previous limited working memory resources to understanding other unknowns.

Requirement 4: Immediate and informative feedback

You practice so that you can get the mistakes out of your system. Of course, this requires that you’re able to spot the mistakes in the first place. You have to collect data and establish a feedback loop. Informative feedback is one reason coaching is so important. A coach’s primary role is to help spot the mistakes you’re making and equip you with the tools to overcome them.

Requirement 5: Repetition, reflection, and refinement

When you combine all the elements I’ve mentioned thus far, you get the blueprint for practice. High-quality practice means repeating skills, reflecting on how well you completed the skill, and refining your approach to the skill.  These things must all be deliberate. You have to practice with the goal of getting better. Just going through the motions won’t get you anywhere.

 

Out of Practice

Infosec practitioners stink at deliberate practice.

It’s something that most of us don’t spend any time thinking about. I ask every analyst I meet how they practice their craft. The answer I always get without fail is this:

 

Attacking a VM and/or reviewing the logs generated from the attacks can be an effective practice strategy, but most never follow through with it and even more don’t approach it the right way. If there were a graveyard for VMs built for this purpose but only used once, it’d be overflowing. I don’t want those VM’s to have died in vain, so I’m going to tell you how you can practice smarter.

 

Developing a Practice Plan

If you want to become an expert at anything and accelerate the accumulation of experience you need to deliberately practice it. You have to be strategic about how you focus your practice. I recommend creating a practice plan, which is built from a list of skills used during performance of a job and scenarios where you might encounter them.

In our basketball example, skills include shooting, passing, and rebounding. Scenarios include different schemes like man-to-man defense, dribble-drive offense, and inbounding plays. Multiple skills will be encountered differently based on the scenario where they are needed.

The same thing applies to information security. Consider the work of a malware analyst. Three skills you’ll use during reverse engineering include:

1. Simulating responsive services
2. Identifying imported code libraries
3. Understanding network communication sequences

Those skills manifest differently depending on the situations you’ll encounter. Those situations can be defined a number of ways:

1. Platform: Windows, Mac, or Linux
2. Malware Function: Droppers, worms, exploits, RATs
3. Malware Techniques: Registry persistence, VM detection, heap spraying, opening network listeners, keylogging

Now you have what you need to make a practice plan. You simply combine skills with the scenarios you’ll encounter them in. So, you might wind up practicing the following things:

1. Simulating a DNS server to resolve a domain requested by an iOS dropper
2. Identifying the code libraries imported by Windows malware to understand its purpose
3. Understanding the network communication of a RAT to build IDS signatures

If you practice these things, you’ll be able to do them effortlessly when you’re under the gun to analyze a real malware sample you’ve found on your network.

This example is focused on malware analysis, but it can easily be applied to red teaming, alert review, threat hunting, web application development, socket programming, or just about any technical skill you can think of. You just need to clearly identify the skills and situations associated with the job.

 

Challenge

Now that you know about deliberate practice, I’m challenging you to take action. I want you to examine a facet of your job you want to get better at and break it down like I just did. Figure out the skills you need to be good at the job, and the scenarios where you’ll use those skills. Make a list of both and reply to the comments on this post with your practice plan.