Collecting Threat Intelligence
February 5th, 2011
No comments
One of the more important skills in intrusion detection and analysis is the ability to evaluate an IP address or domain name in order to build an intelligence profile on that host. Gathering this intelligence can help guide you to making more informed decisions regarding the remote hosts that are communicating with your network in order to determine if they are of a malicious or hostile nature. I recently wrote a two-part article on collecting threat intelligence for WindowsSecurity.com which describe some methods that can be used to collect threat intelligence on a host or network.
Categories: Intrusion Detection, Network Security analysis, detection, intelligence, intrusion, Intrusion Detection, threat
The most important thing an analyst can have ingrained into them in their importance. An analyst is the first line of defense. The analyst is sitting in the crows nest watching for the icebergs. It is the analyst who can keep attacks from happening and can stop attacks from getting worse. Most security incidents begin with an analyst providing a tip based upon an IDS alert and end with an analyst putting in new signatures and developing new tools based up on intelligence gained from a declared incident. The analyst is the beginning and the end in information security. The alpha and omega. Okay, maybe that’s a bit dramatic, but the importance of an intrusion analyst can’t be understated.
Analysis happens in a world of assumptions and its important to remember that. Most of the decisions you will make are centered around a packet or a log entry and then honed based upon intelligence gathered through research. 
The fact is that the analyst isn’t the one who gen
erated the traffic, so every decision you will make is based upon an assumption. Don’t worry though; there is nothing wrong with that. Ask your friendly neighborhood chemist or physicist. Most of their work is based upon assumptions and they have great success. The takeaway here is that there are no absolutes. Is that IP address REALLY a known legitimate host? Does that domain REALLY belong to XYZ company? Is that DNS server REALLY supposed to be talking to that database server? There are no absolutes, merely assumptions, and because of that remember that assumptions can change. Always question yourself and stay on your toes.
An analyst depends on data to perform their function. This data can come in the form of a PCAP file, an IIS log file, or SYSLOG file. Since most of your time will be spent using various tools to interact with data it’s crucial to be mindful of how that tool interacts with the data. Did you know that if you run Tcpdump without specifying otherwise, it will only capture the first 68 bytes of data in a packet? How about that Wireshark displays sequence and acknowledgement numbers within TCP packets in a relative manner by default? Tools are made by people and sometimes “features” can cloud data and prevent proper analysis. I think both of the features I described earlier are great, but I’m also mindful that they exist so I can see all of the packet data available or view the real sequence and acknowledgement numbers when needed. In a job where reliance upon data is critical, you can’t afford to not understand exactly how tools interact with that data.
This is to be expected because different people come from different backgrounds. I work with the government so the first thing I look at when examining network traffic is the source and destination country. I’ve worked with people who have systems administration backgrounds and as a result, will look at the port number of the traffic first. I’ve even worked with people who have a number crunching background who will look at the packet size first. This demonstrates that our experiences shape our tactics a bit differently. This means that the numbers guy might see something that the sysadmin didn’t see or that the government guy might have insight that the numbers guy didn’t. Whenever possible it’s always a good idea to have a second set of eyes look at the issue you are facing.
This is something I’ve believed since the first day I ever fired up a Snort sensor, but IDS guru Mike Poor phrased it best while I was attending one of his SANS classes when he said that you should never invite an attacker to dance. As an analyst its very tempting to want to investigate a hostile IP address a bit beyond conventional means. Trust me, there have been many occasions where I’ve been tempted to port scan a hostile that kept sending me painfully obviously crafted UDP packets. Even more so, any time someone attempts to DOS a network I’m responsible for defending, I wish nothing more than to be able to unleash the full fury of a /8 network on their poor unsuspecting DSL connection. The problem with this is that 99% of the time we don’t know who or what we are dealing with. Although you may just be seeing scanning activity, the host that is originating the traffic could be operated by a large group or even a military division of another country. Even something as simple as a ping could tip off an attacker that you know they exist, prompting them to change their tactics, change source hosts, or even amplify their efforts. You don’t know who you are dealing with, what their motivation is, and what there capabilities are, so you should never invite them to dance.
Network diagrams, listings of servers and their roles, breakdowns of IP address allocations, and more can be your best friend. Basically any and everything that can be used to document the assets within the network, how they function, and how they relate to other assets are beneficial in running down anomalous events. Depending upon your role in the organization you may not be in a position to obtain these things and if they don’t already exist you are going to have a heck of a time getting the systems folks to put in the leg work to create them. However, as difficult as this may be, its an effort that’s worth pursuing. Whether you have to present your case to the CIO or just buy your network engineers a case of their favorite adult beverage its ultimately worth the effort.
The ultimate argument in life is whether or not people are inherently good or inherently evil. This same argument can be had for packets as well. You can either be the analyst that believes all packets are inherently evil or the analyst that believes all packets are inherently good. I’ve noticed that most analysts typically start their career as for the former and quickly progress the later. That’s because its simply not feasible to approach every single piece of traffic as something that could be a potential root level compromise. If you do this, you’ll eventually get fired because you spent your entire day running down a single alert or you’ll just get burnt out. There is something to be said for being thorough but the fact of the matter is that most of the traffic that occurs on a network isn’t going to be evil, and as such, packets should be treated innocent until proven guilty.
An effective analyst has to understand that while different tools may be the most important part of the job, those things are merely pieces of the puzzle. Just like an astronomer’s telescope is just another tool in his arsenal that allows him to figure out what makes the planets orbit the sun, Wireshark is just another tool in an analysts arsenal that allows him to figure out what makes a packet bypass a firewall rule. Start with the science, add in a few tools and processes, stay cognizant of the big picture, keep an attention to detail, and eventually the combination of all of those things and the experience you gain over time will help you develop your own analysis philosophy. It’s at that point you have taken your analysis to the level of an art, and made it so that your worthy enough to not be replaced by a machine.
No matter how hard you try there will come a point in which the network you are defending gets successfully attacked and compromised. In the modern security landscape its inevitable and there isn’t a lot you can do about it. In these times its likely that the analyst will take the heat over the incident. Because of this, you need to be prepared when it happens. An incident won’t be remembered for how an intrusion occurred, but rather how it was responded to, the amount of downtime that occurred, the amount of information that was lost, and ultimately the amount of money it costs the organization. What recommendations can you make to management to ensure a similar incident doesn’t occur again? What can you show your superiors to explain why the attack wasn’t detected? What shortcomings do your tools have? These are questions that can’t fully be answered until an intrusion has occurred and you have the context of an attack, but you can definitely consider the questions now and have a plan for how your information will be presented to key figures. You will get caught off guard and you will be blind sided, but its important that you don’t appear as such and you keep your game face on. This can make the difference between a promotion and a pink slip.
A defender has to control 65,535 ports. An attacker has to compromise one. A defender has to protect 10,000 users. An attacker has to deceive one. A defender has to examine millions of packets. An attacker has to hide a malicious payload in one. What can you do to increase your visibility into the data? What proficiency can you develop that gives you that edge against the attacker? You have a hunch that there is more than meets the eye, so what can you do to dig deeper?
