Chris Sanders

I’m sitting in my hotel room after just finishing my last session at US-CERT GFIRST in Nashville, TN. This was my first time at GFIRST both as an attendee and presenter, and I really had a great time. Where I’m originally from in Kentucky isn’t too far from Nashville so I am familiar with the area and the venue choice, the Gaylord Opryland Hotel, is a beautiful facility and top-notch for this kind of conference. I wanted to take a moment to address where people can find the resources for my presentation as well as my thoughts on some of the presentations I had a chance to see and the conference as a whole.

My Presentation

Along with my friend and colleague Jason Smith, we presented a talk on Real World Security Scripting. At a bare minimum, we wanted to share some quick and dirty scripts we wrote to do some pretty neat things within our security operations center (SOC) at SPAWAR. At a higher level, we really hoped that we could encourage some people to get involved with low level BASH, Python, and PERL scripting to automate tasks within their SOC environment as well as increase capabilities of the SOC and its staff. We generated quite a bit of interest, and as a result it looks like several people were turned away because the room was filled to fire code capacity. Our sincere apologies to those who missed to talk. We got some really positive feedback from folks who did make it to the presentation.

As promised, we will be releasing our slides and source code for the presentation. The slides can be downloaded here. As for the source code, we are maintaining the distribution release on https://www.forge.mil, which requires a DOD CAC or ECA certificate to access. I understand that a lot of government folks outside of DOD don’t have access to forge.mil, so we are trying to find another place to host this code where we can control access to only people in the .gov or .mil space. In the meantime, if you would like to get copies of the code, please e-mail me at my mil address (chris.sanders.ctr@nsoc.med.osd.mil) from your mil/gov address and I will get it over to you. We are hoping to get all of that bundled up by next week.

Presentations I Attended

Keynote Panel Discussion – “Unplug to Save”

I started the week on Tuesday by attending the opening ceremony in which there was a panel discussion between several leaders in the government cyber defense community. The panel included Winn Schwartau, Mark Bengel, Doris Gardner, John Linkous, and John Pray, Jr and was moderated by Bobbie Stempfley. If you aren’t familiar with those individuals I’ll leave the Googling to you .

The discussion was centered on the concept of “unplug to save”, focusing on whether it was an acceptable solution to unplug an entity from the Internet in order to prevent a catastrophic event from occurring as a result of a cyber attack. The panel was split and brought up several good points about the interdepencies between certain aspects of government and national defense, namely citing the one that were unknown. Truth be told, sometimes we just don’t know the affect removing certain networks from the Internet would have. I’m of the opinion that in some cases hitting the kill switch is the best policy, but that is only in an extreme and I’m not sure who that authority should be put on. The panel also got into a discussion of the inherently flawed nature of the Internet and the need for an architecture redesign. That was all fine and dandy and I won’t disagree…but until some form of governing body takes on the task of redesigning the fundamental protocols of the Internet and it is taken seriously then this is just a pie in the sky dream.

The only thing that really irked me during the discussion was when one of the panelist mentioned how we could “solve the cyber problem” by hiring the types of hackers who can’t get clearances. It would seem to be that doing such a thing would be a prime way to generate more Bradley Manning-esque cases. Granted, Manning wasn’t a computer security expert by any means, but imagine what someone with his kind of access could do with a bit of hacking knowledge. I’d just asoon we make cyber jobs within the government more attractive to young professionals so that they stay on the straight and narrow instead of the USG resorting to hiring criminals.

Internet Blockades

This talk was presented by Dr. Earl Zmijewski from Renesys and was one of the talks I enjoyed the most. He described several types of Internet censoring, blocking, and filtering techniques used across the world citing recent examples of Egypy, Libya, North Korea, and of course, the great firewall of China. All of his examples had technical data to back them up which really left me with satisfied. Random fact – N. Korea only has 768 public IP addresses.

Using Differential Network Traffic Analysis to Find Non-Signature Threats

This talk was centered on the creation of metadata of layer 7 data on the network. This isn’t entirely a new concept, but its one that most people are just now keying in on. The general idea is that you can strip out only the layer 7 data from HTTP/DNS/EMail streams, index it, and store it so that you can perform analysis on it. The benefit here is that the amount of disk space required for storage of this type of data is much less than storing full PCAP, allowing for more long term analytics. The talk was presented by David Cavuto from Narus, who did describe a few useful analytics I hadn’t though of. For example, collecting the length of HTTP request URIs and performing a standard deviation of those to look for outliers. This could potentially find incredibly long or incredibly short URIs that might be generated by malicious code.

Unfortuantely, being a vendor talk, Mr. Cavuto didn’t provide anything that would help people generate layer 7 metadata, but he did have a product he was selling that would do it. Fortunately, I have some code that will generate this type of metadata from PCAP. I’m going to button that up and release it here at some point…for free

Getting Ahead of Targeted and Zero-Day Malware Using Multiple Concurrent Detection Methodologies

This was, by far, my favorite presentaiton of the week. It was given by Eddie Schwartz, the new CSO at RSA. The talk was centered around investing time in the right areas of analysis. Namely, looking across the data sources that matter and not relying on the IDS to do all the work. Once Mr. Schwartz releases his slides I would recommend checking them out. He is a man who understands intrusion detection and how to make it effective. My favorite part of his talk was something he said a couple of times: Yes, doing it this way is hard. Suck it up. It gets easier.

They Are In Your Network, Now What?

This talk was presented by Joel Esler of Sourcefire. Joel is a really smart guy and a great presenter and he didn’t disappoint. My big take away from this one was his discussion of Razorback, which I really think is going to be one of the next big things in intrusion detection. I think a lot of the crowd missed the point on this. There were a lot of complaints because of the amount of legwork required to integrate the tool, but I think most of those people were overlooking the early stage the tool was in and the potential impact of the community released nuggets and detection plugins. I played with Razorback when it was first released and look forward to digging into it again once some of the setup and configuration pains are eased. I’ve already thought of quite a few nuggets that I could possibly write for it.

Analysis Pipeline: Real-time Flow Processing

I’m a huge fan of SiLK for netflow collection and analysis so I was excited to hear Daniel Ruef from CERT|SEI talk about Analysis Pipeline, a component that adds some cool flexibility to SiLK. Overall, I was really impressed with the capability and am looking forward to playing with the next version when it comes out in a couple of months. I always say that if you aren’t collecting netflow you are missing out on some great data, and SiLK is a great way to start collecting and parsing netflow for free. If you are already using SiLK, please do yourself a favor and look into the free add-on Analysis Pipeline.

Advanced Command and Control Channels

I thought this was an awesome overview of traditional and more advanced C2 channels that malware use. I don’t think anything here was really new, but the way the presentation was broken down was very intuitive and the examples that were given were rock solid. This was given by Neal Keating, a cyber intel analyst with the Department of State.

Final Thoughts

I really enjoyed the conference and honestly consider it one of the best and most relevant conferences for folks in cyber security within the gov/mil space. My only major complaint was that a few vendors managed to sneak their way into speaking and basically giving product sales pitches rather than technical talks. I’m hoping that feedback will make it back to the US-CERT folks and more effort will go into preventing that from happening in the future. I hate showing up to a talk that I hope to learn something from and being drilled with sales junk about products I don’t want. Yes, I’m looking at you General Dynamics and Netezza.

Overall, the staff did a great job of organizing and I’d be happy to have the opportunity to attend and speak at GFIRST 2012 in Atlanta next year.

TL;DR – Real World Security Scripting Presentation Slides – http://chrissanders.org/pub/GFIRST2011-SandersSmith.pdf – Please e-mail me for full code.

Derek Melber wrote a great little article about the top ten security settings to make directly after installing Active Directory. I’d recommend all of these. Our server guys here actually have a very similar procedure they follow when creating a new network.

Read the full article here.

Unfortunately, sniffing packets isn’t always as easy as plugging into an open port and firing up Wireshark. In fact, it is sometimes more difficult to place a packet sniffer on a network’s cabling system than it is to actually analyze the packets. In the grand ole days of packet analysis when everybody used hubs you could plug in and sniff all of the traffic on a network segment. As most of you know now however, the advent of switched networks prevents this. When you plug a sniffer in to a port on a switch, you can only see broadcast traffic and the traffic transmitted and received by your machine. Because of that we have had to come up with a few alternative techniques to getting the traffic we need.

The three most popular techniques for doing this are port mirroring, hubbing out, and ARP cache poisoning. The goal of this article is to give a brief overview of port mirroring and hubbing out, which are very commonly used, and then to give a detailed explanation of ARP cache poisoning, the least well known of the trio.

The Common Techniques

Port Mirroring is probably one of the easiest ways to capture the traffic you are looking for. Also called port spanning, this is a feature available on most managed network switches. This is configurable by accessing the command line or GUI management for the switch the target and sniffer systems are plugged in to and entering commands which mirror the traffic of one port to another. For instance, to capture the traffic of a device plugged in to port 3 on a switch, you could plug your sniffer into port 6 and enter a vendor specific mirroring command that mirrors port 3 to port 6.

Hubbing out is a technique in which you localize the target device and your analyzer system on the same network segment by plugging them directly in to a hub. In order to do this, all you need is an old hub and a few network cables. Simply go to the switch that the target computer resides on and unplug it from the network. Plug the targets network cable, along with the cable for your sniffer, into the hub, and then plug the hub into the network switch. This will put your sniffer and the target machine on the same broadcast domain and allow you to see all of the packets going to and from the target machine, as well as yours. Since this does involve a brief moment of connectivity loss, I do highly recommend letting the user of the target system know that you will be briefly disrupting their connectivity, especially if it is someone in management!

Poisoning the ARP Cache

The ARP protocol was designed out of necessity to facilitate to translation of addresses between the second and third layers of the OSI model.  The second layer, or data-link layer, uses MAC addresses so that hardware devices can communicate to each other directly on a small scale. The third layer, or network layer, uses IP addresses (most commonly) to create large scalable networks that can communicate across the globe. The data link layer deals directly with devices connected together where as the network layer deals with devices that are directly connected AND indirectly connected. Each layer has its own addressing scheme, and they must work together in order to make network communications happen. For this very reason, ARP was created with RFC 826, “An Ethernet Address Resolution Protocol”. I’m not going to go into detail on the whole ARP process here, but I highly recommend reading my Packet School 201 write up on it here in order to better understand this process.

ARP cache poisoning is a more advanced form of tapping into the wire on a switched network. It is commonly used by hackers to send falsely addressed packets to client systems in order to intercept certain traffic or cause denial of service (DoS) attacks on a target, but ARP cache poisoning can still serve as a legitimate way to capture the packets of a target machine on a switched network.

ARP cache poisoning, sometimes referred to as ARP spoofing, is the process of sending ARP messages to an Ethernet switch or router with fake MAC (Layer 2) addresses in order to intercept the traffic of another computer.

Using Cain & Abel 

When attempting to poison the ARP cache, the first step is to download the required tools and collect some necessary information. We’ll use the popular security tool Cain & Abel from Oxid.it (http://www.oxid.it). The installation is pretty straight forward so I won’t go through that here.

Once you have installed the Cain & Abel software, you need to collect some additional information including the IP addresses of your analyzer system, the remote system you wish to capture the traffic from, and the router that the remote system is downstream from.

When you first open Cain & Abel, you will notice a series of tabs near the top of the window. (ARP cache poisoning is only one of a variety of Cain & Abel’s features.) For our purposes, we’ll be working in the Sniffer tab. When you click this tab, you will see an empty table. In order to fill this table you will need to activate the program’s built-in sniffer and scan your network for hosts.

Click the second icon on the toolbar, which resembles a network card. The first time you do this you will be asked to select the interface you wish to sniff. This interface should be the one that is connected to the network you will be performing your ARP cache poisoning on. Once you’ve selected this interface, click OK to activate Cain & Abel’s built-in sniffer. To build a list of available hosts on your network, click the icon that resembles a plus (+) symbol, and click OK.

The once-empty grid should now be filled with a list of all the hosts on your attached network, along with their MAC addresses, IP addresses, and vendor identifying information. This is the list you will work from when setting up your ARP cache poisoning.

At the bottom of the program window, you will see a set of tabs that will take you to other windows under the Sniffer heading. Now that you have built your host list, you will be working from the APR tab. Switch to the APR window by clicking the tab.

Once in the APR window, you are presented with two empty tables: an upper and a lower one. Once you set them up, the upper table will show the devices involved in your ARP cache poisoning, and the lower table will show all communication between your poisoned machines.

Continue setting up your ARP poisoning by clicking the icon resembling the plus (+) symbol on the program’s standard toolbar. The window that appears has two selection columns side by side. On the left side, you will see a list of all available hosts on your network. Click the IP address of the target computer whose traffic you wish to sniff. This will result in the right window showing a list of all hosts in the network, omitting the target machine’s IP address. In the right window, click the IP address of the router that is directly upstream of the target machine, and click OK.

The IP addresses of both devices should now be listed in the upper table in the main application window. To complete the process, click the yellow-and-black radiation symbol on the standard toolbar. This will activate Cain & Abel’s ARP cache poisoning features and allow your analyzing system to be the middleman for all communications between the target system and its upstream router.

You can now fire up your packet sniffer and begin the analysis process. When you are finished capturing traffic, simply click the yellow-and-black radiation symbol again to stop ARP cache poisoning.

A Final Note

As a final note on ARP cache poisoning, you should be very aware of the roles of the systems you implement this process for. For instance, do not use this technique when the target device is something with very high network utilization, such as a fileserver with a 1Gbps link to the network (especially if your analyzer system only provides a 100Mbps link). When you perform this rerouting of traffic, all traffic transmitted and received by the target system must first go through your analyzer system, therefore making your analyzer the bottleneck in the communication process. This can create a DoS-type effect on the machine you are analyzing, which will result in degraded network performance and faulty analysis data.

That is all there really is to ARP cache poisoning. This technique has always proved significantly useful in packet analysis experience and I hope it does in yours as well.