Chris Sanders

There are a lot of things that the industry does well when it comes to network security monitoring (NSM). For instance, I tend to think that we have data collection figured out reasonably well. I also think that signature-based intrusion detection is a really well developed science. However, with NSM having only existed for a short period of time there are several facets of it that aren’t too well defined. One such aspect is the actual diagnostic method that people use to analyze NSM events. That is, the process an analyst uses to connect the dots between the initial alert and the final diagnosis. In this article I’m going to discuss the use of a common medical diagnostic method called differential diagnosis and how it can be applied to NSM.

 

Understanding Normal

The first thing that was ever taught to me when I started my career as an NSM analyst was that if you know what normal looks like, then you can determine what is bad. I trusted in this concept for many years and even taught it to others. As true as this statement may be, I believe it is relied on entirely too much. This is primarily due to a failure in separating the collection, detection, and analysis processes.

 

Collection centers on the hardware and software used to collect NSM related data. Consider the collection of full content packet capture (PCAP) data. The use a network tap and DaemonLogger allow you to store this data on disk so that it may be used for the identification and analysis of network security related events. Collection occurs with a combination of hardware and software.

Detection is the process by which collected data is examined and anomalies are identified, typically through some form of signature, anomaly, or statistically based detection. Snort is software that is an example of signature-based intrusion detection that compares collected network traffic to signatures of known malicious activity in an effort to perform pattern matching to determine if something bad has occurred. Detection is typically software focused.

Analysis is what occurs when a human interprets the results of the output of an identification tool. Although Snort may detect a pattern match in a communication sequence and generate an alert, it is a human who is ultimately responsible for reviewing the alert and investigating it to an end determination on its validity. The key concept here is that analysis is human focused.

 

With those three terms more clearly defined and distinctions drawn, it would stand to reason that the concept of knowing what normal looks like in order to determine what is bad is actually more relevant to detection than analysis. Realistically speaking, it’s not feasible in the modern state of network computing to be well versed in every aspect of normal communications. Although some traffic patterns may remain fairly static, the open nature and loose standards that govern network communication protocols result in a constant evolution of traffic patterns. Don’t be mistaken, this is still an important concept that must be incorporated into the analytic approach, it’s just not strong enough to stand on its own as the singular concept new analysts should be taught. Knowing what normal looks like is best used when analyzing specific facets of a potential breach rather than as a holistic method to classify all network traffic you may be capturing.

 

A Differential Approach

The general goal of an NSM analyst is to digest the alerts generated by various detection tools and investigate multiple data sources and perform relevant tests and research to see if their findings represent a network security breach. This is very similar to that of a physician, whose goal is to digest the symptoms a human presents and investigate multiple data sources and perform relevant tests and research to see if their findings represent a breach in the person’s immune system.  Both practitioners share a similar of goal of connecting the dots to find out if something bad has happened and/or is still happening.

Although NSM has only been around a short while, medicine has been around for centuries. This means that they’ve got a head start on us when it comes to developing their diagnostic method. One of the most common diagnostic methods used in clinical medicine is one called differential diagnosis. If you’ve ever seen an episode of “House” then chances are you’ve seen this process in action. The group of doctors will be presented with a set of symptoms and they will create a list of potential diagnosis on a whiteboard. The remainder of the show is spent doing research and performing various tests to eliminate each of these potential conclusions until only one is left. Although the methods used in the show are often a bit unconventional they still fit the bill as a part of the differential diagnosis process.

The differential method is one based upon a process of elimination. It consists of five distinct steps, although in some cases only two will be necessary. The differential process exists as follows:

1. Identify and list the symptoms
   
   In medicine, symptoms are typically initially conveyed verbally by the individual experiencing them. In NSM, a symptom is most commonly in the form of an alert generated by some form of intrusion detection system or other detection software. Although this step focuses primarily on the initial symptoms, more symptoms may be added to this list as additional tests or investigations are conducted.

 

2. Consider and evaluate the most common diagnosis first
   
   A statement every medical student is taught in their first year is “If you hear hoof beats, look for horses…not zebras.” This is to state to that the most common diagnosis is likely the correct one. As a result, this diagnosis should be evaluated first. The analyst should focus his investigation on doing what is necessary to quickly confirm this diagnosis. If this common diagnosis cannot be determined to be true during this initial step then the analyst should proceed to the next step.

 

3. List all possible diagnosis for the given symptoms
   
   The next step in the differential process is to list every possible diagnosis based upon the information currently available with the initially assessed symptoms. This step requires some creative thinking is often most successful when multiple analysts participate in generating ideas. Although you may not have been able to completely confirm the most common diagnosis in the previous step, if you weren’t able to rule it out completely then it should be carried over into the list generated in this step. Each potential diagnosis on this list is referred to as a candidate condition.

 

4. Prioritize the list of candidate conditions by their severity
   
   Once a list of candidate conditions is created a physician will prioritize these listing the condition that is the largest threat to human life at the top. In the case of an NSM analyst you should also prioritize this list, but the prioritization should focus on which condition is the biggest threat to your organizations network security. This will be highly dependent upon the nature of your organization. For instance, if “MySQL Database Root Compromise” is a candidate condition then a company whose databases contains social security numbers would prioritize this condition much higher than a company who uses a simple database to store a list of its sales staffs on-call schedule.

 

5. Eliminate the candidate condition, starting with the most severe
   
   The final step is where the majority of the action occurs. Based upon the prioritized list created in the previous step the analyst should begin doing what is necessary to eliminate candidate conditions, starting with the condition that poses the greatest threat to network security. This process of elimination requires considering each candidate condition and performing tests, conducting research, and investigating other data sources in an effort to rule them out as a possibility. In some cases investigation on one candidate condition may effectively rule out multiple candidate condition, speeding up this process. Alternatively, investigation of other candidate conditions may prove inconclusive leaving one or two conditions that are unable to be definitively eliminated as possibilities. This is acceptable however as sometimes in network security monitoring (as in medicine) there are anomalies that can’t be explained that require more observation before determining a diagnosis. Ultimately, the goal of this final step is to be left with one diagnosis so that either the incident handling process may begin or the alert can be dismissed as a false positive. It’s very important to remember that “Normal Communication” is a perfectly acceptable diagnosis, and will be the most common diagnosis an NSM analyst arrives at. I also find that remembering that all packets are good unless you can prove they are bad is an important concept to remember during this step.

 

 

Let’s consider this process with a couple of broad case scenarios.

 

Scenario 1

Step 1: Identify and List the Symptoms

Symptoms:

• Internal host appears to be sending outbound traffic to a Russian IP address
• The traffic is occurring at regular intervals, every 10 minutes
• The traffic is HTTPS over port 443, and as such is encrypted and unreadable

Step 2: Consider and Evaluate the Most Common Diagnosis First

It’s been my experience that most entry level analysts will see these symptoms and automatically think that this machine is infected with some form of malware and is phoning home for further instructions. Those analysts tend to key in on that fact that the traffic is going to a Russian IP address and that it is occurring at regular 10 minute intervals. Although those things are worth noting (I wouldn’t have listed them if they weren’t), I don’t buy into the malware theory so easily. I believe entirely too much emphasis is placed on the geographic location of IP addresses, so the fact that the remote IP address is Russian means little to me. Additionally, there are a whole variety of normal communication mechanisms that talk on regular periodic intervals. This includes things like web-based chat, RSS feeds, web-based e-mail, stock tickers, software update processes, and more. Operating on the principal that all packets are good unless you can prove they are bad, I think the most common diagnosis here is that this is normal traffic.

That said, how we can confirm this potential diagnosis? Confirming something is normal can be hard. In this particular instance we could start with some open source research on the Russian IP. Although it’s located in Russia it still may be owned by a legitimate company. If we were to look up the host and find that it was registered to a popular AV vendor we might be able to use that information to conclude that this was an AV application checking for updates. I didn’t mention the URL that the HTTPS traffic is going to, but quickly Googling it may yield some useful information that will help you determine if it is a legitimate site or something that might be hosting malware or some type of botnet C2. Another technique would be to examine the host physically if you have ready access to it in an effort to see if any processes are launched on the machine at the same intervals the traffic is occurring at.

Let’s assume that we weren’t able to make a final determination on whether or not this was normal communication.

Step 3: List all Possible Diagnosis for the Given Symptoms

*There are obviously more candidate conditions in the realm of possibility, but for this and the other scenario I’ve kept it to some of the more common ones for the sake of brevity.

Candidate Conditions:

• Normal Communication
  We weren’t able to rule this out completely in the previous step so we carry it over to this step.

 

• Malware Infection / Installed Malicious Logic
  This is used as a broad category. We typically don’t care about the specific strain until we determine that malware may actually exist. If you are concerned about a specific strain then it can be listed separately. Think of this category as a doctor listing “bacterial infection” as a candidate condition knowing that they can further narrow it down later.

 

• Data Exfiltration from Compromised Host
  Potential that the host could be sending proprietary or confidential information out. This sort of thing would likely be part of a coordinated or targeted attack.

 

• Misconfiguration
  It’s well within the realm of possibilities that a system administrator fat-fingered an IP address and a piece of software that should be trying to communicate periodically with an internal IP is now trying to do so with a Russian IP. This is really quite common.

 

Step 4: Prioritize the List of Candidate Conditions by their Severity

These priorities are fairly generalized since they are dependent upon your organization.

Priority 1: Data Exfiltration from Compromised Host

Priority 2: Malware Infection / Installed Malicious Logic

Priority 3: Misconfiguration

Priority 4: Normal Communication

Step 5: Eliminate the Candidate Conditions, Starting with the Most Severe

Priority 1: Data Exfiltration from Compromised Host

This one can be a bit tricky to eliminate as a possibility. Full packet capture won’t be of the most assistance here since the traffic is encrypted, but if you can create some statistics from this traffic, or better yet, if you have netflow available, you should be able to determine the amount of data going out. If only a few bytes are going out every then minutes than it’s likely that this is not data exfiltration. The host based research you did earlier on the Russian IP address may also provide some value here in determining the reputation of this host. It would also be of value to determine if any other hosts on your network are talking to this IP address or any other IPs in the same address space. Finally, baselining normal communication for your internal host and comparing it with the potentially malicious traffic may provide some useful insight.

Priority 2: Malware Infection / Installed Malicious Logic

At this point the research you’ve already done should give you a really good idea on whether or not this condition is true. It will be likely that by examining the potential for data exfiltration you will rule this condition out as a result, or will have already been able to confirm it to be true.

Priority 3: Misconfiguration

This condition can best be approached by comparing the traffic of this host against the traffic of one or more hosts with a similar role on the network. If every other workstation on that same subnet has the same traffic pattern, but to a different IP address, then it’s likely that the wrong IP address was entered into a piece of software somewhere proving that a misconfiguration exists. Having access to host-based logs can also be useful in figuring out if a misconfiguration exists since they might exist in Windows or Unix system logs.

Priority 4: Normal Communication

If you’ve gotten this far, then the diagnosis of normal communication should be all that remains on your list of candidate conditions.

Concluding a Diagnosis

At this point you have to use your experience as an analyst and your intuition to decide if you think something malicious is really occurring. If you were able to complete the previous analysis thoroughly, then operating on the assumption that all packets are good unless you can prove they are bad would mean your final diagnosis here should be that this is normal communication. If you still have a hunch something quirky is happening though, there is no shame in monitoring the host further and reassessing once more data has been collected.

 

Scenario 2

Step 1: Identify and List the Symptoms

Symptoms:

• A web server in our DMZ is receiving massive amounts of inbound traffic
• The inbound traffic is unreadable and potentially encrypted or obfuscated
• The inbound traffic is coming to multiple destination ports on the internal host
• The inbound traffic is UDP based

Step 2: Consider and Evaluate the Most Common Diagnosis First

With the amount of traffic being received by the internal host being very large and the packets using the UDP protocol with random destination ports, my inclination would be that this is some form of denial of service attack.

The quickest way to determine whether something is a denial of service is to assess the amount of traffic being received compared with the normal amount of traffic received on that host. This is something that is really easy to do with netflow data if you have it available. If the host is only receiving 20% more traffic than it normally would then I would consider other alternatives to a DoS. However, if the host is receiving ten or one hundred times its normal amount of traffic then DoS is very likely and almost a certainty.  It’s important to remember that a DoS is still a DoS even if it is unintentional.

Once again, for the sake of this scenario we will continue as though we weren’t able to make a clear determination on whether or not a DoS condition exists.

Step 3: List all Possible Diagnosis for the Given Symptoms

Candidate Conditions:

• Denial of Service
  We weren’t able to rule this out completely in the previous step so we carry it over to this step.

 

• Normal Communication
  It doesn’t seem incredibly likely, but there is potential for this to be normal.

 

• Misdirected Attacks
  When a third party chooses to attack another they will often spoof their source address for the sake of anonymity and to prevent getting DoS’d themselves. This will result in the owner of the spoofed IP they are using seeing that traffic. This web server could be seeing the effects of this.

 

• Misconfigured External Host
  A misconfiguration can happen on somebody else’s network just as easily as it could on yours. This misconfiguration could result in an external host generating any number of types of traffic and sending them to the web server.

 

• SPAM Mail Relay
  The server could be misconfigured or compromised in a manner that allows it to be used for relaying SPAM across the Internet.

 

Step 4: Prioritize the List of Candidate Conditions by their Severity

Priority 1: Denial of Service

Priority 2: SPAM Mail Relay

Priority 3: Misconfigured External Host

Priority 4: Misdirected Attacks

Priority 5: Normal Communication

Step 5: Eliminate the Candidate Conditions, Starting with the Most Severe

Priority 1: Denial of Service

We’ve already gone through the paces on this one without being able to identify that it is the definitive diagnosis. Even though this is the most severe we would have to proceed to attempt to eliminate other candidate conditions to help in figuring out if a DoS is occurring. Of course, depending on the effect of the attack it may make the most sense to contain the issue by blocking the traffic before spending more time investigating the root cause.

Priority 2: SPAM Mail Relay

This one is relatively easy to eliminate. If the server was being used as a mail relay then you would have a proportionate amount of traffic going out as you do going in. If that’s not the case and you don’t see any abnormal traffic leaving the server then it is likely that it is not relaying SPAM. If the web server is also running mail services then you can examine the appropriate logs here as well. If it is not supposed to be running mail services you can examine the host to see if it is doing so in an unauthorized manner.

Priority 3: Misconfigured External Host

This one is typically pretty tricky. Unless you can identify the owner of the IP address and communicate with them directly then the most you can hope to do is block the traffic locally and/or report abuse at their ISP level.

Priority 4: Misdirected Attacks

This is another tricky one along the same lines as the previous candidate condition. If it’s an attacker somewhere else whose antics are causing traffic redirection to your server then the most you can do is to report the issue to the ISP responsible for the IP address and block the traffic locally.

Priority 5: Normal Communication

This doesn’t seem likely, but you can’t say this for sure without baselining the normal traffic for the host. Compare its traffic at similar times on previous days to see if you can draw any conclusions. Is the pattern normal and it’s just the amount of traffic that anomalous? Is it both the pattern and the amount that’s anomalous? Does the server ever talk to the offending IP prior to this?

 

Concluding a Diagnosis

In this scenario, it’s very possible that you are left with as many as three candidate conditions that you cannot rule out. The good thing here is that even though you can’t rule these out, the containment and remediation methods would be the same for all of them so you still have gotten to a state of diagnosis that allows the network to recover from whatever is occurring. If the amount of traffic isn’t too great then you may not need to block the activity and you may be able to monitor it further in order to attempt to collect more symptoms that may be useful in providing a more accurate diagnosis.

 

Conclusion

I’ve spent quite a bit of time doing analysis with this differential approach and also reviewing previous investigations post-mortem while applying these concepts and I’ve been really pleased with my findings. I think that if you are struggling with being able to grasp a firm analytical method then this may be a great one to start with. I’m not entirely sure that the differential method is appropriate for all organizations, but just as with medicine, there are competing approaches and I hope to examine more of those in the future so that I can draw more comparisons between the medical field and NSM. If you have any scenarios in which you’ve used this differential approach (for better or for worse), I’d love to hear about them.

I’ve been a huge fan of CloudShark ever since it was launched by QA Café back in 2010. I even wrote about CloudShark here when it was first released. I’m always finding unique uses for the web-based service. This includes everything from sharing PCAP files on my blog to viewing PCAPs on my iPad. CloudShark has the potential to make life much better anywhere you don’t have Wireshark but you do have an Internet connection.

 

The only real downside to CloudShark is that there is no guarantee of privacy for uploaded PCAP files. I recently spoke with Joe at QA Café and this is an issue they are aware of, and as a result they’ve developed the CloudShark appliance. This is a standalone instance of CloudShark that you can deploy within your organization so that you can get all of the benefits of CloudShark while keeping sensitive data contained within your packet captures private to only those who need access.

 

I wanted to briefly mention a few of my favorite CloudShark features:

 

View

This may sound a bit odd, but my absolutely favorite thing about CloudShark is the ability for me to view PCAP files on my iPad. Quite honestly, I’ve wanted an iPad for a while but could never talk myself into buying one until I figured out I could use it to view PCAP files. Now I use it for that purpose on a daily basis. CloudShark is compatible with a variety of smartphones and tablets.

 

 

Annotate

I work in a network security monitoring environment so we look at a LOT of PCAP data. Typically, this involves keeping track of a lot of notebooks where I’ve scribbled notes about the contents of capture files. One of the cooler things about CloudShark is the ability to annotate within the capture files. You can even use these annotations to link to other captures.

 

 

Share

I hate sending PCAP files around via e-mail. People rename things, filter things out and then save them, and do other weird things that may result in multiple people looking at different data thinking they may be looking at the same thing. When you upload a PCAP into CloudShark it generates a hyperlink that you can use to share your PCAPs.

 

 

Organize

When you deal with a lot of PCAP files it’s easy to lose track of what you are looking for. CloudShark has a very “Web 2.0” system for tagging packet captures so that they can be easily searched. This feature can be used for organizing in a lot of way, so you can organize your PCAPs by tagged by protocol, vulnerability, system, or even investigation case.

 

 

There are a lot of different use cases for the CloudShark appliance depending on the needs of your organization. It can be purchased as hardware with the platform pre-installed on to it, or as software that you can install on your own hardware running a Linux based OS. I saw recently that it was also updated to allow for integration with external LDAP authentication, so that adds some more flexibility to the system overall.

 

You can read more about it at http://appliance.cloudshark.org/.

 

One of the more useful network forensic skills is the ability to extract files from packet captures. This process, known as packet data carving, is crucial when you want to analyze malware or other artifacts of compromise that are transferred across the network. That said, packet data carving has varying degrees of difficulty depending on the type of traffic you are attempting to extract data from. Carving files from simple protocols like HTTP and FTP is something that can be done in a matter of minutes and is usually cut and dry enough that it can be done in an automated fashion with tools like Foremost and Network Miner.

There are articles all over the Internet about carving files from simple protocols so I won’t rehash those. Instead, I want to take a look at a two more complex protocols that are extremely common in production networks. Server Message Block (SMB) is the application-layer protocol that Microsoft operating systems use for file sharing and communication between networked devices. If you live on a Microsoft network (or a Unix network that utilizes SAMBA) then you are a user of SMB or SMB2, depending on your operating system version. In this article I’m going to discuss the art of carving files from SMB and SMB2 traffic. If you want to follow along you’ll need to download a copy of Wireshark (http://www.wireshark.org) and your favorite hex editor. I’ve used Cygnus Hex Editor (http://www.softcircuits.com/cygnus/fe/) for the purpose of this article since it’s simple and a free version exists.

 

Carving Files from SMB Packets

 

The first version of SMB is in use on all modern Microsoft operating systems prior to Windows Vista. In order to setup a packet capture for this scenario I took two Windows XP SP3 virtual machines running on VMWare Workstation and placed them in the same network. Once they were able to communicate with each other I setup a shared folder on one host (192.168.47.132) that is acting as the server. I then fired up Wireshark and began capturing packets as I copied an executable file from the client (192.168.47.133) to the servers shared folder. The resulting packet capture is called smb_puttyexe_xfer.pcap.

If you’ve never looked at SMB traffic then don’t get scared by all the different types of SMB packets in the capture, we will only be looking at a few of them. This article isn’t meant to be an exhaustive reference on each and every type of SMB packet (there are over a hundred of them), so if you want the gory details then take a look at the references at this end of this article.

In order to carve the file out of these packets we have to find some basic information about it. Before and after transferring a file to a server the client will attempt to open the file in order to see if it exists. This is done with an SMB NT Create AndX Request packet.  The response from the server to this is an SMB NT Create AndX Response, which contains the name, extension, and size of the file being transferred. This is everything we need to get started. You can filter for Create AndX Response packets in Wireshark with the filter (smb.cmd == 0xa2) && (smb.flags.response == 1). If we examine one of those requests that occur after the file has been transferred, we can identify that the file being transferred is putty.exe and its file size is 454,657 bytes. We will use this information later.

Figure 1: Note the file name, extension, and size.

The next step we have to take in order to extract this file is to isolate the appropriate block of traffic. Wireshark makes this pretty easy with its Follow TCP Stream functionality. Start by right-clicking any packet in the capture file and selecting Follow TCP Stream. This will bring up a window that contains all of the data being transferred in this particular communication stream concatenated together without all of the layer 2-4 headers getting in the way. We are only concerned about the traffic transferred from the client to the server so we will need to specify this in the directional drop down box by selecting 192.168.47.133 –> 192.168.47.132 (458592 bytes). Click Save As and save the file using the name putty.raw.

Figure 2: Saving the isolated traffic from Wireshark

If you were to view the properties of the data you just extracted and save you should find that its file size is 458,592 bytes. This is 3,935 bytes more than the size of the actual file that was transferred. This means that our goal is to get this raw files size down to exactly 454,657 bytes. This is where the real carving begins.

First things first, we have to delete all of the extra data that occurs before the executable data actually begins. Since we do know that the transferred file is an executable the quickest way to do this is to look for the executable header and delete everything that occurs before it. The executable header begins with the hex bytes 4D 5A (MZ in ASCII), which occurs approximately 1112 bytes into the putty.raw file. Once deleted, resave the file as putty.stage1. You should now be down to a file size of 457,480 bytes.

Figure 3: Removing added bytes from the beginning of the file

Now things get a bit trickier. SMB transmits data in blocks. This is great for reliability since a lost or damaged block can be retransmitted, but it adds some extra work for us. This is because each block must contain some bytes of SMB header data in order to be interpreted correctly by the host that is receiving it. The good thing is that the size of this data is somewhat predictable, but you have to understand a bit more about SMB in order to put the rubber to the road. The thing to know here is that the data block size in SMB is limited to 64KB, or 65536 bytes.  Of this amount, only 60KB is typically used for each block. These 61,440 bytes are combined with an additional 68 bytes of SMB header information. This means that after every 61,440 bytes of data we will have to strip out the next 68 bytes.

There is one thing to add to this that must be taken into consideration before stripping out those bytes. As a part of the normal SMB communication sequence, an additional packet is sent right after the first block. This is an NT Trans Request packet, which is packet 77 in the capture file. The SMB portion of this packet is 88 bytes, which means we will have to remove those 88 bytes in addition to the 68 bytes that make up the normal SMB block header, for a total of 156 bytes.

Now that we have all that sorted out let’s start removing bytes. In your hex editor, skip one byte past the 61,440^th byte. This will be offset 0x0F000. You should start with this byte and select a range of 156 bytes and delete them. Save this file as putty.stage2.

Figure 4: Removing the initial 156 bytes

Things get a bit easier now as we are just concerned with stripping out the 68 bytes after every block. Skip through the file in 61,440 byte increments deleting 68 bytes each time. This should occur X times in this file at offsets 0x1e000, 0x2d000, and 0x3c000, 0x4b000, 0x5a000, 0×69000. Once finished, save the file as putty.stage3.

Figure 5: Removing a 68 byte SMB header block

Go ahead and take a look at the file size of putty.stage3. We are still XXX bytes off from our target, but luckily the last part is the easiest. The data stream is actually just padded by some extra information that needs to be deleted. We know that the file should be 454,657 bytes, so browse to that byte and delete everything that occurs after it.

Figure 6: Trimming the extra bytes off the end of the file

Save the final product as putty.exe and if you did everything right, you should have a fully functioning executable.

Figure 7: Success! The executable runs!

 

The whole process can be broken down into a series of repeatable steps:

1. Record the file name, extension, and size by examining one of the SMB NT Create AndX Response packets
2. Isolate and extract the appropriate stream data from Wireshark by using the Follow TCP Stream feature and selecting the appropriate direction of traffic
3.  Remove all of the bytes occurring before the actual file header using a hex editor
4. Following the first 61,440 byte block, remove 156 bytes
5. Following each successive 61,440 byte block, remove 68 bytes
6. Trim the remaining bytes off of the file so that it matches the file size recorded in step 1

 

Carving Files from SMB2 Packets

 

Microsoft introduced SMB2 with Windows Vista and began using it with its newer operating systems moving forward. In order to setup a packet capture for this scenario I took two Windows 7 (x32) virtual machines running on VMWare Workstation and placed them in the same network. Once they were able to communicate with each other I setup a shared folder on one host (192.168.47.128) that is acting as the server. I then fired up Wireshark and began capturing packets as I copied an executable file from the client (192.168.47.129) to the servers shared folder. The resulting packet capture is called smb2_puttyexe_xfer.pcap.

You should notice that this traffic is a little bit cleaner than the SMB traffic we looked at earlier. This is because SMB2 is optimized so that there are a lot less commands. Whereas SMB had over a hundred commands and subcommands, SMB2 only has nineteen. Regardless, we still need to find the filename being transferred and the size of that file. One of the best places to do this is at one of the SMB2 Create Response File packets. This packet type serves a purpose similar to that of the SMB NT Create AndX Response packet. You can filter these out in Wireshark with the filter (smb2.cmd == 5) && (smb2.flags.response == 1). The last one of these in the capture, which is packet 81, is the one we want to look at since it occurs after the file transfer is complete. This identifies the file name as putty.exe and the file size as 454,656 bytes. This is indeed the same file as our earlier example, but it is being reported as being one byte smaller. The missing byte is just padding at the end of the file and has a null value so it’s not of any real concern to us.

Figure 8:  Once again we note the file name, extension, and size

At this point you should perform the same steps as we did earlier to isolate and extract the data stream from the capture using Wiresharks Follow TCP Stream option. Doing this should yield a new putty.raw file whose file size is 459,503 bytes. This is 4,847 too big, so it’s time to get to carving.

Once again we need to start by stripping out all of the data before the executable header. Fire up your favorite hex editor and remove everything before the bytes 4D 5A. This should account for a deletion of 1,493 bytes.

Figure 9: Removing the extra bytes found prior to the executable header

Now things change a bit. SMB2 works in a method similar to SMB, but it actually allows for more data to be transferred at once. SMB had a maximum block size of 64K because it has a limit of 16-bit data sizes. SMB2 uses either 32-bit or 64-bit data sizes, which raises the 64KB limit. In the case of the transfer taking place in the sample PCAP file, these were two 32-bit Windows 7 hosts under their default configuration which means that the block size is set at 64KB. Unlike SMB however, the full 64KB is used, so we will see data in chunks of 65,536 bytes being transferred. These 65,536 bytes combine with a 116 byte SMB2 header to form the full block.

SMB2 doesn’t include an additional initial request packet like the SMB Trans Request, so we don’t have to worry about stripping out any extra bytes right off the bat. As a matter of fact, some might say that carving data from SMB2 is a bit easier since you only have to strip out 116 bytes after each block of 65,536 bytes. You can do this now on putty.stage1. In doing so you should be deleting 116 bytes of data at offsets 0×10000, 0×20000, 0×30000, 0×40000, 0x50000and 0×60000.

Figure 10: Removing 116 bytes of data following the first 65,536 chunk

Once you’ve finished this save the file as putty.stage2. All that is left is to remove the final trailing bytes from the file. In order to do this, browse to by 454,656 and delete every byte that occurs after it.

Figure 11: Removing the final trailing bytes

Finally, save the file as putty.exe and you will have a fully functioning executable. The process of carving a file from an SMB2 data stream breaks down as follows:

1. Record the file name, extension, and size by examining one of the SMB2 Create Response File packets
2. Isolate and extract the appropriate stream data from Wireshark by using the Follow TCP Stream feature and selecting the appropriate direction of traffic
3.  Remove all of the bytes occurring before the actual file header using a hex editor
4. Following each successive 65,536 byte block (assuming a 64K block size), remove 116 bytes
5. Trim the remaining bytes off of the file so that it matches the file size recorded in step 1

 

Conclusion

 

That’s all there is to it. I’ll be the first to admit that I didn’t cover every single aspect of SMB and SMB2 here and there are a few factors that might affect your success in carving files from these streams, but this article shows the overall process. Taking this one step farther, it’s pretty reasonable to assume that this process can be automated with a quick Python script, but this is something I’ve not devoted the time to yet. If you feel like taking up that challenge then be sure to get in touch and I’ll be glad to post your code as an addendum to this post. In the mean time, happy carving!

 

References

http://msdn.microsoft.com/en-us/library/cc246231%28v=PROT.10%29.aspx

http://msdn.microsoft.com/en-us/library/cc246482%28v=PROT.10%29.aspx

http://channel9.msdn.com/Blogs/Darryl/Server-Message-Block-SMB21-Drill-down

 

In the realm of network security monitoring and intrusion analysis we are all slaves to our data. Typically speaking, we rely on two different types of data at the network layer; full content data (PCAP) and session data (Netflow). Both are pretty easy to generate given the right sensor placement, and there are a lot of great resources out there for learning how to get good value out of the data. That said, they do each have their own shortcomings as well.

 

Session Data (Netflow)

Netflow is a standard form of session data that details the ‘who, what, when, and where’ of network traffic. I tend to equate this to the call records you’ll see on your monthly cell phone bill.

 

 

 

 

Figure 1: Partial Netflow Records Exported from SiLK

 

The best thing about netflow is that it provides a lot of value with minimal disk storage overhead. It’s really a lot of bang for your buck. Most commercial grade routers and firewalls will generate netflow, and there are a lot of free and open source tools, such as SiLK, that can be used to generate and analyze netflow as well. There is even a yearly conference called FloCon where people get together and talk about cool things you can do with netflow. The only real downside to netflow data is that it doesn’t paint a complete picture, so it’s often best used as a complement to full content data.

 

Full Content Data (PCAP)

If netflow session data is equivalent to a call log, then full content data in the form of PCAP is just like having a full recording of all of your calls.

 

 

 

Figure 2: PCAP Data Investigation with Wireshark

 

 

The PCAP format has become very universal and can be collected and analyzed with a variety of free and open source applications like Dumpcap, Tcpdump, Wireshark, and more. A lot of the more popular intrusion detection systems, such as Snort, use the PCAP format as well. As an analyst, having PCAP data available tends to make the analytical process a dream come true as it provides the highest level of context when investigating an anomaly. The primary downside to full content data is that it has an incredibly high disk storage overhead, which prevents most organizations from collecting and storing any reasonable amount of it. In my experience, the organizations that are capable of collecting and storing PCAP can only measure the amount stored in hours, rather than days. In addition to this, unless you have an idea of what you are looking for within a reasonable time range, it can be a bit difficult to locate things as well, somewhat impeding flexibility in analysis.

 

Application Layer Metadata

The concept of application layer metadata originally presented itself to me in a discussion regarding additional data types that are useful within the network security monitoring function that were sort of a happy medium in between session data and full content data. It didn’t take a lot of number crunching to find that on most of the networks we monitored, the vast majority of the traffic was the application layer data of a few common protocols. The largest of these was HTTP, followed by the other usual suspects; SSL, DNS, and SMTP.

Starting with a couple of these protocols as a baseline, we quickly realized that we could save ourselves a lot of disk storage overhead by actually eliminating the stuff we didn’t need. There are an unlimited number of ways to do this, but we wanted to go with the keep it simple philosophy, so we started by using tcpdump to read in our PCAP data, outputting the ASCII formatted data to a file. Then, we ran the Unix strings command on that file to get read of any binary data that we couldn’t read anyways. We weeded out a few more things that we didn’t want through a magical combination of SED and AWK, added in the appropriate timestamps, formatted the data a bit prettier, and we had achieved our goal.

The end result of a reasonably small bash script was the ability generate application layer metadata in the form of something we call a Packet String, or PSTR file (pronounced pee-stur).  The script is ideally designed to run as a cron job where it parses continually generated PCAP files in order to generate accompanying PSTR files.

 

 

 

 

 

 

 

 

Figure 3: Sample PSTR Data

 

You can download the bash script that generates this data from PCAP files here. This is provided as a simple proof of concept and takes an input PCAP file and generates an output PSTR file. Now that we’ve got application layer metadata being generated in the form of PSTR files, let’s take a look at a few use cases.

 

Using PSTR as a Data Source

The original goal of generating PSTR files was to provide a data format with a low disk storage overhead that provided value to analysts as a secondary NSM data source. In a typical workflow, analysts would take an input from a detection capability, such as an IDS, and then PSTR would be another data source available to the analyst in order to provide supporting evidence in the analysis of a potential event or incident. I’ve written a few use cases here. Some of these are theoretical, but others are examples of actual things that have happened since implementing the PSTR data type.

 

Malware Infection Use Case

Let’s look at an example in which we’ve just received an alert from our IDS stating that an internal system has been detected as exhibiting symptoms of infection. The signature that fired did so because it saw a malicious GET request associated with a known botnet C2 server. The host was examined, and it appeared as though the GET request matches what is expected as a result of the signature that fired, so were able to determine with a pretty reasonable certainty that this box was infected.

Upon closer examination, we also notice that the infected host was also sending an HTTP POST with a very unique string. This looked like it might be an indicator of malicious activity, but it wasn’t something that any of your existing signatures fired on. In this case, an analyst was very quickly able to use GREP to quickly find other instances of this same string within the HTTP header data of all traffic on our monitored networks. PSTR data proved to be incredibly useful in finding other infected boxes across multiple networks.

 

Targeted Phishing Use Case

As a theoretical example, consider another example where several users have contacted your security team because they’ve received a very suspicious e-mail that seems to be targeted specifically at your company. This e-mail mentions a payroll adjustment and asks the client to access the provided link and log in with their employee ID number and password.

After examining the e-mail, you’ve determined that it has been sent from a spoofed e-mail address and that it uses a slightly modified subject line that is unique to each recipient. You’ve also noticed that based upon the reports you’ve received from users, these e-mails have come in over the past several weeks. One of the things you would want to do in this case would be to find who within your organizations received this e-mail. The purpose of this is to be able to warn the users not to click the link in the e-mail and also in hopes that you might be able to find a pattern as to why the selected recipients were chosen (access to certain systems, high profile employees, etc).

Typically, you might search through Exchange or Postfix logs to see if you can find who the recipients were. This of course relies on your organization having adequate logging and retention of those logs. The unique nature of the string however, makes it difficult to query these data sources. Using PSTR data, you can write a quick regular expression to match the semi unique subject lines and run a very quick query that will give you these results.

 

Using PSTR as a Detection Capability

The one thing we didn’t really anticipate when we created the PSTR file type was its use as a second level detection capability. When I refer to second level analysis and detection, I’m referring to moving past near real-time detection to the point in which analysts start reviewing traffic retrospectively to find things that signatures don’t catch. This often involves statistical and anomaly based detection with large data sets. This is something PSTR is perfect for.

 

User Agent Use Case

The user agent field within an HTTP header is always a good source for catching the low hanging fruit when it comes to malware infections on a network. Lots of malware will use a custom value in this field that deviates from standard browser identifying strings. The detection technique I’ve seen most commonly deployed to catch these types of malware infections at the network level rely on IDS/IPS signatures. As a matter of fact, if you subscribe to the common popular Snort rule sets then probably are using their user agent rules to detect known bad user agents.
The only problem with that detection scenario is that malware is now being generated at a rate much faster than the AV and ISD companies can keep up with. As a result, there are a LOT of malicious user agents out there that aren’t accounted for. In addition to this, some malware uses randomly generated user agent strings, meaning it’s much more difficult to write adequate signatures for detection.

One day, one of our analysts started playing around with PSTR data and wrote a quick script to parse all of the PSTR data for a given site, grab all of the user agent strings, and sort those by uniqueness. As expected, there were thousands of occurrences of the typical Firefox and Internet Explorer user agents, but what was really interesting was that there were several user agent strings only seen a handful of times that didn’t correlate to any particular known browsers. After a bit more analysis, we ended up finding quite a few machines that were infected with malware variants using these custom user agent strings. This one was a home run.

 

E-Mail Subject Use Case

The previous use case got us to thinking about other common fields with application layer metadata that we could do the same types of analysis on. One such field was the e-mail subject line field.  We modified our user agent parsing code to look at all PSTR data related to e-mail subject lines, and again had some very cool results.

Instead of most of the distribution being focused on one or two unique strings like we saw with user-agents, we saw that the distribution was spread very widely across thousands of different subject lines. This was expected, since most e-mails have a unique subject line. What interested us here however, was that we had a few subject lines that were used in excess. The first item of interest we found was that some sites had misconfigured applications that were mailing things to places they shouldn’t go, which was worth pursuing and getting fixed. We also found a user who was e-mailing all of his work documents to himself as a scheduled task every night, which was a policy violation.

This was all found with a very basic level of analysis, and it had some very real and useful results.

 

Additional Analytic Capabilities

The thing I love about this data format is that we can store a lot of it and it’s really quick to search through. With those things being true, there are a ton of things that can be done with it from a detection standpoint. A few immediate ideas include:

• Searching for unique values with HTTP, SMTP, DNS, and SSL headers

This is what we did in most of these examples. You can really quickly sort through the unique values within certain fields and find the outliers that warrant additional investigation.

• Byte entropy of certain fields to locate encrypted data where it shouldn’t be

It’s a common tactic to exfiltrate encrypted data through commonly used channels in an effort to hide in plain sight. Performing entropy calculations on GET and POST requests in an effort to find encrypted data would be a good way to detect where this might be occurring.

• Checking the length of certain fields for anomalies

You can do some statistical analysis and determine that certain fields will often have values that have a length falling in a particular range. Using that, you can flag on outliers that are far too short or too long in order to look for anomalies. I’ve seen good success in doing this with the various HTTP header fields, e-mail subject lines, and SSL certificate exchanges.

• Enumerating Downloads of Particular File Types

There is a great deal of value to being able to list all of the executables or PDF files downloaded within a certain time span. This is pretty easily achievable really quickly with analysis of HTTP header data within PSTR files.

Of course, all of these things CAN be done on PCAP data as well, but it would take significantly more processing power and it’s likely that you can’t store enough PCAP data at a given time to make it worth useful.

 

Conclusions

The concept of collecting and storing application layer metadata isn’t anything revolutionary. As a matter of fact, the idea isn’t even completely original as I’ve encountered other organizations that do similar things. There are even some commercial products that do this as well. However, I do know that nobody is sharing their methods and code with the world, which is the purpose of this post. Analysts live and die by their data feeds, and I think application layer metadata in whatever form it takes has its place amongst the other primary network data types. You can download the proof of concept code to generate and parse PSTR files here. I’m excited to see this data format evolve as we find more and more use for it. Look for more updates on this front as the code base continues to advance.

 

 

* A special thanks to my colleague Jason Smith for doing most of the legwork on writing the POC code.

 

I’m very excited to announce that my latest book, Practical Packet Analysis, Second Edition, has been released. Even more so, I’m thrilled that 100% of author proceeds for this book will be going to support the Rural Technology Fund to provide scholarships to students from rural areas pursuing further education in computer related sciences. You can read more about the Rural Technology Fund at http://www.ruraltechfund.org.

Book Description

I’ve been actively involved in the training and development of intrusion detection analysts for a few years now which includes being a SANS Mentor for SEC 503: Intrusion Detection In-Depth. One thing I find myself constantly doing is trying to evolve my philosophy on effective intrusion detection. While doing this, some themes arise that tend to stay consistent no matter how that philosophy changes. Through that, I’ve written up something I call the “10 Commandments of Intrusion Analysis” which highlight some of those themes that seem to be at the core of what I try to instill in the analysts I train and in my own analysis. They don’t really command you to anything, but there are 10 of them, so the name kind of fits. These may not fit you or your organizational goals or personal style, but they work for me!

1. Analysts, Analysts, Analysts!

The most important thing an analyst can have ingrained into them in their importance. An analyst is the first line of defense. The analyst is sitting in the crows nest watching for the icebergs. It is the analyst who can keep attacks from happening and can stop attacks from getting worse. Most security incidents begin with an analyst providing a tip based upon an IDS alert and end with an analyst putting in new signatures and developing new tools based up on intelligence gained from a declared incident. The analyst is the beginning and the end in information security. The alpha and omega. Okay, maybe that’s a bit dramatic, but the importance of an intrusion analyst can’t be understated.

2. Unless you created the packet yourself, there are no absolutes.

Analysis happens in a world of assumptions and its important to remember that. Most of the decisions you will make are centered around a packet or a log entry and then honed based upon intelligence gathered through research. The fact is that the analyst isn’t the one who generated the traffic, so every decision you will make is based upon an assumption. Don’t worry though; there is nothing wrong with that. Ask your friendly neighborhood chemist or physicist. Most of their work is based upon assumptions and they have great success. The takeaway here is that there are no absolutes. Is that IP address REALLY a known legitimate host? Does that domain REALLY belong to XYZ company? Is that DNS server REALLY supposed to be talking to that database server? There are no absolutes, merely assumptions, and because of that remember that assumptions can change. Always question yourself and stay on your toes.

3. Be mindful of how far abstracted from the data you actually are.

An analyst depends on data to perform their function. This data can come in the form of a PCAP file, an IIS log file, or SYSLOG file. Since most of your time will be spent using various tools to interact with data it’s crucial to be mindful of how that tool interacts with the data. Did you know that if you run Tcpdump without specifying otherwise, it will only capture the first 68 bytes of data in a packet? How about that Wireshark displays sequence and acknowledgement numbers within TCP packets in a relative manner by default? Tools are made by people and sometimes “features” can cloud data and prevent proper analysis. I think both of the features I described earlier are great, but I’m also mindful that they exist so I can see all of the packet data available or view the real sequence and acknowledgement numbers when needed. In a job where reliance upon data is critical, you can’t afford to not understand exactly how tools interact with that data.

4. Two sets of eyes are always better than one.

There is a reason authors have editors, policemen have partners, and there are two guys sitting in every nuclear silo. No matter how much experience you have and how good you are you will always miss things. This is to be expected because different people come from different backgrounds. I work with the government so the first thing I look at when examining network traffic is the source and destination country. I’ve worked with people who have systems administration backgrounds and as a result, will look at the port number of the traffic first. I’ve even worked with people who have a number crunching background who will look at the packet size first. This demonstrates that our experiences shape our tactics a bit differently. This means that the numbers guy might see something that the sysadmin didn’t see or that the government guy might have insight that the numbers guy didn’t. Whenever possible it’s always a good idea to have a second set of eyes look at the issue you are facing.

5. Never invite an attacker to dance.

This is something I’ve believed since the first day I ever fired up a Snort sensor, but IDS guru Mike Poor phrased it best while I was attending one of his SANS classes when he said that you should never invite an attacker to dance. As an analyst its very tempting to want to investigate a hostile IP address a bit beyond conventional means. Trust me, there have been many occasions where I’ve been tempted to port scan a hostile that kept sending me painfully obviously crafted UDP packets. Even more so, any time someone attempts to DOS a network I’m responsible for defending, I wish nothing more than to be able to unleash the full fury of a /8 network on their poor unsuspecting DSL connection. The problem with this is that 99% of the time we don’t know who or what we are dealing with. Although you may just be seeing scanning activity, the host that is originating the traffic could be operated by a large group or even a military division of another country. Even something as simple as a ping could tip off an attacker that you know they exist, prompting them to change their tactics, change source hosts, or even amplify their efforts. You don’t know who you are dealing with, what their motivation is, and what there capabilities are, so you should never invite them to dance.

6. Context!

One word can drastically change the dynamic of your monitoring and detection capabilities. In order to be effective you must have context into the network you are defending. Network diagrams, listings of servers and their roles, breakdowns of IP address allocations, and more can be your best friend. Basically any and everything that can be used to document the assets within the network, how they function, and how they relate to other assets are beneficial in running down anomalous events. Depending upon your role in the organization you may not be in a position to obtain these things and if they don’t already exist you are going to have a heck of a time getting the systems folks to put in the leg work to create them. However, as difficult as this may be, its an effort that’s worth pursuing. Whether you have to present your case to the CIO or just buy your network engineers a case of their favorite adult beverage its ultimately worth the effort.

7. Packets, in a word, are good.

The ultimate argument in life is whether or not people are inherently good or inherently evil.  This same argument can be had for packets as well. You can either be the analyst that believes all packets are inherently evil or the analyst that believes all packets are inherently good. I’ve noticed that most analysts typically start their career as for the former and quickly progress the later. That’s because its simply not feasible to approach every single piece of traffic as something that could be a potential root level compromise. If you do this, you’ll eventually get fired because you spent your entire day running down a single alert or you’ll just get burnt out. There is something to be said for being thorough but the fact of the matter is that most of the traffic that occurs on a network isn’t going to be evil, and as such, packets should be treated innocent until proven guilty.

8. Analysis is no more about tcpdump than astronomy is about a telescope.

Whenever I interview someone for any analyst position that’s above entry level I always ask them to describe how they would investigate a typical IDS alert. I get frustrated when someone gives answers along the lines of “I use  Tcpdump, Wireshark, Network Miner, Netwitness, Arcsight, Xeyes, etc” with no further clarification. Although their are processes and sciences in intrusion analysis, intrusion analysis itself is not a process or a science, but rather an art. If this wasn’t the case then it wouldn’t even be necessary to have humans in the loop when it comes to intrusion detection. An effective analyst has to understand that while different tools may be the most important part of the job, those things are merely pieces of the puzzle. Just like an astronomer’s telescope is just another tool in his arsenal that allows him to figure out what makes the planets orbit the sun, Wireshark is just another tool in an analysts arsenal that allows him to figure out what makes a packet bypass a firewall rule. Start with the science, add in a few tools and processes, stay cognizant of the big picture, keep an attention to detail, and eventually the combination of all of those things and the experience you gain over time will help you develop your own analysis philosophy. It’s at that point you have taken your analysis to the level of an art, and made it so that your worthy enough to not be replaced by a machine.

9. Sometimes, we lose.

No matter how hard you try there will come a point in which the network you are defending gets successfully attacked and compromised. In the modern security landscape its inevitable and there isn’t a lot you can do about it. In these times its likely that the analyst will take the heat over the incident. Because of this, you need to be prepared when it happens. An incident won’t be remembered for how an intrusion occurred, but rather how it was responded to, the amount of downtime that occurred, the amount of information that was lost, and ultimately the amount of money it costs the organization. What recommendations can you make to management to ensure a similar incident doesn’t occur again? What can you show your superiors to explain why the attack wasn’t detected? What shortcomings do your tools have? These are questions that can’t fully be answered until an intrusion has occurred and you have the context of an attack, but you can definitely consider the questions now and have a plan for how your information will be presented to key figures. You will get caught off guard and you will be blind sided, but its important that you don’t appear as such and you keep your game face on. This can make the difference between a promotion and a pink slip.

10. Dig deeper.

At the end of the day you have to have something to rest your laurels on and that has to be the fact that you’ve done your due diligence and that you’ve given your best. My “motto” per se when it comes to intrusion analysis is “Dig Deeper”. A defender has to control 65,535 ports. An attacker has to compromise one. A defender has to protect 10,000 users. An attacker has to deceive one. A defender has to examine millions of packets. An attacker has to hide a malicious payload in one. What can you do to increase your visibility into the data? What proficiency can you develop that gives you that edge against the attacker? You have a hunch that there is more than meets the eye, so what can you do to dig deeper?

It happens pretty often that I’ll come across an interesting PCAP file that I want to share with others. Unfortunately, divulging these packet captures can give away certain sensitive information such as an organizations internal IP range, IP addresses of sensitive company assets, MAC addresses of critical hardware that could identify the product vendors, and more.

Fortunately, there is a tool which helps alleviate some of these issues. The tool is called Tcprewrite and is actually a part of the Tcpreplay suite. Tcpreplay is used to send packets from a PCAP back across the wire, but the suite actually contains a few other useful tools.Tcprewrite itself can be used to add and modify packet fields within PCAP files. For example, if you wanted to replace the layer two addressing information within a PCAP so that all of the packets have a specified source and destination MAC address you could use the following syntax:

tcprewrite --enet-dmac=00:55:22:AF:C6:37 --enet-smac=00:44:66:FC:29:AF --infile=input.pcap --outfile=output.pcap

Tcprewrite can do neat things at layer four as well, including remapping ports used in sessions. The following example will remap all port 80 traffic to port 8080.

tcprewrite --portmap=80:8080 --infile=input.pcap --outfile=output.pcap

The examples shown above were taken directly from the tcprewrite wiki page (http://tcpreplay.synfin.net/wiki/tcprewrite) where you can find quite a few other usage examples.

The real value of tcprewrite, and the reason for this article, is its ability to randomize the addressing information in a PCAP file. This is done with the following syntax:

tcprewrite --seed=423 --infile=input.pcap --outfile=output.pcap

In this line, the seed option is used in the randomization of the addresses. This will replace all of the IP addresses in the IP headers of the packets and will also modify any ARP packets in the traffic accordingly.

From what I’ve been able to determine this option doesn’t randomize and rewrite and MAC addresses, which is a bit of a problem since MAC addresses can give away the vendor of a piece of hardware. The last thing I want is the entire world knowing that I use Cisco/Juniper/Enterasys/Etc based external firewalls. The ability to rewrite MAC addresses is there but its not random. What you can do in this case is to split a PCAP file into two separate files representing each direction of traffic. This can be done with tcpdump or tcpprep, which is a part of the tcprewrite suite as well. Using tcprewrite you can split the traffic like this:

tcpprep --auto=bridge --pcap=input.pcap --cachefile=input.cache

From there you can use syntax similar to what was shown above to replace the MAC addresses. This isn’t randomized so you will basically have to make something up. At the very least I’d recommend replacing the OUI section of the MACs. That syntax would look something like this:

tcprewrite --enet-dmac=00:44:66:FC:29:AF,00:55:22:AF:C6:37 --enet-smac=00:66:AA:D1:32:C2,00:22:55:AC:DE:AC --cachefile=input.cache --infile=input.pcap --outfile=output.pcap

It wouldn’t be too much of a stretch to write a python script that uses tcpprep and tcprewrite to automate the randomization of MAC addresses as well.

You can download tcprewrite as part of the tcpreplay suite at http://tcpreplay.synfin.net/ or just apt-get/yum install tcpreply. The tool is Unix only (or you can use Cygwin if you are tied to Windows).

I woke up this morning and was very excited to see a post on a blog a frequent, Packet Life. It looks like the folks at QA Cafe have just launched a new project called CloudShark. I’ve been playing with CloudShark all morning and I’m very impressed. A colleague of mine wrote something similar to this a while back with intentions of publishing it but never did, so I’m glad someone set forth on a similar project. I plan on using CloudShark as a component of this blog, so from now on any packet captures I post will have a “view online” link that should display the captures directly in your browser.

The best resource for more information on CloudShark seems to be their FAQ:

What is CloudShark?

CloudShark is a web site that displays network capture files right in your browser instead of running desktop tools such as Wireshark. You upload, link, or email your capture files and we’ll display them.

Why CloudShark?

We work with network capture files on a daily basis. After trying to view capture files on mobile devices without Wireshark support, we realized it was time to move packets to the cloud. The CloudShark idea was born. CloudShark was created to make viewing capture files easy from any device ranging from desktops to smart phones. After creating our own solution, we decided to make it available to everyone as CloudShark.org.

How does it work?

* Generate your capture file or use an existing capture file
* Email it, upload it, or link it
* CloudShark does the rest by providing a decode session
* If you email CloudShark with an attached capture file, we’ll email you back with a link to your decode session
* Send your capture files as an attachment to cap@cloudshark.org
* If you are in the browser already, we’ll drop you into your decode session

Are my capture files publicly accessible?

While the URLs to your decode session are not publicly shared, we make no claims that you data is not viewable by other CloudShark users. For now, if you want to protect sensitive data in your capture files, don’t use CloudShark.

Is there any limit to the size of the capture file I can upload?

Capture files are currently limited to 512 Kbytes. Larger files will be rejected.

Can I delete my decode session after I am done with it?

Not directly. Eventually it will be deleted when the disk space is recycled.

How long is my decode session available?

CloudShark is not a file storage site. We’ll try to keep your files around, but obviously there is a limit to the amount of files we can keep around. If the link to your decode session is no longer working, you may need to upload the capture file again. In the future we may provide persistent storage, but for now you should store your capture files somewhere else.

What capture formats are supported?

CloudShark uses tshark to do the actual decoding. tshark supports several capture files from other tools besides Wireshark. See http://wiki.wireshark.org/FileFormatReference.

I have a capture file hosted on my web site. Is there an easy way I can link a CloudShark decode session to this capture file?

Yes. You can create a CloudShark link that includes a URL to your capture file. Here is an example:

http://www.cloudshark.org/view?url=http://packetlife.net/captures/TCP_SACK.cap

Who are you?

CloudShark was created by QA Cafe. We are the creators of CDRouter, the leading CPE testing solution. We spend a lot of time working with capture files. You can visit us at qacafe.com.

Is this project connected to Wireshark.org?

No, not directly. We do use Wireshark, tshark actually, on our back end.

How can I contribute?

If you have any ideas, you can contact us at info@cloudshark.org.

Kudos to the folks at QA Cafe for putting this together! You can visit CloudShark at http://www.cloudshark.com and you can follow Cloudshark developments on Twitter at @Cloudshark.

The newest issue of (In)Secure Magazine has been released today. This issue contains an article I’ve written entitled “Using Wireshark to Capture and Analyze Wireless Traffic”.

 

Article Introduction:

 

The tricky thing about a wireless network is that you can’t always see what you’re dealing with. In a wireless network, establishing connectivity isn’t as simple as plugging in a cable, physical security isn’t nearly as easy as just keeping unauthorized individuals out of a facility, and troubleshooting even trivial issues can sometimes result in a few expletives being thrown in the general direction of an access point. That being said, it shouldn’t come as a surprise that analyzing packets from a wireless network isn’t as uninvolved as just firing up a packet sniffer and hitting the capture button.

 
In this article I’m going to talk about the differences between capturing traffic on a wireless network as opposed  to a wired network. I’ll show you how to capture some additional wireless packet data that you might not have known was there, and once you know how to capture the right data, I’m going to jump into the particulars of the  802.11 MAC layer, 802.11 frame headers, and the different 802.11 frame types.

The goal of this article is to provide you with some important building blocks necessary for properly analyzing wireless communications.

 

 

 

 

 

You can view the full article in the (In)Secure Magainze June issue, which can be obtained here: http://www.net-security.org/insecuremag.php.

Laura Chappell is now doing regularly scheduled online training seminars. I had the privelege of attending one of these last Thursday called the “Top 10 Reasons Your Network is Slow” and it was really great.

You can see a schedule of Laura’s training at http://www.chappellseminars.com/schedule-name.html.

Do you have a story about a time when you used packet analysis to solve a problem on your network? If so, I want to hear that story. E-Mail me at chris@chrissanders.org and your story could be featured on this site or even in the next edition of Practical Packet Analysis.

I’ve just announced my second online training event. This event will be happening on Wednesday, May 27th at 2 PM EST.

 

Course Description:

This is an introductory level packet analysis course with a focus on practical usage. The goal of this course will be to give you exactly what you need to jump deep into your network with Wireshark and begin getting value out of these skills immediatley. This course will use completely new files and scenarios and will not repeat any real-world scenarios taught in my book or in my previous trainings.

 

Prerequisites:

In order to understand what is going on in this course you will need to have a decent level of experience troubleshooting networks and client/server communications. You won’t be expected to know how individual protocols look on the wire (I’ll teach you that) but you will be expected to know what DHCP/DNS/SMTP/ETC are used for. 

The course will be administered using Citrix Go2Meeting which will transmit live audio and video from my computer. Because of this, some form of broadband Internet connection is recommended. I’ve used this format before and it seemed to work really well as all users were able to connect and listen/watch successfully.

 

Who Should Attend:

If you troubleshoot or maintain a network on a daily basis then this course will provide immediate value to you. Packet Analysis is one of the hottest growing skill sets amongst IT staff in the world and is an absolute requirement to troubleshoot certain problems that may be faced. If you want to save yourself time, save your organization money, or make yourself more marketable by increasing your skill set, then this is the course for you.

 

Cost:

The early registration cost for this course is $100 USD. This pricing is valid until May 5th. After May 5th, the cost goes up to $150 USD. If you work for a non-profit or in education, please e-mail me for a discounted rate. The course is limited to a set number of participants so that I can get to all questions that may be asked, so your best bet is to get in early.

 

Curriculum:

Hour 1 – Intro, Theory, and Getting Your Feet Wet

• How Packet Analysis Can Help You
• “War Stories”
• How a Packet Sniffer Works
• Getting and Installing Wireshark
• Sniffer Placement on Your Network
• Walkthrough of Wiresharks Features Using Real Trace Files

Hour 2 – Protocols and Performance with Real World Case Scenarios

• Analyzing Common Protocols When They Work and When They Don’t
• Troubleshooting Network Performance Problems
• Steps for Creating a Network Baseline
• The 7 Deadly Sins of the Network

Hour 3 – Security, Wireless, and More Real World Scenarios

• Analyzing Common Network Attacks
• Wireless Packet Analysis
• Additional Tools and Resources
• Q&A

 

Registration:

In order to sign up for this course, please fill out the registration form below. At some point after registering, you should receive an e-mail from me with payment details.

 

 

As always, if you  have any questions regarding this training please e-mail me.

When you are capturing a lot of traffic the size of your capture files can grow really quickly. When this happens you are really going to have a hard time getting anything done when trying to sort through the file. There are a couple of things you can to do prevent this from happening.

Use an Effective Capture Filter

Capture filters are great when you know what you are looking for. If you ONLY want SMTP traffic, you can capture only that traffic. If you ONLY want to see HTTP POSTs, then you can capture only that traffic. If you aren’t sure what you are looking for then its bests to stick to capturing everything and using display filters, but when you have an eye on your target then capture filters are a great way to cut through the weeds. You will find this especially beneficial when capturing packets from a busy server or network segment.

Some of the things you can filter based upon include:

• Specific Protocols
• A Particular IP/MAC Address
• Incoming/Outgoing Traffic Only

Split the Capture File as It’s Being Captured

Wireshark has some really great flexibility in allowing you to split a capture file as its being created. You can access this by selecting Capture from the main drop-down menu and selection Options, or by pressing Ctrl+K.

You have a couple of options here and they all become available to you when you place a check mark next to the Use Multiple Files box. There are two primary sections which I’ve creatively labeled the Multiple File and Stop Capture sections.

The multiple file section lets you specify a point at which a new file is created, either by reaching a certain size limit or at a certain time interval. I find that I typically use the size option for typical uses, but specifying a time interval for the packet capture can become very useful when you are trying to pinpoint when a certain event is happening. In this scenario you could start the capture at 12:00 and place and set the multiple file option to create the next file every 1 hour which should create a nice clean display of capture files by the hour.

It’s important to note that you can specify both of these criteria and in this case, a new file will be created when EITHER condition is met. In this top section you can also specify a maximum number of files to be created (don’t underestimate the value of this, I’ve accidentally filled up a hard drive on many occasions) and specify a ring buffer. A ring buffer uses a set number of files, and after the last file has been written it will begin overwriting the first file and cycling back through.

The stop capture section of this area is very straightforward and allows you to stop a capture after it reaches a certain point, either at a certain number of packets captured, a certain size limit, or a particular time interval. This comes in handy when you want to start a capture and run off to lunch or take a call.

Ensure You Are Capturing in the Best Location

One of the most overlooked parts of the packet analysis process is ensuring you are properly tapped into the network and getting the packets you need. Although the typical concern is whether or not you are getting enough packets, there are some cases in which you may be capturing TOO MUCH information. If you are having trouble weeding through a large capture file then you need to ask yourself if you really need to be where you are at. If it is a client/server issue, do you really need to capture from the server or would capturing from the client yield the traffic you need? If you are analyzing a slow network link, do you really need to be inside the router or would you be best suited to tap the outside interface of the router and get away from the internal networks broadcast domain?

Working with larger capture files is a real quick way to bog down your system and further complicate what may already be a long drawn out process. Using these techniques you should be able to keep tabs on your capture file size and make your analysis process quite a bit more efficient.

I’ve already written quite a bit about getting on the wire as it pertains to packet analysis. Half the battle when you are capturing packets is placing the sniffer computer so that it captures the packets you need. The advent of switched networks makes this a bit harder on us as traffic is now directed and not free-flowing across every port on a network. In a post a few months ago I outlined three methods for getting on the wire. Those three methods were ARP Cache Poisoning, Hubbing Out, and Port Mirroring. One other technique which I had not previously used, but have now grown to love is using a network tap.

 

A tap is basically a hardware device that you can place on the wire to intercept the right packets.

 

The tap has at least three ports. These are inbound and outbound ports and a monitor port.

 

Say you wanted to intercept all network traffic entering your router. Typically, you would have a single cable going from a switch to your router. In order to insert the tap into the mix, you would unplug the current cable from the router and plug it into the inbound port on the tap. You would add an additional cable from the outbound port of the tap into the port on your router. Lastly, you would place a cable into the monitor port that leads to your analysis machine. The analysis machine will then capture all traffic flowing between the switch port and the router.

 

The great thing about doing this as opposed to hubbing out is that you aren’t using an old school hub that could cause dropped packets and limits you to half-duplex communication. This is also advantageous over ARP cache poisoning because it doesn’t generate any extra traffic on the wire, which is something you typically want to avoid doing…especially in security scenarios.  If your layer three switches typically have a very high processor utilization, you could also consider this over port mirroring. The tap adds no extra traffic or latency to the traffic on the wire and is completely undetectable.

 

That all being said I recommend the Barracuda network tap. They run about $130 and have an added benefit of having TWO monitor ports. One port monitors all inbound traffic and the other monitors all outbound traffic rather than having a single port for both, which can add some flexibility in your analysis. The Barracuda tap also allows for the use of a nine volt battery in situations where a power outlet isn’t handy or you just want to capture some packets quickly.

 

 

 

You can get the Barracuda network tap from http://www.barracudanetworks.com/tap/.

Laura Chappell, one of the packet analysis world’s best, has just announced Chappell University. Here is her official statement from her newsletter:

“Chappell University (www.chappellU.com) is open for registration today. Subscription-level service will be open soon – I’ll let you know. Chappell University is an affordable, on-demand, online training system to maintain and enhance IT skills in the area of analysis, troubleshooting and security. Last night I uploaded two lab workbooks with over 100 lab exercises using Wireshark to spot network problems, security breaches, and analyze normal and abnormal TCP/IP communications. I’ve recoreded video answers to all the lab exercises. In addition, I’ve uploaded my trace file respository and you’ll see me uploading additional WLAN, VoIP, bot-infections, application, etc., trace files each quarter. Check out the new YouTube Channel for Chappell University at www.youtube.com/chappellU and the video “Ethical Hacking with NetScanTools Pro: Tutorial on ARP Scanning to Discover All Local Hosts” (even those hidden behind firewall applications). “

If you haven’t had the pleasure of experiencing Laura’s training on-site, or via Wireshark University, I would highly reccommend both. As I said, she is one of the best in the field.