By Richard Bejtlich, Principal Security Strategist, Corelight
Recently a fan of network security monitoring (NSM) asked me for advice on his current instrumentation situation. He said his organization was new to NSM and was interested in pursuing solutions with Corelight. However, the company did not have any network taps in place. He wanted to know if he should wait for the installation of taps before trying a Corelight sensor in production.
If you’ve read my previous post Don’t Delay – Corelight Today, you can probably anticipate my response to this question. While enterprise-grade network taps are the most reliable way to access network traffic, the lack of such infrastructure should not postpone your NSM deployment. This post will discuss four dimensions of this problem and offer advice on making the best of available visibility options.
Off-line, Not In-Line
Let’s begin by understanding the infrastructure needed to make effective use of an NSM solution like Corelight. The Corelight sensor watches network traffic and interprets what it sees, generating compact, high-fidelity transaction logs. At the moment, one cannot load saved network traffic, perhaps stored in packet capture (PCAP) format, into a Corelight sensor. The platform must be watching live traffic. This is a common aspect of most production-grade NSM platforms.
NSM solutions like Corelight are not designed, or at least should not be designed, to sit “in-line.” In other words, Corelight does not physically sit between network infrastructure devices, like a firewall and a switch. Corelight sits “off-line,” accessing a copy of network traffic provided by another infrastructure device. This is an important architectural aspect of properly designed NSM solutions. NSM platforms are not engineered for redundancy or to be single points of failure in the network, due to their general underlying complexity.
Span Ports vs Taps vs Packet Brokers
Once we understand how the Corelight sensor accesses network traffic, we must ask what network infrastructure device should provide that visibility. There are essentially three mainstream options, each with advantages and disadvantages. Although there are a few more exotic options available, this post addresses the three most popular versions.
The first is to leverage a network switch to provide copies of traffic to send to the Corelight sensor. The mechanism offered is called a span port. Depending on who you ask, span is either an independent, “lower-case” term, or an acronym for “Switched Port ANalyzer.” Virtually all managed switches provide span ports. Enterprise switches will differ in the number of span ports they provide, with a minimum of one, and higher numbers as the hardware costs increase. Generally network administrators will “span the uplink,” meaning they will identify the switch port that carries traffic to the next network device in the path towards the Internet, such as a firewall or another switch. The administrator will select a free port as the span port. That span port will receive copies of a subset of the traffic seen on the uplink port. Corelight’s monitoring port will connect to the span port, thus seeing traffic.
In contrast to span ports, network administrators can deploy dedicated visibility devices called network taps. Again, depending on who you ask, tap may be an independent term, or an acronym for “Test Access Port.” An enterprise-grade tap is a hardware device designed to be placed in-line, and engineered to be reliable as a single point of failure. The quality of properly engineered taps may vary, but at the very least I recommend avoiding so-called “homebrew” taps. Why would you risk your network uptime on a device you built yourself, splitting and combining cables? You would be better off buying a cheap managed gigabit switch for less than $50, rather than creating a so-called “tap” designed in a YouTube video.
Packet brokers, for the purposes of this post, are essentially smart and complex taps. They offer a variety of input and output ports, plus associated logic to access, filter, combine, slice, and otherwise manipulate network traffic. Manufacturers invented packet brokers to give tap customers more options for handling complex enterprise network requirements. Hereafter I will consider packet brokers to be a specialized network tap.
Advantages of Span Ports over Taps
In general, I recommend taps whenever possible. I offer five reasons why taps are the best choice for accessing traffic, based on my 2009 TaoSecurity blog post:
- Taps free span ports for tactical, on-demand monitoring, especially intra-switch monitoring. Many switches have only two ports capable of spanning, and some offer only one. If you commit a span port for permanent monitoring duties, and you need to reassign it for some sort of troubleshooting on a VLAN or other aspect of the traffic, you have to deny traffic to your sensor while the span port is doing other work. Keep your span ports free so you can do intra-switch monitoring when you need it.
- Taps provide strategic, persistent monitoring. Installing a tap means you commit to a permanent method of access to network traffic. Once the tap is installed you don’t need to worry about how you are going to access network traffic again. Taps should really be part of any network deployment, especially at key points in the network.
- Selected taps do not permit injected traffic onto the monitored link. Depending on the tap you deploy, you will find that it will not be physically capable of transmitting traffic from the sensor to the monitored link. This is not true of span ports. Yes, you can configure span ports to not transmit traffic, but I have seen that setup violated.
- What taps see is not influenced by configuration (as is the case with span ports); i.e., what you see is really what is passing on the link. This is key, yet underestimated. If you own the sensor connected to a span port, but not the switch, you are at the mercy of the switch owner. If the switch owner mistakenly or intentionally configures the span port to not show all the traffic it should, you may or may not discover the misconfiguration.
- Taps do not place traffic on a switch data plane, like a span port does. This point is debatable. Depending on switch architecture, span ports may or may not affect the switch’s ability to pass traffic. A span port may not receive all traffic when the switch is loaded, because forwarding may take precedence over spanning.
Clearly I am a fan of network taps!
Span Away, and More
It would seem that, given the advantages of network taps, one should wait for their placement before deploying a Corelight sensor. At this point, one of my security rules takes precedence over my preference for taps: something is better than nothing. I would rather access at least some traffic from today’s span port than have nothing from tomorrow’s tap! Yes, the switch span port is not ideal, but I will take it as soon as possible and plan for a future tap deployment.
What if a span port isn’t available? Network administrators sometimes cringe at the prospect of enabling them, so what can be done in the mean time? This is one of the advantages of Corelight being built on real open source Zeek software. If I did not have access to network infrastructure via tap or span to see traffic and generate logs, I would consider three other options. Both leverage Zeek.
First, I would consider capturing network traffic using Tcpdump, Windump, or Dumpcap on one or more important servers. One could also collect traffic on one or more important client devices, like laptops or desktops. Open source Zeek could then process that traffic by reading the saved PCAP files, generating transaction logs. The idea is to gain familiarity with the sorts of logs Zeek generates, as they will be very similar to those created by Corelight. (I saw “very similar” as Corelight offers options to enrich, filter, or otherwise extend default Zeek formats.) While learning about Zeek logs, analysts can also learn about how their network looks from the perspective of a server or client. This is a subtle yet important point, and will permit investigators to have a head start once they are working with real Corelight sensors.
Second, platform-permitting, administrators could run open source Zeek on computers directly. This would be easiest on Unix-based systems, like Linux, FreeBSD, or others that Zeek supports. Administrators should be careful to not jeopardize the underlying operation of the server in question, and should test deployments before committing to production use.
Third, if the organization utilizes virtualization-based offerings like VMware or Hyper-V, administrators could deploy a Corelight virtualized sensor, and watch traffic to and from the virtualization server. This would generate NSM logs for the subset of devices hosted by the virtualization server, again offering experience with the software and format to security and network teams.
There are many deployment options available for Corelight, thanks to its relationship with the open source Zeek network security monitoring solution. I hope this blog post has provided a few ideas for those who want to better understand what’s happening on their network, regardless of the state of visibility infrastructure in play.