If you want a copy of all the traffic going through an interface, mirroring is probably the best pick. Anyhow, mirroring might be expensive as it means copying every single packet and generates a high volume of traffic.
If you do not need to see every single packet but simply to collect information about the flows that traversed that interface, then jflow is the way to go.
Jflow is juniper’s implementation of the more known netflow.
Netflow is a protocol bringing flow information. Netflow packets are sent towards a collector and they include a wide variety of data: addresses, ports, packets, …
Initially, jflow was only supported via a service card (e.g. MS-MPC). This limitation was overcome thanks to the introduction of inline jflow.
Inline jflow does not require any service card but, as the name suggests, takes place inline, on the FPC. As a consequence, no additional hardware is required!
To learn how it works, I used a small lab made of some vMXs
As the image shows, packets will be sampled on the interface connected to the client. Sampling will be performed inline on the FPC that interface belongs to.
Now, let’s have a look at a minimal jflow configuration.
First, we configure sampling:
set forwarding-options sampling instance jflow_inst input rate 1
set forwarding-options sampling instance jflow_inst family inet output flow-server 192.168.4.2 port 12345
set forwarding-options sampling instance jflow_inst family inet output flow-server 192.168.4.2 version9 template jflow_templ
set forwarding-options sampling instance jflow_inst family inet output inline-jflow source-address 3.3.3.3- sampling rate is set to 1; this means that every packet will be sampled. If we had rate 10, then we would sample one packet out of 10. Of course, lower the rate (1), the higher the load on the device as every packet will require processing
- we tell junos where to send flows data. In this case to a server with IP 192.168.4.2, port 12345. UDP is used. To reach the collector, inet.0 will be used, standard routing.
- we also “link” the collector with a jflow template (shown later)
- last, we have to specify inline jflow is used. We also set jflow packets source address
Next, we move to the jflow template:
set services flow-monitoring version9 template jflow_templ flow-active-timeout 300
set services flow-monitoring version9 template jflow_templ flow-inactive-timeout 60
set services flow-monitoring version9 template jflow_templ template-refresh-rate packets 480000
set services flow-monitoring version9 template jflow_templ template-refresh-rate seconds 60
set services flow-monitoring version9 template jflow_templ option-refresh-rate packets 480000
set services flow-monitoring version9 template jflow_templ option-refresh-rate seconds 120
set services flow-monitoring version9 template jflow_templ ipv4-template
set services flow-monitoring version9 template jflow_templ flow-key flow-direction- we use jflow version 9 (ipfix is also available)
- active timeout tells how often junos must export data about that flow towards the collector. This parameter is useful for long-lived session. Let’s assume we have a long TCP download session. Every 5 minutes (300 seconds), junos will send a jflow packet for that flow with updated data (number of packets, bytes, etc…)
- inactive timeout, instead, tells how long junos has to wait before declaring a flow “dead”, export its data to the collector and purge the flow. For example, assume we have a flow and all of a sudden no more packets belonging to that flow are seen. At that point, junos will wait one minute (60 seconds), then it will purge the flow and send jflow packet to the collector.
Be aware, inactive timeout only plays a role for flows that suddenly disappear. If a flow is closed normally (e.g. FIN), the MX purges the flow and sends jflow packet to collector without waiting for the inactive timeout to expire - template and option refresh rate are used to tell how often junos has to send a packet to the collector saying how jflow data will look like. In the above configuration, refresh packets are sent every 2 minutes or after 480k packets
- we specify the template is for ipv4 traffic (we might have templates for ipv6 or mpls)
- last, we tell to include flow-direction when building flow keys (along with the usual 5-field tuples)
Jflow instance must be “attached” to a FPC:
set chassis fpc 0 sampling-instance jflow_instLast, we create a firewall filter to sample traffic and apply it to the client-facing interface:
set firewall family inet filter sample term sample then count sampled
set firewall family inet filter sample term sample then sample
set firewall family inet filter sample term sample then accept
set interfaces ge-0/0/0 unit 0 family inet filter input sample
set interfaces ge-0/0/0 unit 0 family inet filter output sample
set interfaces ge-0/0/0 unit 0 family inet address 192.168.1.1/24We commit and verify inline jflow is active:
root@mirror# run show services accounting flow inline-jflow fpc-slot 0
Flow information
FPC Slot: 0
Flow Packets: 477, Flow Bytes: 42297
Active Flows: 0, Total Flows: 37
Flows Exported: 27, Flow Packets Exported: 26
Flows Inactive Timed Out: 14, Flows Active Timed Out: 23
Total Flow Insert Count: 14
IPv4 Flows:
IPv4 Flow Packets: 477, IPv4 Flow Bytes: 42297
IPv4 Active Flows: 0, IPv4 Total Flows: 37
IPv4 Flows Exported: 27, IPv4 Flow Packets exported: 26
IPv4 Flows Inactive Timed Out: 14, IPv4 Flows Active Timed Out: 23
IPv4 Flow Insert Count: 14It seems to work.
We move to the collector to see how netflow packets look like:
[root@colector ~]# tcpdump -n -i ens3f2 -n
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ens3f2, link-type EN10MB (Ethernet), capture size 262144 bytes
13:41:53.264318 IP 3.3.3.3.50103 > 192.168.4.2.12345: UDP, length 56
13:41:53.267639 IP 3.3.3.3.50103 > 192.168.4.2.12345: UDP, length 48Here they are! Notice, udp destination port 12345.
We can save captured packets on a pcap file and open it in wireshark.
In order to see jflow packets content, we have to “Decode As” cflow.
We can see three type of packets:
- data, contain actual flow information
- data template, template info
- options template, options info
Notice, there is 1 minute between the two “data template” packets. This makes sense as template-refresh-rate was set to 60.
Let’s look inside a flow packet:
This jflow packet is about a SSH session between client and its gateway. Traffic is untagged (vlan id 0) and direction is ingress (from client). Eight packets belonging to this flow in this direction were sampled.
That’s it! Working sampling!
Configuring sampling is pretty easy. What takes time and can only come with direct experience is tuning parameters, sample rate mainly. As said, sample rate impacts device load. The more I sample, the bigger the burden on the device. The less I sample, the less the device suffers. At the same time, the more I sample, the more I get accurate data.
Let’s make an example. Let’s assume there is a flow with 1M packets and I set rate 10. Statistically, I might sample about 100k packets. The expected error is sqrt(10/10M) which is about 0,003, let’s say 0,3% (3000 packets out of 1M). This means, with 100k sampled packets, the original packet count for that flow might be in the range 997000-1003000.
As often, it is a tradeoff…but this goes beyond junos 🙂
Ciao
IoSonoUmberto
IoSonoUnRouter 