Installed xenoeyes on Linux Debian

[walterpoli]

Hi, I installed the xenoeyes netflow collector on linux debian ; I follow the STEP By STEP instructions but the executable xenoeyes doesn't work. After experted the data to DB postgresql, all counter are 0 :
xenoeyedb=# select time, iana_protocols.name, octets, packets from ingress_by_proto join iana_protocols on ingress_by_proto.proto=iana_protocols.num;
time | name | octets | packets
------------------------+--------+--------+---------
2023-04-26 13:46:57+02 | HOPOPT | 0 | 0
2023-04-26 13:46:57+02 | ICMP | 0 | 0
2023-04-26 13:46:57+02 | TCP | 0 | 0
2023-04-26 13:46:57+02 | UDP | 0 | 0
2023-04-26 13:51:57+02 | HOPOPT | 0 | 0
2023-04-26 13:51:57+02 | ICMP | 0 | 0
2023-04-26 13:51:57+02 | TCP | 0 | 0
2023-04-26 14:14:29+02 | HOPOPT | 0 | 0
2023-04-26 14:14:29+02 | ICMP | 0 | 0
2023-04-26 14:14:29+02 | TCP | 0 | 0
2023-04-26 14:14:29+02 | UDP | 0 | 0
2023-04-26 14:19:29+02 | HOPOPT | 0 | 0
2023-04-26 14:19:29+02 | ICMP | 0 | 0
2023-04-26 14:19:29+02 | TCP | 0 | 0
2023-04-26 14:19:29+02 | UDP | 0 | 0
2023-04-26 14:24:29+02 | HOPOPT | 0 | 0
2023-04-26 14:24:29+02 | ICMP | 0 | 0
2023-04-26 14:24:29+02 | TCP | 0 | 0
2023-04-26 14:24:29+02 | UDP | 0 | 0
2023-04-26 14:29:29+02 | HOPOPT | 0 | 0
2023-04-26 14:29:29+02 | ICMP | 0 | 0
2023-04-26 14:29:29+02 | TCP | 0 | 0
2023-04-26 14:29:29+02 | UDP | 0 | 0
2023-04-26 14:47:57+02 | ICMP | 0 | 0
2023-04-26 14:47:57+02 | TCP | 0 | 0
2023-04-26 14:47:57+02 | UDP | 0 | 0
2023-04-26 14:55:32+02 | ICMP | 0 | 0
2023-04-26 14:55:32+02 | TCP | 0 | 0
2023-04-26 14:55:32+02 | UDP | 0 | 0

On the stdout I see the correct packets parse xenoeye:
xenoeye: Unknown field 148: 0x48 0x9e 0xaa 0x8c ; IPv4 src addr: 10.38.241.215; Src port: 54260; Input SNMP index: 3; IPv4 dst addr: 10.16.7.12; Dst port: 53; Output SNMP index: 4; Protocol: 17; Unknown field 176: 0x00 ; Unknown field 177: 0x00 ; Unknown field 225: 0x0a 0x26 0xf1 0xd7 ; Unknown field 226: 0x0a 0x10 0x07 0x0c ; Unknown field 227: 0xd3 0xf4 ; Unknown field 228: 0x00 0x35 ; Unknown field 233: 0x01 ; Unknown field 33002: 0x00 0x00 ; Unknown field 323: 0x00 0x00 0x01 0x87 0xbd 0xa8 0x86 0x28 ; Unknown field 152: 0x00 0x00 0x01 0x87 0xbd 0xa8 0x86 0x28 ; Unknown field 33000: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 ; Unknown field 33001: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 ; Unknown field 40000: 0x61 0x6e 0x61 0x2e 0x6c 0x65 0x6f 0x6e 0x40 0x69 0x74 0x2e 0x61 0x62 0x62 0x2e 0x63 0x6f 0x6d 0x00 ; *dev-ip: 10.39.6.108; *dev-id: 0, *rate: 256

Both files ...../ingress/mo.conf and .../egress/mo.conf are
{
"filter": "dst net ABB-nets",

    "debug": {
            "dump-flows": "none"
    },

    "fwm": [
            {
                    /* the total number of packets and bytes from our networks, throughout the monitored object */
                    "name": "all",
                    "fields": ["packets", "octets"],
                    "time": 300
            },
            {
                    /* src IP and number of packets/bytes per address */
                    "name": "by_src",
                    "fields": ["packets", "octets", "src host"],
                    "time": 300
            },
            {
                    /* protocol numbers and number of packets/bytes for each protocol */
                    "name": "by_proto",
                    "fields": ["packets", "octets", "proto"],
                    "time": 300
            },
            {
                    /* dst IP and number of bytes to each address */
                    "name": "by_dst",
                    "fields": ["packets", "octets", "dst host"],
                    "time": 300
            }
    ]

}
cat /var/lib/xenoeye/iplists/ABB-nets
10.0.0.0/8

[vmxdev]

Hi.
Looks like the template (and therefore the flow records) doesn't have fields with the number of packets and bytes.

I have unwrapped the flow info with a little more detail:

Unknown field 148: 0x48 0x9e 0xaa 0x8c ; flowId
IPv4 src addr: 10.38.241.215;
Src port: 54260; Input SNMP index: 3;
IPv4 dst addr: 10.16.7.12; Dst port: 53;
Output SNMP index: 4; Protocol: 17;
Unknown field 176: 0x00 ; icmpTypeIPv4
Unknown field 177: 0x00 ; icmpCodeIPv4
Unknown field 225: 0x0a 0x26 0xf1 0xd7 ; postNATSourceIPv4Address
Unknown field 226: 0x0a 0x10 0x07 0x0c ; postNATDestinationIPv4Address
Unknown field 227: 0xd3 0xf4 ; postNAPTSourceTransportPort
Unknown field 228: 0x00 0x35 ; postNAPTDestinationTransportPort
Unknown field 233: 0x01 ; firewallEvent
Unknown field 33002: 0x00 0x00 ;
Unknown field 323: 0x00 0x00 0x01 0x87 0xbd 0xa8 0x86 0x28 ; observationTimeMilliseconds
Unknown field 152: 0x00 0x00 0x01 0x87 0xbd 0xa8 0x86 0x28 ; flowStartMilliseconds
Unknown field 33000: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 ;
Unknown field 33001: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 ;
Unknown field 40000: 0x61 0x6e 0x61 0x2e 0x6c 0x65 0x6f 0x6e 0x40 0x69 0x74 0x2e 0x61 0x62 0x62 0x2e 0x63 0x6f 0x6d 0x00 ;

If you can add packets and bytes to the template, then everything should work.

[walterpoli]

Hi, excuse me in which file I can add packets and bytes ?

[vmxdev]

No, this is not configurable in the collector. This information must come from your device. In the flows that come from the device, there is no number of packets and bytes. Look at the fields that the collector receives - this information is not there, even in non-standard fields

[vmxdev]

Please, don't pay attention to the emoji reactions, it's a friend of mine having fun. I just don't know how to disable them

[vmxdev]

According to the set of fields, it looks like you have Cisco ASA or something similar.

It has a slightly non-standard netflow:
https://www.cisco.com/c/en/us/td/docs/security/asa/special/netflow/asa_netflow.html

We do not have such devices, therefore there is no support for them.

Even if you configure the export of the number of bytes (NF_F_FWD_FLOW_DELTA_BYTES/NF_F_REV_FLOW_DELTA_BYTES fields), we will still need to think about how to handle it all correctly.

If you have the time and the will, we can try to add support for such a netflow to the collector.

If not, you can try another collector, many have Cisco ASA support

[walterpoli]

hi, yes the device is cisco asa : in our environment we have many cisco asa and checkpoint : I also tried gonfcollector but there is no compatibility between binary nfcol and linux. If also this collector is not comatible, which netflow collector (open source) can you suggest me ? Il 2023-04-26 17:55 vmxdev ha scritto:

…

According to the set of fields, it looks like you have Cisco ASA or something similar. It has a slightly non-standard netflow: https://www.cisco.com/c/en/us/td/docs/security/asa/special/netflow/asa_netflow.html We do not have such devices, therefore there is no support for them. Even if you configure the export of the number of bytes (NF_F_FWD_FLOW_DELTA_BYTES/NF_F_REV_FLOW_DELTA_BYTES fields), we will still need to think about how to handle it all correctly. If you have the time and the will, we can try to add support for such a netflow to the collector. If not, you can try another collector, many have Cisco ASA support -- Reply to this email directly, view it on GitHub [1], or unsubscribe [2]. You are receiving this because you authored the thread.Message ID: ***@***.***> [ { ***@***.***": "http://schema.org", ***@***.***": "EmailMessage", "potentialAction": { ***@***.***": "ViewAction", "target": "#2 (comment)", "url": "#2 (comment)", "name": "View Issue" }, "description": "View this Issue on GitHub", "publisher": { ***@***.***": "Organization", "name": "GitHub", "url": "https://github.com" } } ] Links: ------ [1] #2 (comment) [2] https://github.com/notifications/unsubscribe-auth/ARJZ3EQZA2MRAIRD4LAPSK3XDFAQBANCNFSM6AAAAAAXMPYLVI

[vmxdev]

Ok, I read the specification more carefully. Looks like Cisco ASA should send octets information with different templates.

If so, then you should see other flows, i.e. the dump will contain some lines with "Unknown field 231: ...; Unknown field 232: ...".

If such lines exist, then we can modify the xenoeye a little, it will work, although with some oddities - for example, I still can't figure out how to get the number of packets in a flow.

It is better to choose a collector based on what you want to get. If just a simple reporting then probably any of the modern ones will do - https://github.com/akvorado/akvorado , https://github.com/netsampler/goflow2 etc

[vmxdev]

I think you have already found a Netflow collector that is more suitable for you.
However, I have added Cisco ASA fields for the number of octets and packets.

I got the field numbers from here:
pmacct/pmacct#502
phaag/nfdump#115

There are a couple of points left.

Instead of "octets" and "packets" fields in configs, you should use "ioctets/rocktets" and "ipackets/rpackets".
I still can't figure out what bi-directional flows are in Cisco ASA. But, most likely, if you try different combinations of "ioctets"/"rocktets"/etc in the reports, something will become more clear.

Without access to hardware (or at least to Netflow dumps), it's hard to do anything more