Provide an explicit state for the flow classification process

[IvanNardi]

Application should keep calling nDPI until flow state became
NDPI_STATE_CLASSIFIED.

The main loop in the application is simplified to something like:

res = ndpi_detection_process_packet(...);
if(res->state == NDPI_STATE_CLASSIFIED) {
  /* Done: you can get finale classification and all metadata.
     nDPI doesn't need more packets for this flow */
} else {
  /* nDPI needs more packets for this flow. The provided
     classification is not final and more metadata might be
     extracted.
     If `res->state` is `NDPI_STATE_PARTIAL`, partial/initial
     classification is available in `res->proto`
     as usual but it can be updated later.
  */
}

/*
    Example A (QUIC flow):
     pkt 1: proto QUIC state NDPI_STATE_PARTIAL
     pkt 2: proto QUIC/Youtube  state NDPI_STATE_CLASSIFIED
    Example B (GoogleMeet call):
     pkt 1:   proto STUN state NDPI_STATE_PARTIAL
     pkt N:   proto DTLS state NDPI_STATE_PARTIAL
     pkt N+M: proto DTLS/GoogleCall state NDPI_STATE_CLASSIFIED
    Example C (standard TLS flow):
     pkt 1:   proto Unknown state NDPI_STATE_INSPECTING
     pkt 2:   proto Unknown state NDPI_STATE_INSPECTING
     pkt 3:   proto Unknown state NDPI_STATE_INSPECTING
     pkt 4:   proto TLS/Facebook state NDPI_STATE_PARTIAL
     pkt N:   proto TLS/Facebook state NDPI_STATE_CLASSIFIED
 */
}

You can take a look at ndpiReader for a slightly more complex example.

API changes:

• remove the third parameter from ndpi_detection_giveup(). If you need
  to know if the classification flow has been guessed, you can access
  flow->protocol_was_guessed
• remove ndpi_extra_dissection_possible()
• change some prototypes from accepting ndpi_protocol foo to
  ndpi_master_app_protocol bar. The update is trivial: from foo to
  foo.proto

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[mmanoj]

@IvanNardi useful feature, I'm currently developing a Flow State Machine (FSM) with an accumulator mechanism to optimize how packets are sent to nDPI based on protocol characteristics. For example, only a single packet is sent for DNS flows, while TLS flows require at least 6–8 packets before classification.

The goal is to prevent the application from continuously invoking nDPI until the flow state reaches NDPI_STATE_CLASSIFIED, which should significantly reduce CPU cycles and improve performance on hot paths.

If this approach sounds reasonable, I’ll be happy to share the implementation details and results once it’s completed. Please share your thoughts or suggestions.

[IvanNardi]

@IvanNardi useful feature, I'm currently developing a Flow State Machine (FSM) with an accumulator mechanism to optimize how packets are sent to nDPI based on protocol characteristics. For example, only a single packet is sent for DNS flows, while TLS flows require at least 6–8 packets before classification.

The goal is to prevent the application from continuously invoking nDPI until the flow state reaches NDPI_STATE_CLASSIFIED, which should significantly reduce CPU cycles and improve performance on hot paths.

If this approach sounds reasonable, I’ll be happy to share the implementation details and results once it’s completed. Please share your thoughts or suggestions.

Let me be clear about nDPI capabilities: libnDPI stops processing flow packets when it gets all the information (classification and metadata) *** required by the application***. That means:

• if you configure nDPI to avoid DNS metadata, processing stops after the first DNS request
• if you are interested only in (sub-)classification, TLS processing (usually) stops after the 4th packets (i.e. after the Client Hello)
• if you are not interested in metadata, STUN (i.e. Whatapp/Signal/Telegram/Messenger calls) stops after (usually) the first packet

In other words, nDPI already processes the minimum number of packets depending on the configuration; with the default configuration we usually want to get everything but you can tune that depending on your usercase.

It is a tradeoff between performance and information extracted.

All of that was true even before this PR has been merged.

Are you doing something different, @mmanoj ?