Add functionality to graph DHCP stats on each VLAN page and Prefix page in NAV

[jorund1]

Scope and purpose

Fixes #2373. Another PR intends to add documentation and release notes.

This pull request

• Adds DHCP stats graphs to VLAN and Prefix pages in NAV where recent-enough stats for that VLAN/Prefix are found in the Carbon/Graphite database:
  

• Changes how paths are stored under "nav.dhcp" in the Carbon/Graphite database:

  • previously, paths followed this pattern:

    nav.dhcp.4.pools.<server-name>.<pool-name>.<first-ip>.<last-ip>.{assigned,total,declined}
    

  • now, paths follow this pattern:

    nav.dhcp.4.<server-name>.{range,pool,subnet}.{special_groups,custom_groups}.<group-name>.<first-ip>.<last-ip>.{assigned,total,declined}
    

    Basically, we
    • Rename <pool-name> to <group-name> in all pieces of documentation.
    • Differentiate between a range, a pool, and a subnet, because different DHCP servers have different semantics and data-coarseness. A range is the set of all IPs between a first-ip and a last-ip, whereas a pool is an arbitrary set of IPs greater than first-ip and less than last-ip. Further, all IPs in a range and a pool are available for lease, whereas this is not necessarily the case for a subnet.
    • Add an additional segment {special_groups,custom_groups} to differentiate between group-names that NAV must create and manage itself and group-names that NAV is able to infer from some piece of externally sourced information such as DHCP server config files.

• Expects [server_<server-name>] sections in dhcpstats.conf instead of [endpoint_<server-name>] for consistency

  • The new code has backwards-compability with the old naming-scheme

• Changes the name of the user_context_poolname_key option in dhcpstats.conf to user_context_groupname_key.

  • The new code does not have backwards-compability with the old naming-scheme here because there is already breaking changes to Graphite paths. Also, NAV works fine when this option is absent.

Contributor Checklist

Every pull request should have this checklist filled out, no matter how small it is.
More information about contributing to NAV can be found in the
Hacker's guide to NAV.

• Added a changelog fragment for towncrier
• Added/amended tests for new/changed code
• Added/changed documentation See independent PR
• Linted/formatted the code with ruff, easiest by using pre-commit
• Wrote the commit message so that the first line continues the sentence "If applied, this commit will ...", starts with a capital letter, does not end with punctuation and is 50 characters or less long. See https://cbea.ms/git-commit/
• Based this pull request on the correct upstream branch: For a patch/bugfix affecting the latest stable version, it should be based on that version's branch (<major>.<minor>.x). For a new feature or other additions, it should be based on master.
• If applicable: Created new issues if this PR does not fix the issue completely/there is further work to be done TODO: add a new issue for using Rickshaw graphs instead of PNG images in graph navlets
• If it's not obvious from a linked issue, describe how to interact with NAV in order for a reviewer to observe the effects of this change first-hand (commands, URLs, UI interactions) See screencast below
• If this results in changes in the UI: Added screenshots of the before and after
• If this adds a new Python source code file: Added the boilerplate header to that file

Testing It For Yourself

You can run this script to fill the prefixes 192.0.2.0/24 and 198.51.100.0/24 with DHCP stats (you might need to install socat and gawk, and you might need to change 127.0.0.1 to the address of your Carbon/Graphite database):

#!/bin/sh

cat <<EOF |
kea-trd staff 192.0.2.1 192.0.2.16 16
kea-trd staff 192.0.2.17 192.0.2.64 48
kea-trd staff 192.0.2.65 192.0.2.126 64
kea-trd guest 192.0.2.200 192.0.2.255 56
kea-trd guest 198.51.100.1 198.51.100.255 255
kea-osl staff 198.51.100.1 198.51.100.255 255
EOF

gawk '
BEGIN {
  NOW = systime()
  INTERVAL = 300
  DAYS = 30
}

{
  a = rand()
  b = rand()
  c = rand()
  _ = 1 / (a + b + c)
  a = a * _
  b = b * _
  c = c * _
  d = rand()*3.14*2
  e = 20 + rand()*8
  f = rand()

  server_name = $1
  group_name = $2
  first_ip = $3; gsub(/\./, "_", first_ip)
  last_ip = $4; gsub(/\./, "_", last_ip)
  total_ips = $5
  noise = rand()
  for (offset = 0; offset < 3600*24*DAYS; offset += INTERVAL) {
    sine = (sin((offset*3.14*2)/(3600*e) + d)+1)/2
    noise = noise*f + rand()*(1-f)
    assigned_ips = int(sine*total_ips*a+total_ips*b+noise*total_ips*c)
    printf("nav.dhcp.4.%s.range.custom_groups.%s.%s.%s.assigned %d %d\n", server_name, group_name, first_ip, last_ip, assigned_ips, NOW - offset)
    printf("nav.dhcp.4.%s.range.custom_groups.%s.%s.%s.total %d %d\n", server_name, group_name, first_ip, last_ip, total_ips, NOW - offset)
  }
}' |

socat - tcp:127.0.0.1:2003

Here's a screencast:

screencast.mp4

At 2:00 in the video, notice that the graph labelled DHCP ranges in 'guest' on server 'kea-trd' has stats from ranges in both 192.0.2.0/24 and 198.51.100.0/24 despite vlan 20 only containing 192.0.2.0/24. This is because whenever a group (here: 'guest' on server 'kea-trd') has at least one range/pool/subnet overlapping with the VLAN/Prefix, the whole group will be displayed on that VLAN/Prefix's page.

[codecov]

Codecov Report

❌ Patch coverage is 81.66667% with 44 lines in your changes missing coverage. Please review.
✅ Project coverage is 62.74%. Comparing base (34c2f3b) to head (50cb13a).
⚠️ Report is 3 commits behind head on master.

Files with missing lines	Patch %	Lines
python/nav/dhcpstats/common.py	77.16%	29 Missing ⚠️
python/nav/bin/dhcpstats.py	41.66%	7 Missing ⚠️
python/nav/metrics/graphs.py	75.00%	4 Missing ⚠️
python/nav/dhcpstats/kea_dhcp.py	93.93%	2 Missing ⚠️
python/nav/web/info/prefix/views.py	88.88%	1 Missing ⚠️
python/nav/web/info/vlan/views.py	85.71%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3633      +/-   ##
==========================================
+ Coverage   62.67%   62.74%   +0.06%     
==========================================
  Files         611      612       +1     
  Lines       45102    45290     +188     
  Branches       43       43              
==========================================
+ Hits        28268    28416     +148     
- Misses      16824    16864      +40     
  Partials       10       10              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[sonarqubecloud]

Quality Gate failed

Failed conditions
C Reliability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE