Weewx +5(1)ish Deltas, Reset, Total Rain and sdr.py wxformulas.py

[gremlin205]

TLDR; Weather station reset rain. Weewx-sdr has aged against weewx. Weewx now provides (better?) handling of (rain) Deltas in wxformulas. Weewx-sdr didn't seem to not handle reset well - null rain/rainRate values. Commented out the Delta stanzas in sdr.py and using the weewx.conf driver section fixed my problem but I can't explain why. Maintainers, please consider deprecating the _calculate_delta function(s) in favor of those provided by weewx wxformulas.

The Problem: Two days ago my SainLogic weather station needed replacement batteries. After replacing the Rain_mm counter value reset. My weewx reported None values for Rain/RainRate. Via troubleshooting I noticed sdr.py and wxformulas.py had near identical delta handling functions. I added additional logging to see what the function values were. At the time the Rain delta never showed up in the log. After commenting out the sdr.py deltas and adding them to the weewx.conf the issue was corrected.

Main Questions:

1. Even though I am using the weewx.conf deltas the driver deltas took over. Is there anyway to force the issue?
2. How are Station Counter resets supposed to happen?
3. When I added a new delta it reset what ever and where ever the counter value was. Where are the delta counters stored?
4. Is there a procedure that you can reset counters for the SDR driver?

/etc/weewx/bin/user/sdr.py

    @staticmethod
    def _calculate_delta(label, newtotal, oldtotal):
        delta = None
        loginf("[sdr.py][_calculate_delta] label: %s new: %s old: %s" % (label, newtotal, oldtotal))
        if newtotal is not None and oldtotal is not None:
            if newtotal >= oldtotal:
                delta = newtotal - oldtotal
                loginf("[sdr.py][_calculate_delta] Delta: %s is new: %s old: %s" % (label, newtotal, oldtotal))
            else:
                loginf("[sdr.py][_calculate_delta] %s decrement ignored: new: %s old: %s" % (label, newtotal, oldtotal))
                loginf("[sdr.py][_calculate_delta] '%s' counter reset detected: new=%s old=%s" % (label, newtotal, oldtotal))
                delta = None
        return delta

/usr/share/weewx/weewx/wxformulas.py

def calculate_delta(newtotal, oldtotal, delta_key='rain'):
    """Calculate the differential given two cumulative measurements."""
    log.info("[wxformulas.py][calculate_delta] label: %s new: %s old: %s", delta_key,
                         newtotal, oldtotal)
    if newtotal is not None and oldtotal is not None:
        if newtotal >= oldtotal:
            delta = newtotal - oldtotal
            log.info("[wxformulas.py][calculate_delta] Delta: %s is new: %s old: %s", delta_key,
                                     newtotal, oldtotal)
        else:
            log.info("[wxformulas.py][calculate_delta] '%s' counter reset detected: new=%s old=%s", delta_key,
                     newtotal, oldtotal)
            delta = None
    else:
        delta = None
    return delta

No rain delta log:

2024-07-09T10:54:49.811121-05:00 weewxdev weewxd[4584]: INFO user.sdr: [SDR][_calculate_delta] label: strikes_total new: 182 old: 182
2024-07-09T10:54:49.811307-05:00 weewxdev weewxd[4584]: INFO user.sdr: [SDR][_calculate_delta] Delta: strikes_total is new: 182 old: 182
2024-07-09T10:54:49.837260-05:00 weewxdev weewxd[4584]: INFO user.sdr: [SDR][_calculate_delta] label: strikes_total new: 182 old: 182
2024-07-09T10:54:49.837414-05:00 weewxdev weewxd[4584]: INFO user.sdr: [SDR][_calculate_delta] Delta: strikes_total is new: 182 old: 182
2024-07-09T10:54:49.862886-05:00 weewxdev weewxd[4584]: INFO user.sdr: [SDR][_calculate_delta] label: rain_total new: 22.8 old: 22.8
2024-07-09T10:54:49.863042-05:00 weewxdev weewxd[4584]: INFO user.sdr: [SDR][_calculate_delta] Delta: rain_total is new: 22.8 old: 22.8

Log after making modifications (during rain event)

2024-07-09T10:39:21.238372-05:00 weewxdev weewxd[4584]: INFO user.sdr: [SDR][_calculate_delta] label: rain_total new: 18.9 old: 18.9
2024-07-09T10:39:21.238488-05:00 weewxdev weewxd[4584]: INFO user.sdr: [SDR][_calculate_delta] Delta: rain_total is new: 18.9 old: 18.9
2024-07-09T10:39:37.492391-05:00 weewxdev weewxd[4584]: INFO user.sdr: [SDR][_calculate_delta] label: strikes_total new: 182 old: 182
2024-07-09T10:39:37.492790-05:00 weewxdev weewxd[4584]: INFO user.sdr: [SDR][_calculate_delta] Delta: strikes_total is new: 182 old: 182
2024-07-09T10:39:37.560856-05:00 weewxdev weewxd[4584]: INFO user.sdr: [SDR][_calculate_delta] label: rain_total new: 19.2 old: 18.9
2024-07-09T10:39:37.561070-05:00 weewxdev weewxd[4584]: INFO user.sdr: [SDR][_calculate_delta] Delta: rain_total is new: 19.2 old: 18.9
2024-07-09T10:41:13.528023-05:00 weewxdev weewxd[4584]: INFO user.sdr: [SDR][_calculate_delta] label: rain_total new: 19.5 old: 19.2
2024-07-09T10:41:13.528642-05:00 weewxdev weewxd[4584]: INFO user.sdr: [SDR][_calculate_delta] Delta: rain_total is new: 19.5 old: 19.2
2024-07-09T10:41:13.557724-05:00 weewxdev weewxd[4584]: INFO user.sdr: [SDR][_calculate_delta] label: rain_total new: 19.5 old: 19.5
2024-07-09T10:41:13.557879-05:00 weewxdev weewxd[4584]: INFO user.sdr: [SDR][_calculate_delta] Delta: rain_total is new: 19.5 old: 19.5
2024-07-09T10:41:35.979854-05:00 weewxdev weewxd[4584]: INFO user.sdr: [SDR][_calculate_delta] label: strikes_total new: 182 old: 182
2024-07-09T10:41:35.980040-05:00 weewxdev weewxd[4584]: INFO user.sdr: [SDR][_calculate_delta] Delta: strikes_total is new: 182 old: 182

[matthewwall]

the functions wxformulas.calculate_delta and sdr.SDRDriver._calculate_delta do exactly the same thing. you should be able to define the differencing behavior in either SDR.deltas or StdWXCalculate, but probably not good to do both.

the drivers have always been responsible for handling rain counters; the formulas were added later to provide some consistency/continuity across differing hardware for other, typically derived quantities. later this was generalized. for example, the driver will know whether and when a specific instrument's counter wraps around, and if so it can handle that. the generic approach can only detect a decrement, log it, then (probably) reset the counting. (there have been cases where we saw counter decrements that were not actually resets, most often with the ws1080/ws2080 stations, but others too). the sdr driver is close to the hardware, further from it than most drivers, but still closer than the wxformulas.

anyway, not sure why the sdr delta did not handle your case. did you have both an sdr delta and wxcalculate specified at the same time, for the same observation?

i hesitate to remove the function from sdr.py, since that makes sdr.py incompatible with older weewx releases.

1. Even though I am using the weewx.conf deltas the driver deltas took over. Is there anyway to force the issue?

specify your own deltas in the SDR section. for example, this will disable SDR difference calculations:

[SDR]
    ...
    [[sensor_map]]
        ...
    [[deltas]]
        dummy = dummy_total

2. How are Station Counter resets supposed to happen?

when a counter decrements, the last total is set to None, so that the next total will be whatever the instrument next reports. so you will miss one interval of count. this is true for both the wxformulas and sdr implementations.

3. When I added a new delta it reset what ever and where ever the counter value was. Where are the
   delta counters stored?

in memory for sdr. i think it depends for wxformulas, since xtypes is capable of database queries, and it is the xtypes that invoke the wxformulas implementation.

4. Is there a procedure that you can reset counters for the SDR driver?

every time weewx restarts, the counters are reset. this is true for wxformulas and sdr, since both maintain the last state of the counter in memory.