Re: update to SPW

[mvmaltitz]

Refer to email

Hello Danang,

There is a small update to the SPW model - see the attached narrative (its an internal white paper - it is long and designed for overall communication).
Basically... I do not want to utilize forecast days today + 10, 11, 12, 13 -- too much noise / inaccuracy. And... for SPW I am backing up 2 days...

So some steps - as I do not know your code at all, what you see below is how I adapted my R script - the key changes so to speak. You will have to interpret this for how you created the operational SPW model.

Not complicated changes - but i am being very careful to communicate to you what 'I changed' in my model...

First - once you pull the data which can go out 14 days (and you could likely just pull out today + 9 days - but I do know all the other things you might do with 14 day forecast!). Since I pull all 14 days, I have to trim those days out for use in the SPW rolling 11 and 8 day work.

Reading and initial processing

data_filename <- paste0("weatherDat/", data_filename)
df2 <- readr::read_csv(data_filename) %>%
  select(c(month_day, date, rainAccumulationSum, LTNPrecip, evapotranspirationSum, LTNPET))  

df2$date <- as.Date(df2$date, format = "%m/%d/%Y")

latest_date <- max(df2$date, na.rm = TRUE)
today <- latest_date - 13 #obviously TODAY - the zero forecast day
df2 <- df2 %>% filter(date <= (today + 9) ) # ADJUSTING to clip out forecast days 10 11, 12, 13
# this will change the number of 11 day totals later
# does not change # of 8 day - because I am backing up 2 days (5 historical not 3 as in current/original)
latest_date <- max(df2$date, na.rm = TRUE) #reset the latest date

Second: again in my code - because I sometimes pull a whole season's worth of data I 'trim' the file to make the counts of TRUE and FALSE
df2_datefiltered <- df2 %>%
filter(date >= (today - 5) & date <= (today + 9)) # ADJUSTING to last 5 days! Feb 2025
The original was today - 3 now go back 5 days - and out 9, not 13

Third: when you subset prior to counting the TRUE and FALSE (there should be only 8 rows in this df), we have changed the total number of possible models from 15 to 13. There are 8 30mm8day models and 5 40mm11day models thus 13 total for current and for LTN.
Thus

Define a date range for testing based on "today"

start_date <- today - 5 #This is where I constrain the 'counts of T or F'
end_date <- today + 2

Filter df2x to include only dates within the range

subset_df <- df2x[df2x$date >= start_date & df2x$date <= end_date, ]

Calculate counts of 'TRUE' occurrences for both LTN models and the current model

count_ltn8 <- sum(subset_df$LTNw8_30 == TRUE)
count_ltn11 <- sum(subset_df$LTNw11_40, na.rm = TRUE)
count_cur8 <- sum(subset_df$Cw8_30 == TRUE)
count_cur11 <- sum(subset_df$Cw11_40, na.rm = TRUE)

Sum the counts for LTN and current models respectively

count_ltn <- count_ltn8 + count_ltn11
count_cur <- count_cur8 + count_cur11

Calculate percentages of days with true conditions for LTN and current models

there are only

nearDaysLTNPercent <- (count_ltn / 13) * 100
nearDaysCurPercent <- (count_cur / 13) * 100

Last:
We are not changing the 'percentages' that define the message... We will be monitoring the performance etc.

Apologies in advance for the nature of this 'please adjust your SPW code' as I realize R and my code are not operational but I figured sending my code would actually make more sense to you than simply sending a narrative of the changes. Overall, we are moving the assessment days back 2 days (from today-3 to today -5) and we are chopping off the forecast days today +10, +11, 12, 13. There are likely other small changes you will/could make to keep your code clear as one of the variables printed out in the location by location report (which I assume you are NOT doing anyway) was days_f6to_f13 and now days_f6to_f9 etc. (there are / were a bunch of f13 variables... all f9 now with this change

Hope this all makes sense. We are planning to scale up - not sure how much - SPW for this upcoming march april may rainy season. More on this as things become more clear (we are in discussion with KALRO now).

Best,
John