Homeval analytics dashboard by Damonamajor · Pull Request #142 · ccao-data/homeval

Damonamajor · 2025-10-02T12:21:59Z

This adds a pin-val analytics dashboard.

Files are stored in the folder reports/analytics with a subfolder of data.
Queries are often multiple components, meaning that we group them by a matched pin (if found) and time with a one second lag. This is because there were queries which went over the second interval.
Browser parsing comes from the uaparserjs package. Although there is a good deal of commenting on the internet which says the methods are not particularly reliable.
I'm also having renv issues, so I don't think the new lock file is updated.

Damonamajor · 2025-10-03T08:08:29Z

@jeancochrane
Just an interesting CVS of one query output. I will have to spend a bit of time dissecting how to modify the output as the result of it. Each query seems to have multiple messages, both hitting and missing, and with different years.

jeancochrane

Awesome start! I'm only about halfway through, and haven't started looking at the charts yet, but I wanted to pause and give feedback on the data transformation so that we can focus on that first. Let me know if any of my comments or questions are unclear and I'm happy to talk them through.

jeancochrane · 2025-10-07T17:57:25Z

+  # Cluster ID is a way to identify duplicate requests within the same query.
+  # We build it on one second intervals.
+  # When there are two pins, we separate them.
+  # For these, we remove the NA observations since we don't know which category
+  # they belong to. Otherwise we leave in NA pins for relevant metadata.


[Suggestion, optional] Nice work identifying that we need to deduplicate clusters of requests. However, I think there's an opportunity to simplify this clustering quite a bit so that we don't need to tune our notion of a cluster ID.

Stepping back a bit, the primary reason that we experience clusters of requests is that each HomeVal report requests a bunch of static files every time a user loads the page. These static files should all have a Referer header pointing to the HomeVal report that requested them, and since we currently parse PINs from the Referer field, we get a duplicate PIN for each static file request.

If we implement my comment above to stop parsing pins from the Referer field, I think the particular issue where static file requests are causing duplicate counts for PINs will go away on its own. Once we exclude static files from our PIN parsing, I don't think we'll need to do any further refinement to get counts of requests by PIN. However, we'll still need to keep in mind that a majority of the raw count of requests (i.e. not counts of individual PINs) will be for static files, which is rarely interesting for analysis. Instead of clustering PINs, I think we should instead tag all static file requests so that we can group them separately for analysis. This should be pretty easy to do by just checking for rows whose URI stem starts with the string /static/.

I'll add more detail here about what we should do with these tagged static file requests once I review the charts below, so I can figure out exactly what grouping makes sense.

There's other values such as /apple-touch-icon-precomposed.png so I used starting with the four digit year, which seems like a more standardized metric (even though there may be able to be a misclicked 5 digit value, I'm not sure if that's possible).

Co-authored-by: Jean Cochrane <jeancochrane@users.noreply.github.com>

Damonamajor · 2025-10-17T14:27:40Z

I got nerdsniped by the duplicate rows, so once again, I have not yet looked at the charts 🙈 In the meantime, here are some more comments to keep you going! I'll review the charts tomorrow.

One quick chore to add: Did you get your renv environment fixed? If so, can you commit the following renv artifacts so that the environment is reproducible?

renv/.gitignore

renv/activate.R

.Rprofile

analytics.Rproj

`renv.lock

Yup, so we want to create a new folder environment for this?

jeancochrane · 2025-10-17T16:04:48Z

Yup, so we want to create a new folder environment for this?

No, I don't think so -- these R environment files should live in reports/analytics/.

jeancochrane

I think this is just about ready to go! Some small thoughts below, but nothing serious.

One thing I totally forgot about when writing the issue: It might be useful to get a rough sense of the distributions of the time-taken and time-to-first-byte fields, which measure the amount of time it takes for the server to complete specific actions. Would you be down to take a quick stab at adding simple charts for those two metrics? I think it'll be fine to combine both query + static requests, and maybe create a new section called something like "Request time" with two tabs, "Time to First Byte" (time-to-first-byte) and "Time To Last Byte" (time-taken). Does that make sense to you?

jeancochrane · 2025-10-21T21:05:45Z

+data_class <- data %>%
+  filter(uri_indicator == "Query") %>%
+  # Make sure we identify PINs which are incorrect or not joined to assessment_card
+  # For example PIN 14110010074923


[Thought, non-blocking] I do think the nulls are useful in general, since they give a rough count of PINs/requests that might be getting dropped in the join! I think it would be useful to roll it out to any charts that don't have it where you think it would be feasible.

Co-authored-by: Jean Cochrane <jeancochrane@users.noreply.github.com>

Damonamajor · 2025-10-24T20:50:21Z

  tidyr::unnest_wider(parsed, names_repair = "minimal") %>%
  mutate(
    pin = coalesce(
-      str_extract(`cs-uri-stem`, "(?<=/)\\d{14}(?=\\.html)"),


There were a couple pins which just didn't have the .html suffix.

Damonamajor · 2025-10-24T21:17:27Z

@jeancochrane @wrridgeway

Updating the renv to get out of the digest lock has created a cyclical lintr / styler issue. Don't know if we want to work on that here or in another issue.

Edit: I tried to manually revert the pre-commit-config.yaml but the digest error comes up when I do it.

jeancochrane

Awesome work! I think this dashboard is just about ready to go. Thanks for your patience during a few rounds of review as we narrowed down the specification.

Updating the renv to get out of the digest lock has created a cyclical lintr / styler issue. Don't know if we want to work on that here or in another issue.

I think I was able to figure out the problem in #143, though I don't think we should merge that PR ibecause the diff is weird. Here are the steps I took to resolve the pre-commit errors, try taking these steps on your PR to see if they work:

Update .lintr to disable return_linter, which raises an error by default in new versions of lintr (5d032e5)
Tweak some lines in the 2-PIN comparison report to fix indentation problems (37a3d3f)
Merge the main branch into your branch to get rid of deprecated R files that we removed in #141 that are now causing lintr errors (505b28d)

jeancochrane · 2025-10-24T22:47:33Z

+  mutate(
+    reason_clean = case_when(
+      is.na(reason_report_ineligible) ~ "Un-joined PIN",
+      TRUE ~ as.character(reason_report_ineligible)
+    )
+  ) %>%


[Suggestion, optional] As long as we're adjusting the definition of this column in the previous chart ("Type of Ineligibility Reason"), it might be worthwhile to factor it out in such a way that the three ineligibility charts can share it, perhaps by assigning the output to a new variable -- particularly since it seems like we may have missed this column in the third chart ("% Over Time). I'll leave it up to you to decide if you think that's worthwhile, though, or if you'd rather just change the 3 charts separately.

jeancochrane · 2025-10-24T22:48:23Z

+    # keep ineligible or null eligible cases.
+    # This allows us to find unjoined pins
+    is_report_eligible %in% c(FALSE, NA),
+    !is.na(pin) # Removes queries which do not have a PIN associated with it


[Question, non-blocking] Why remove these rows, given that we're including them in the "Type of Ineligibility Reason" chart above?

When I was looking at the html suffix, there were just two pins which didn't comply with it, one with a pin10 and one with hyphens. With that being said, I'll take out the line, you are right that it shouldn't be in.

If you are referring to this line
is_report_eligible %in% c(FALSE, NA),
it's because eligible pins have an NA value for the reason, so this separates the unjoined pins from the eligible pins as they are two distinct categories.

jeancochrane · 2025-10-24T22:49:04Z

+```{r}
+data %>%
+  filter(
+    !is.na(reason_report_ineligible),


[Question, non-blocking] Why remove unmatched PINs from this chart, given that we show them in the previous two charts?

Co-authored-by: Jean Cochrane <jeancochrane@users.noreply.github.com>

…board

Damonamajor · 2025-10-26T15:43:18Z

It also seems like something happened involving Schaumburg. Do we want do remove homeval scraper from the querys

jeancochrane

Nice work! This is just about ready to go. The scraper issue is making me realize that we might benefit from some controls that allow us to control the filters on the queries. I opened up #144 to track that effort, but in the meantime, I think this PR is ready to merge.

Co-authored-by: Jean Cochrane <jeancochrane@users.noreply.github.com>

Damonamajor added 3 commits October 2, 2025 07:29

initial push

a5751b9

1st draft

dba30c1

update panels and string detection

8d3ec2d

Damonamajor linked an issue Oct 2, 2025 that may be closed by this pull request

HomeVal analytics dashboard #139

Closed

Damonamajor added 3 commits October 2, 2025 12:33

lintr

67914a1

capitalize

87ef920

move some file locations

368f290

Damonamajor self-assigned this Oct 3, 2025

Damonamajor added 4 commits October 3, 2025 12:41

deduplicate observations

95c5878

update url detection

9b760f6

commenting

970e622

update eligibility logic

66c97eb

Damonamajor marked this pull request as ready for review October 6, 2025 09:45

Damonamajor commented Oct 6, 2025

View reviewed changes

Comment thread reports/analytics/analytics_dashboard.qmd

Damonamajor added 2 commits October 6, 2025 09:52

update coalesce logic

6812e2f

spellcheck

b29e9d1

Damonamajor commented Oct 6, 2025

View reviewed changes

Comment thread reports/analytics/analytics_dashboard.qmd

Damonamajor commented Oct 6, 2025

View reviewed changes

Comment thread reports/analytics/analytics_dashboard.qmd Outdated

Damonamajor commented Oct 6, 2025

View reviewed changes

Comment thread reports/analytics/analytics_dashboard.qmd

improve commenting

04c5989

jeancochrane self-requested a review October 6, 2025 16:02

jeancochrane requested changes Oct 7, 2025

View reviewed changes

Damonamajor and others added 7 commits October 8, 2025 09:53

Update reports/analytics/analytics_dashboard.qmd

5215388

Co-authored-by: Jean Cochrane <jeancochrane@users.noreply.github.com>

Update reports/analytics/analytics_dashboard.qmd

2ef38ad

Co-authored-by: Jean Cochrane <jeancochrane@users.noreply.github.com>

Update reports/analytics/analytics_dashboard.qmd

622523b

Co-authored-by: Jean Cochrane <jeancochrane@users.noreply.github.com>

Update reports/analytics/analytics_dashboard.qmd

b0e68ff

Co-authored-by: Jean Cochrane <jeancochrane@users.noreply.github.com>

Update reports/analytics/analytics_dashboard.qmd

b8458b1

Co-authored-by: Jean Cochrane <jeancochrane@users.noreply.github.com>

remove cluster logic

d682227

lintr

bc88b70

Update reports/analytics/analytics_dashboard.qmd

c2c831e

Co-authored-by: Jean Cochrane <jeancochrane@users.noreply.github.com>

create project

d8a1483

jeancochrane approved these changes Oct 21, 2025

View reviewed changes

Damonamajor and others added 4 commits October 22, 2025 10:28

Update reports/analytics/analytics_dashboard.qmd

003b9c4

Co-authored-by: Jean Cochrane <jeancochrane@users.noreply.github.com>

Update reports/analytics/analytics_dashboard.qmd

f2aff9e

Co-authored-by: Jean Cochrane <jeancochrane@users.noreply.github.com>

update yaml

8a1c7a1

Jean edits

bdaac3a

Damonamajor commented Oct 24, 2025

View reviewed changes

Damonamajor requested a review from jeancochrane October 24, 2025 21:16

Damonamajor commented Oct 24, 2025

View reviewed changes

Comment thread reports/analytics/analytics_dashboard.qmd

jeancochrane mentioned this pull request Oct 24, 2025

Pre-commit fix for 2-PIN comparison report lintr errors #143

Closed

jeancochrane approved these changes Oct 24, 2025

View reviewed changes

Damonamajor and others added 3 commits October 24, 2025 18:23

Update reports/analytics/analytics_dashboard.qmd

a463f94

Co-authored-by: Jean Cochrane <jeancochrane@users.noreply.github.com>

Update reports/analytics/analytics_dashboard.qmd

6709c9f

Co-authored-by: Jean Cochrane <jeancochrane@users.noreply.github.com>

Merge remote-tracking branch 'origin' into 139-homeval-analytics-dash…

d408e7d

…board

Damonamajor added 2 commits October 26, 2025 18:08

lintr

05ab376

Jean edits

de81507

Damonamajor requested a review from jeancochrane October 28, 2025 14:54

improve commenting

a87fadf

jeancochrane mentioned this pull request Oct 29, 2025

Add control parameters to analytics dashboard #144

Closed

jeancochrane reviewed Oct 29, 2025

View reviewed changes

Comment thread reports/analytics/analytics_dashboard.qmd Outdated

Comment thread reports/analytics/analytics_dashboard.qmd Outdated

Damonamajor and others added 3 commits October 29, 2025 12:21

Update reports/analytics/analytics_dashboard.qmd

6311e49

Co-authored-by: Jean Cochrane <jeancochrane@users.noreply.github.com>

Update reports/analytics/analytics_dashboard.qmd

36696e4

Co-authored-by: Jean Cochrane <jeancochrane@users.noreply.github.com>

lintr

09d4d84

Damonamajor merged commit 4bbcbc9 into main Oct 30, 2025
1 check passed

Damonamajor deleted the 139-homeval-analytics-dashboard branch October 30, 2025 14:38

Conversation

Damonamajor commented Oct 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Damonamajor commented Oct 3, 2025 • edited by jeancochrane Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jeancochrane left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Damonamajor commented Oct 17, 2025

Uh oh!

jeancochrane commented Oct 17, 2025

Uh oh!

jeancochrane left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Damonamajor Oct 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Damonamajor commented Oct 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

jeancochrane left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Damonamajor commented Oct 26, 2025

Uh oh!

jeancochrane left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Damonamajor commented Oct 2, 2025 •

edited

Loading

Damonamajor commented Oct 3, 2025 •

edited by jeancochrane

Loading

Damonamajor Oct 24, 2025 •

edited

Loading

Damonamajor commented Oct 24, 2025 •

edited

Loading