Supplemental figure - linear regression model

[chantelleleveille]

Goal:

The goal of this PR was to add a new linear regression model to our analysis. This branch is a paired down version of the features_LRM (draft) with just the code used to generate the figure elements.

Features:

There are new features that have been added in the context of the LRM

• Neighborhood average of feature over a tracks lifetime
• Feature at start of growth
• Normalization of sum of neighbor events

Analysis:

• linear_regeression.py is the script that contains the functions to run the LRM.
• supplemental_lrm_figure_workflow.py is the script to generate the plots for the supplemental figure.

Figures:

coefficients_delta_volume_BC_alpha_11.6.pdf
coefficients_duration_BC_alpha_0.8.pdf
delta_volume_BC_prediction_r_squared_matrix_alpha_0.pdf
duration_BC_prediction_r_squared_matrix_alpha_0.pdf
feature_correlation_clustermap.pdf

[cfrick13]

Found some bugs that need changing.

1. adjust exclude_outliers=False to be exclude_outliers=True
2. make get_early_transient_gr_of_neighborhood pulls features from the track itself by referencing dft instead of df

[chantelleleveille]

Thank you @cfrick13 for your careful review! Please re-review now that I have made the changes you caught and recommended.

I re-ran the workflow now that LRM 90 um radius neighborhood features have minimal filtering applied and ~early transient growth rate in 90 um radius is now track based on not whole colony. New figure outputs are attached.

The maximum alpha reached for tolerance of 0.5 changed for each prediction feature but the actual results don't change that much. See below and I am happy to discuss!!

coefficients_delta_volume_BC_alpha_10.2.pdf
coefficients_duration_BC_alpha_1.4.pdf
delta_volume_BC_prediction_r_squared_matrix_alpha_0.pdf
duration_BC_prediction_r_squared_matrix_alpha_0.pdf
feature_correlation_clustermap.pdf

[ritvikvasan]

Looks good to me!

[cfrick13]

the major issue I have this time is rounding of r2 values (and others) that then later get averaged. I strongly encourage changing this even though the errors/artifacts it causes could be very small and not noticeable...but its just not a good practice to round values before taking a mean (or doing any stats).