Difference between insight::get_predicted and ggeffects::ggpredict when calculating PI

[mroumet]

Hi,

I'm quite new and would like to apologise if this is not the correct place to post my question.
I'm currently trying to compute model prediction and associated prediction intervals. My model is a Poisson regression. I currently use 2 methods to derive the prediction intervals: insight::get_predicted and ggeffects::ggpredict. The two methods lead to quite different PI. Moreover the PI obtained when using ggpredict are extremely large (in some cases the upper limit > the maximum observed value). How is that possible? I'm missing something?

I get the same issue when using a very simple model (see code below). I would greatly appreciate some help.

Many thanks in advance,
Marie
A basic example

data("Salamanders")

m <- glm( count ~ mined , family = poisson(link = "log"), data = Salamanders)

d <- data_grid(m, terms = c("mined"))
Using insight::get_predicted

cbind(d,insight::get_predicted(mm, d, ci = 0.95,predict = "prediction") )

mined Predicted CI_low CI_high
1 yes 0.2954545 2.220446e-16 2
2 no 2.2648810 2.220446e-16 6
Using ggpredict

ggpredict(m, terms = c("mined"), interval = "prediction" )
mined | Predicted | 95% CI

yes | 0.30 | [0.01, 6.41]
no | 2.26 | [0.11, 48.83]

[strengejacke]

I agree, the way how prediction intervals are calculated in insight seems to be more appropriate. It also is very close to what you get via type = "simulate":

data("Salamanders", package = "glmmTMB")
m <- glm(count ~ mined, family = poisson(link = "log"), data = Salamanders)
ggeffects::ggpredict(m, terms = c("mined"), type = "simulate")
#> # Predicted counts of count
#> 
#> mined | Predicted |     95% CI
#> ------------------------------
#> yes   |      0.30 | 0.00, 1.97
#> no    |      2.27 | 0.00, 5.69

^{Created on 2024-02-13 with reprex v2.1.0}

[strengejacke]

Thanks, should be fixed in #442

[mroumet]

Many thanks for your rapid answer. Your solution seems to work perfectly on the “simple” example I provided.

However, I’m actually analyzing my data with a Poisson mixed model, fitted using glmmTMB. When I calculate the PIs, I (again) obtain very large PIs that look unrealistic (in some cases the upper limit > the maximum observed value). Unfortunately I can't calculate the PI using the insight package because it doesn't calculate PIs for glmmTMB models.

#The model looks like this
f <- anzahl_eizellen ~ protokoll2.factor * amh_cat + bmi + alter_frau + (1 | site/record_id)
m <- glmmTMB(f,family=poisson, data=data)

#Short description of the data:

<style> </style>

anzahl_eizellen	protokoll2.factor	amh_cat	bmi	alter_frau	site	record_id
Min.   : 0.0	P1    :1500	cat_1     :1345	Min.   :15.43	Min.   :21.00	site_1   :1284	27-1 :  12
1st Qu.: 1.0	P2    : 151	cat_2     :1067	1st Qu.:20.70	1st Qu.:34.00	site_2   : 597	27-7 :  12
Median : 2.0	P3    : 440	cat_3     :1963	Median :22.68	Median :37.00	site_3   : 535	28-12 :  12
Mean   : 3.6	P4    : 213	NA's      :  31	Mean   :23.87	Mean   :36.69	site_4   : 504	27-19:  11
3rd Qu.: 5.0	P5    : 160		3rd Qu.:25.61	3rd Qu.:40.00	site_5   : 497	27-24:  11
Max.   :35.0	P6    : 280		Max.   :46.66	Max.   :48.00	site_6   : 374	27-85 :  11
	P7    :1662		NA's   :10		(Other)   : 615	(Other) :4337

To obtain the predictions intervals I use

ggpredict(m, terms = c("protokoll2.factor","amh_cat"), interval = "prediction", type = "random" )

Example of the output for amh_cat: cat_3

protokoll2.factor | Predicted | 95% CI

P1 | 0.82 | 0.10, 6.51
P2 | 1.34 | 0.17, 10.78
P3 | 1.36 | 0.17, 10.82
P4 | 3.57 | 0.45, 28.45
P5 | 3.62 | 0.45, 28.89
P6 | 3.52 | 0.44, 28.01
P7 | 8.29 | 1.05, 65.77

#The maximum value observed for anzahl_eizellen. A PI of 1.05, 65.77 (i.e. for P7) above seems unrealistic to me.

If I'm using type = "simulate" I obtain more likely results (see below for amh_cat: cat_3). However I'm not 100% sure it is correct. Indeed I read that I should use type = "random" to obtain PI.
What do you think?

ggpredict(m, terms = c("protokoll2.factor","amh_cat"),interval = "prediction",type = "simulate" )

amh_cat: cat_3

protokoll2.factor | Predicted | 95% CI

NC-IVF | 0.92 | 0.00, 3.26
B | 1.54 | 0.00, 4.81
C | 1.56 | 0.00, 4.81
D | 4.03 | 0.07, 9.83
E | 4.08 | 0.11, 9.95
G | 3.90 | 0.05, 9.66
cIVF | 9.18 | 2.22, 20.00

Many thanks in advance for you imput and best,
Marie