Which dissimilarity measure to use for RSA after GLMSingle

[HilalNizamoglu]

Dear Kendrick,
After carefully reading previous discussion points, glmsingle wiki, and the relevant paper in eLife by Prince et al. 2022, I guess I still haven't found an answer to a question relevant to my current project. This is why I decided to raise a Q&A. Thanks in advance for your time and consideration.

In short, I would like to run an RSA on 48 conditions. Each of these conditions (~6sec videos) was only shown once per run and each subject underwent 8 runs. I am using rsatoolbox, and in many sources, the developers suggest using cross-mahalanobis distance (https://rsatoolbox.readthedocs.io/en/stable/distances.html) for fmri data while running RSA (for some such sources, please have a look at : https://youtu.be/AxbfqIBFTT8 ; https://doi.org/10.7554/eLife.82566 -page 18, under "Which distance estimator and RDM comparator?").

In Prince et al. 2022, I see that applying GLMsingle improves both voxel and subject-wise reliability. But I couldn't figure out with which distance measure the RDMs were built (https://github.com/jacob-prince/GLMsingle_paper/blob/main/figure5_RSA.ipynb). I initially wanted to apply cross-mahalanobis distance based on the before mentioned suggestions, however, considering the internal cross-validation in GLMsingle steps, I wasn't sure if applying any form of cross-validated distance measure including cv-euclidean or cv-mahalanobis would be correct. Furthermore, both mahalanobis and cross-mahalanobis distance measures rely on noise precision matrix that is derived from the residuals as calculated by SPM or nilearn GLM (please see https://rsatoolbox.readthedocs.io/en/stable/demo_fmri_nilearn.html ; https://rsatoolbox.readthedocs.io/en/stable/demo_fmri_spm.html ;https://rsatoolbox.readthedocs.io/en/stable/demo_dissimilarities.html).

My questions can be summarized as follows:
1- Is there any distance measure recommended to be used in RSA after applying GLMsingle?
2- Would applying cross-validation based distance measures (e.g. cv-euclidean, cv-mahalanobis) result in double-dipping or false claims/confounds?
3- Is it possible to calculate noise precision matrix (or just residuals like ResMS.mat in the case of default GLM from SPM) from the GLMSingle output?

Thanks again for your time and consideration.

Kind regards,
Hilal

[JAQuent]

Just FYI: rsatoolbox can get the noise matrix via estimating it from repeated presentations (see rsatoolbox.data.noise.prec_from_measurements()). Maybe this is already helpful.

[HilalNizamoglu]

Thank you, that helps with my third question!
Still though, I am mostly puzzled by the cross-validation issue and I am not sure if my concern is valid at all.

[kendrickkay]

The Prince 2022 RSM construction is described in the paper as follows: "we computed the Pearson correlation between the spatial response patterns from each pair of trials in the session, yielding a representational similarity matrix (RSM) where the temporal ordering of trials is preserved."

Re Q1: I think the choice of distance measure is more or less independent with respect to what GLMsingle provides. So it's up to you.

Re Q2: My best guess is that CV-based distance measures (after using GLMsingle) are not problematic. There is a FAQ entry that is most closely related to the issue you raise: https://glmsingle.readthedocs.io/en/latest/wiki.html#i-noticed-that-glmsingle-involves-some-internal-cross-validation-is-this-a-problem-for-decoding-style-analyses-where-we-want-to-divide-the-data-into-a-training-set-and-a-test-set Basically, although CV is used within GLMsingle, it is for fairly minor algorithmic hyperparameter settings, and I can't realistically see how GLMsingle would cause any substantial double-dipping. That being said, I would be curious if you investigate this and notice anything clearly off.

Re Q3: GLMsingle doesn't provide residuals (unfortunately), so the short answer is "not easily". Note that the GLM model that GLMsingle uses is fundamentally different in nature from SPM / nilearn GLM, having to do with the use of regularization (ridge regression). In addition, assuming the user takes the single-trial response estimates and uses them for subsequent analysis (including, for example, estimating noise matrix from repeated trial presentations), the noise properties are different --- in the "residuals" approach, there is extra time-series noise that is in theory removed by GLMsingle (via its provided estimated betas). So, it's sort of an apples-to-oranges comparison whether you are talking about noise via residuals of the time-series or noise inherent across repeated trial betas.

[HilalNizamoglu]

Thank you very much for your detailed answer! Sorry for missing the RSA measure in the paper, thanks for clarification.