Skip to content

Add ReversePredictivity (muzellec2026) benchmark#2304

Merged
KartikP merged 6 commits intobrain-score:masterfrom
S4b1n3:benchmark/muzellec2026
Mar 2, 2026
Merged

Add ReversePredictivity (muzellec2026) benchmark#2304
KartikP merged 6 commits intobrain-score:masterfrom
S4b1n3:benchmark/muzellec2026

Conversation

@S4b1n3
Copy link
Contributor

@S4b1n3 S4b1n3 commented Feb 18, 2026

Benchmark: Muzellec2026

Description

This benchmark evaluates model alignment with neural data using the Reverse Predictivity metric. We are using the initial benchmark MajajHong2015, but measuring how models can be predicted by neural data.

Components

  • Metric: Added Reverse Predictivity to regression correlation metrics.
  • Ceiling: In the regression Y is not noisy (model features) so we don't use internal consistency. But X is noisy so we introduced predictor_consistency metric, computing the internal consistency of the predictor, and use this as the ceiling.

Testing

  • All unit tests pass locally
  • Benchmark loads correctly
  • Ceiling is valid (0.88 ± 0.005)
  • Tested with alexnet: score = 0.10071875

Citation

@Article{muzellec2025reverse,
title={Reverse Predictivity: Going Beyond One-Way Mapping to Compare Artificial Neural Network Models and Brains},
author={Muzellec, Sabine and Kar, Kohitij},
journal={bioRxiv},
pages={2025--08},
year={2025},
publisher={Cold Spring Harbor Laboratory}

https://www.biorxiv.org/content/10.1101/2025.08.08.669382v1.full

Sabine Muzellec and others added 2 commits February 18, 2026 11:05
Using MajajHong2015 neural benchmark to compare with models
Uses reverse predictivity metric and new predictor consistency ceiling
Based on Muzellec & Kar 2026
@KartikP KartikP self-assigned this Feb 18, 2026
@KartikP
Copy link
Collaborator

KartikP commented Feb 19, 2026

PR looks good to me! Made two commits to clean up unnecessary pkl file and change to pyproject.toml.

Upon scoring infrastructure change, this PR will be merged. This will occur next week.

@KartikP KartikP closed this Mar 2, 2026
@KartikP KartikP reopened this Mar 2, 2026
@KartikP KartikP closed this Mar 2, 2026
@KartikP KartikP reopened this Mar 2, 2026
Copy link
Collaborator

@KartikP KartikP left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@KartikP KartikP merged commit e819b40 into brain-score:master Mar 2, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants