diff --git a/benchmarks/templates/benchmarks/tutorials/models/deepdive_2.html b/benchmarks/templates/benchmarks/tutorials/models/deepdive_2.html index bb7446c7..56d480fc 100644 --- a/benchmarks/templates/benchmarks/tutorials/models/deepdive_2.html +++ b/benchmarks/templates/benchmarks/tutorials/models/deepdive_2.html @@ -196,7 +196,62 @@
+ Brain-Score allows users to specify a region-layer mapping for models. This mapping assigns a specific layer from the model.py file + to a particular brain region, ensuring it remains fixed when evaluated across different benchmarks. +
++ We provide two methods for specifying a region-layer mapping: +
+
+1 model_registry['resnet50_tutorial'] = lambda: ModelCommitment(identifier='resnet50_tutorial',
+2 activations_model=get_model('resnet50_tutorial'),
+3 layers=get_layers('resnet50_tutorial'),
+4 region_layer_mapping={"V1": "layer1", "V2": "layer3", "V4": "layer4", "IT": "fc"})
+
+
+1 {
+2 "V1": "layer1",
+3 "V2": "layer3",
+4 "V4": "layer4",
+5 "IT": "fc"
+6 }
+
+ + While optional, we recommend specifying this mapping for models with layers designed to replicate specific brain regions. + If pre-committed, the layer names must match those returned by the get_layers() function in the model.py. +
++ Note: If a submission does not include a region_layer_mapping JSON file, the layer mapping will be automatically computed before scoring using + STANDARD_REGION_BENCHMARKS. +
++ This standard mappping assigns layers to each region independently, meaning: +
+ This intentional flexibility is particularly relevant for models with layers that were intended to replicate specific brain regions, as there is no guarantee that the layer names will match the region names unless pre-committed. + As such, for these types of models, we strongly recommend specifying a region-layer mapping. +
+The (optional) requirements.txt file is where you can add any requirements that your model needs (such as a specific version of a package or an external git @@ -213,7 +268,7 @@
You are almost done! If you were actually submitting a model, the final step prior to submission would be to run your model locally to ensure that everything is in working order. You can do this by first following the @@ -265,7 +320,7 @@
For models built using PyTorch, the torchsummary package can be utilized to get a summary of model information. Install it via pip if necessary