Skip to content

Commit 241f53f

Browse files
abdelrahman725abdelrahman725
committed
allen dataset: Create colab example with Allen demo dataset.
Fixes #102
1 parent 1b5de8c commit 241f53f

1 file changed

Lines changed: 319 additions & 0 deletions

File tree

‎examples/allen_example.ipynb‎

Lines changed: 319 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,319 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"metadata": {
6+
"id": "D0qUybd86M17"
7+
},
8+
"source": [
9+
"# Experanto experiment with Allen dataset\n",
10+
"\n",
11+
"This notebook downloads and exports allen dataset (using allen-exporter) and shows how to use experanto to interpolate the experiment generally and its individual modalities.\n",
12+
"\n",
13+
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/abdelrahman725/experanto/blob/allen-dataset-colab-example/examples/allen_example.ipynb)"
14+
]
15+
},
16+
{
17+
"cell_type": "markdown",
18+
"metadata": {
19+
"id": "2CfCOjHMumyj"
20+
},
21+
"source": [
22+
"# 1. Export allen dataset using allen-exporter\n",
23+
"\n",
24+
"## Important (for developers)\n",
25+
"\n",
26+
"Default Python version in colab is \"Python 3.12\", but `allensdk` (which allen-exporter uses to download the experiment data) does NOT work with that version. While you can simply change Runtime version to \"2025.07\" which supports Python 3.11.13, that \"2025.07\" version will no longer be supported on \"July 2026\" (after a year of its release). So for consistency, we stick to set up our environment using **Micromamba** (a minimal package and environment manager) which gives us Python 3.11 environment.\n",
27+
"\n",
28+
"All shell commands (only in step 1.) should be run using this command: `!micromamba run -n py311 command_name`\n",
29+
"\n",
30+
"\n",
31+
"See [colab supported Runtime versions](https://research.google.com/colaboratory/runtime-version-faq.html#)"
32+
]
33+
},
34+
{
35+
"cell_type": "markdown",
36+
"metadata": {
37+
"id": "kTCZQUCuYONY"
38+
},
39+
"source": [
40+
"## Setup Micromamba environment"
41+
]
42+
},
43+
{
44+
"cell_type": "code",
45+
"execution_count": null,
46+
"metadata": {
47+
"id": "fuipfJgeH1LW"
48+
},
49+
"outputs": [],
50+
"source": [
51+
"# Install Micromamba (fast, minimal implementation of Conda.)\n",
52+
"!curl -Ls https://micro.mamba.pm/api/micromamba/linux-64/latest | tar -xvj bin/micromamba\n",
53+
"!mv bin/micromamba /usr/local/bin/\n",
54+
"\n",
55+
"# Verify\n",
56+
"!micromamba --version\n",
57+
"\n",
58+
"# Create Conda environment with Python 3.11\n",
59+
"!micromamba create -y -n py311 python=3.11\n",
60+
"\n",
61+
"# Verify output is 3.11\n",
62+
"!micromamba run -n py311 python --version"
63+
]
64+
},
65+
{
66+
"cell_type": "code",
67+
"execution_count": null,
68+
"metadata": {
69+
"id": "XFeNUyGgfqup"
70+
},
71+
"outputs": [],
72+
"source": [
73+
"# Install allen-exporter inside Micromamba environment.\n",
74+
"!micromamba run -n py311 pip install git+https://github.com/sensorium-competition/allen-exporter.git \"setuptools<70\" matplotlib-inline"
75+
]
76+
},
77+
{
78+
"cell_type": "markdown",
79+
"metadata": {
80+
"id": "g9RLq5u0Ybne"
81+
},
82+
"source": [
83+
"## Download and export Allen dataset"
84+
]
85+
},
86+
{
87+
"cell_type": "code",
88+
"execution_count": null,
89+
"metadata": {
90+
"id": "aV9cSVZ8Dx42"
91+
},
92+
"outputs": [],
93+
"source": [
94+
"# Export Allen dataset\n",
95+
"\n",
96+
"# Run cell inside Micromamba\n",
97+
"%%bash\n",
98+
"micromamba run -n py311 python\n",
99+
"\n",
100+
"import allensdk\n",
101+
"import warnings\n",
102+
"from allen_exporter.exporter import multi_session_export\n",
103+
"\n",
104+
"# Verify allensdk is installed.\n",
105+
"print(\"Allensdk version : \", allensdk.__version__)\n",
106+
"print(\"\\nImport confirmed\\n\")\n",
107+
"\n",
108+
"\n",
109+
"# Suppress warnings because there are a lot of annoying user warnings when downloading stuff\n",
110+
"warnings.filterwarnings(\"ignore\")\n",
111+
"\n",
112+
"# Download (if /data doesn't already exist) and export Allen dataset,\n",
113+
"# this may take 1-2 mins. After export, data should be\n",
114+
"# available at /data\n",
115+
"cache, ids = multi_session_export(1, val_rate= 0.2, subsample_frac=1)\n"
116+
]
117+
},
118+
{
119+
"cell_type": "markdown",
120+
"metadata": {
121+
"id": "A0c5SqvZNG-w"
122+
},
123+
"source": [
124+
"# 2. Experanto"
125+
]
126+
},
127+
{
128+
"cell_type": "markdown",
129+
"metadata": {
130+
"id": "6u14ha2YeQK6"
131+
},
132+
"source": [
133+
"## Setup environment for experanto\n",
134+
"\n",
135+
"**Note:** After running the next cell, you may be prompted to \"Restart session\". Don't worry, the experiment `data/` folder (created in the previous step) persists across sessions."
136+
]
137+
},
138+
{
139+
"cell_type": "code",
140+
"execution_count": null,
141+
"metadata": {
142+
"id": "83TmjLzOvfs2"
143+
},
144+
"outputs": [],
145+
"source": [
146+
"# Install experanto in default colab environment\n",
147+
"!pip install experanto git+https://github.com/sensorium-competition/experanto.git"
148+
]
149+
},
150+
{
151+
"cell_type": "markdown",
152+
"metadata": {
153+
"id": "oPWIWZSMe7FS"
154+
},
155+
"source": [
156+
"## Load experiment"
157+
]
158+
},
159+
{
160+
"cell_type": "code",
161+
"execution_count": 12,
162+
"metadata": {
163+
"colab": {
164+
"base_uri": "https://localhost:8080/"
165+
},
166+
"id": "cb7BnYLIe_PO",
167+
"outputId": "b5d2067b-2703-4527-d0a1-29f07c5da276"
168+
},
169+
"outputs": [
170+
{
171+
"name": "stdout",
172+
"output_type": "stream",
173+
"text": [
174+
"Available experiment devices: dict_keys(['screen', 'treadmill', 'eye_tracker', 'responses'])\n"
175+
]
176+
}
177+
],
178+
"source": [
179+
"import os\n",
180+
"from experanto.experiment import Experiment\n",
181+
"\n",
182+
"parent_dir = './data/allen_data'\n",
183+
"try:\n",
184+
" # Compute path to the experiment dir dynamically.\n",
185+
" matches = [\n",
186+
" d for d in os.listdir(parent_dir) if d.startswith('experiment')\n",
187+
" and os.path.isdir(os.path.join(parent_dir, d))\n",
188+
" ]\n",
189+
" if len(matches) == 0:\n",
190+
" raise FileNotFoundError()\n",
191+
"\n",
192+
"except FileNotFoundError:\n",
193+
" raise FileNotFoundError(\"No experiment directory found. Make sure you run step 1 correctly\")\n",
194+
"\n",
195+
"experiment_dir = os.path.join(parent_dir, matches[0])\n",
196+
"\n",
197+
"# Load the exported experiment.\n",
198+
"exp = Experiment(experiment_dir)\n",
199+
"\n",
200+
"# See compatible modalities for the experiment\n",
201+
"print(\"Available experiment devices:\", exp.devices.keys())"
202+
]
203+
},
204+
{
205+
"cell_type": "markdown",
206+
"metadata": {
207+
"id": "D7nDlincrkF2"
208+
},
209+
"source": [
210+
"## Interpolation"
211+
]
212+
},
213+
{
214+
"cell_type": "markdown",
215+
"metadata": {
216+
"id": "C4sxqUEpxvS2"
217+
},
218+
"source": [
219+
"### Interpolate from all devices"
220+
]
221+
},
222+
{
223+
"cell_type": "code",
224+
"execution_count": null,
225+
"metadata": {
226+
"id": "-DR181uGrrgY"
227+
},
228+
"outputs": [],
229+
"source": [
230+
"import numpy as np\n",
231+
"\n",
232+
"# Query 100 time points spread evenly over 10 seconds\n",
233+
"times = np.linspace(0, 10, 100)\n",
234+
"\n",
235+
"# Get interpolated data from all devices\n",
236+
"data = exp.interpolate(times)\n",
237+
"\n",
238+
"for device, device_data in data.items():\n",
239+
" print(device, \": \")\n",
240+
" print(device_data)\n",
241+
" print(\"\\n--------------------\\n\")"
242+
]
243+
},
244+
{
245+
"cell_type": "markdown",
246+
"metadata": {
247+
"id": "tGrUCJWsx35k"
248+
},
249+
"source": [
250+
"### Interpolate from a specfic device"
251+
]
252+
},
253+
{
254+
"cell_type": "code",
255+
"execution_count": 20,
256+
"metadata": {
257+
"colab": {
258+
"base_uri": "https://localhost:8080/"
259+
},
260+
"id": "MXQbyba8x2ae",
261+
"outputId": "f1df9219-a39c-4876-922b-eabe988f5eeb"
262+
},
263+
"outputs": [
264+
{
265+
"name": "stdout",
266+
"output_type": "stream",
267+
"text": [
268+
"[[ 0.93657291 0.55576146 0.25939912 0.41124767 0.08977238 0.19020046\n",
269+
" 0.57376695 2.90015626 0.4239428 0.62480003 0.29607722 0.28595215]\n",
270+
" [ 0.58248562 0.74787319 0.30149347 0.19771612 0.07466755 0.02105861\n",
271+
" 0.41276214 0.96703732 0.60503817 0.13580583 0.08253972 0.18900922]\n",
272+
" [ 1.29600501 0.43970796 0.34024861 0.14388797 0.26002109 0.15665245\n",
273+
" 0.12951383 0.56966907 0.64048272 0.10336801 0.23056522 0.01171901]\n",
274+
" [ 0.84489775 -0.04886976 0.12116835 0.13635454 0.34542489 0.13520737\n",
275+
" 0.34949198 0.08619506 0.20034993 0.24432643 0.04468284 0.27774701]\n",
276+
" [ 1.18118751 0.2559959 0.05955438 0.02371915 0.22732048 0.24450895\n",
277+
" 0.53216243 -0.83702052 0.65515441 0.02353403 0.07673727 0.24609959]\n",
278+
" [ 1.27471638 0.10850338 0.18655133 0.06242622 0.14956538 0.00996919\n",
279+
" 0.09380771 0.16637991 -0.16068403 0.06777227 0.43875194 0.23804431]\n",
280+
" [ 1.39416265 0.10290431 -0.02241463 0.07999286 0.2551544 0.13212764\n",
281+
" 0.2290148 0.86402273 0.08026224 -0.17019738 -0.06684026 0.2208316 ]\n",
282+
" [ 1.48386037 0.19765739 0.05368509 -0.06401801 -0.03237728 0.16738932\n",
283+
" 0.14619565 1.96481538 0.17113581 -0.09779061 0.16421646 0.21277732]]\n"
284+
]
285+
}
286+
],
287+
"source": [
288+
"import numpy as np\n",
289+
"\n",
290+
"# Query 100 time points spread evenly over 10 seconds\n",
291+
"times = np.linspace(0, 10, 100)\n",
292+
"\n",
293+
"\n",
294+
"# device can be: screen, treadmill, eye_tracker, or responses\n",
295+
"screen = exp.interpolate(times, device=\"screen\")\n",
296+
"treadmill = exp.interpolate(times, device=\"treadmill\")\n",
297+
"eye_tracker = exp.interpolate(times, device=\"eye_tracker\")\n",
298+
"responses = exp.interpolate(times, device=\"responses\")\n",
299+
"\n",
300+
"# Change here what device you want to see its interpolated signals\n",
301+
"print(responses)"
302+
]
303+
}
304+
],
305+
"metadata": {
306+
"colab": {
307+
"provenance": []
308+
},
309+
"kernelspec": {
310+
"display_name": "Python 3",
311+
"name": "python3"
312+
},
313+
"language_info": {
314+
"name": "python"
315+
}
316+
},
317+
"nbformat": 4,
318+
"nbformat_minor": 0
319+
}

0 commit comments

Comments
 (0)