Quick start guide#
Tutorial overview#
A short introduction to the Visual Coding Neuropixels data and SDK. For more information, see the full example notebook.
This tutorial will demonstrate how to make the following:
import os
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline
from allensdk.brain_observatory.ecephys.ecephys_project_cache import EcephysProjectCache
/opt/envs/allensdk/lib/python3.10/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
from .autonotebook import tqdm as notebook_tqdm
The EcephysProjectCache
is the main entry point to the Visual Coding Neuropixels dataset. It allows you to download data for individual recording sessions and view cross-session summary information.
# Example cache directory path, it determines where downloaded data will be stored
output_dir = '/root/capsule/data/allen-brain-observatory/visual-coding-neuropixels/ecephys-cache/'
manifest_path = os.path.join(output_dir, "manifest.json")
cache = EcephysProjectCache.from_warehouse(manifest=manifest_path)
print(cache.get_all_session_types())
['brain_observatory_1.1', 'functional_connectivity']
This dataset contains sessions in which two sets of stimuli were presented. The "brain_observatory_1.1"
sessions are (almost exactly) the same as Visual Coding 2P sessions.
sessions = cache.get_session_table()
brain_observatory_type_sessions = sessions[sessions["session_type"] == "brain_observatory_1.1"]
brain_observatory_type_sessions.tail()
published_at | specimen_id | session_type | age_in_days | sex | full_genotype | unit_count | channel_count | probe_count | ecephys_structure_acronyms | |
---|---|---|---|---|---|---|---|---|---|---|
id | ||||||||||
773418906 | 2019-10-03T00:00:00Z | 757329624 | brain_observatory_1.1 | 124.0 | F | Pvalb-IRES-Cre/wt;Ai32(RCL-ChR2(H134R)_EYFP)/wt | 546 | 2232 | 6 | [PPT, NOT, SUB, ProS, CA1, VISam, nan, APN, DG... |
791319847 | 2019-10-03T00:00:00Z | 769360779 | brain_observatory_1.1 | 116.0 | M | Vip-IRES-Cre/wt;Ai32(RCL-ChR2(H134R)_EYFP)/wt | 555 | 2229 | 6 | [APN, DG, CA1, VISam, nan, LP, TH, VISpm, POL,... |
797828357 | 2019-10-03T00:00:00Z | 776061251 | brain_observatory_1.1 | 107.0 | M | Pvalb-IRES-Cre/wt;Ai32(RCL-ChR2(H134R)_EYFP)/wt | 611 | 2232 | 6 | [PPT, MB, APN, NOT, HPF, ProS, CA1, VISam, nan... |
798911424 | 2019-10-03T00:00:00Z | 775876828 | brain_observatory_1.1 | 110.0 | F | Vip-IRES-Cre/wt;Ai32(RCL-ChR2(H134R)_EYFP)/wt | 825 | 2233 | 6 | [APN, TH, Eth, LP, DG, HPF, CA3, CA1, VISrl, n... |
799864342 | 2019-10-03T00:00:00Z | 772616823 | brain_observatory_1.1 | 129.0 | M | wt/wt | 604 | 2233 | 6 | [APN, POL, LP, DG, CA1, VISrl, nan, LGd, CA3, ... |
Peristimulus time histograms#
We are going to pick a session arbitrarily and download its spike data.
session_id = 791319847
session = cache.get_session_data(session_id)
WARNING:root:downloading a 2210.106MiB file from http://api.brain-map.org//api/v2/well_known_file_download/1026124645
Downloading: 0%| | 0.00/2.32G [00:00<?, ?B/s]
Downloading: 0%| | 5.64M/2.32G [00:00<00:40, 56.4MB/s]
Downloading: 0%| | 11.3M/2.32G [00:00<01:05, 35.2MB/s]
Downloading: 1%| | 21.4M/2.32G [00:00<00:39, 57.8MB/s]
Downloading: 1%|▏ | 31.6M/2.32G [00:00<00:31, 72.3MB/s]
Downloading: 2%|▏ | 41.5M/2.32G [00:00<00:28, 80.8MB/s]
Downloading: 2%|▏ | 51.7M/2.32G [00:00<00:25, 87.4MB/s]
Downloading: 3%|▎ | 62.1M/2.32G [00:00<00:24, 92.7MB/s]
Downloading: 3%|▎ | 72.4M/2.32G [00:00<00:23, 95.7MB/s]
Downloading: 4%|▎ | 82.7M/2.32G [00:01<00:22, 97.9MB/s]
Downloading: 4%|▍ | 93.1M/2.32G [00:01<00:22, 99.8MB/s]
Downloading: 4%|▍ | 103M/2.32G [00:01<00:22, 100MB/s]
Downloading: 5%|▍ | 114M/2.32G [00:01<00:21, 101MB/s]
Downloading: 5%|▌ | 124M/2.32G [00:01<00:21, 102MB/s]
Downloading: 6%|▌ | 134M/2.32G [00:01<00:21, 101MB/s]
Downloading: 6%|▌ | 144M/2.32G [00:01<00:21, 99.5MB/s]
Downloading: 7%|▋ | 154M/2.32G [00:01<00:21, 98.6MB/s]
Downloading: 7%|▋ | 164M/2.32G [00:01<00:21, 99.1MB/s]
Downloading: 8%|▊ | 175M/2.32G [00:01<00:21, 100MB/s]
Downloading: 8%|▊ | 185M/2.32G [00:02<00:21, 101MB/s]
Downloading: 8%|▊ | 196M/2.32G [00:02<00:20, 102MB/s]
Downloading: 9%|▉ | 206M/2.32G [00:02<00:20, 102MB/s]
Downloading: 9%|▉ | 216M/2.32G [00:02<00:20, 103MB/s]
Downloading: 10%|▉ | 227M/2.32G [00:02<00:20, 103MB/s]
Downloading: 10%|█ | 237M/2.32G [00:02<00:20, 103MB/s]
Downloading: 11%|█ | 248M/2.32G [00:02<00:20, 103MB/s]
Downloading: 11%|█ | 258M/2.32G [00:02<00:19, 103MB/s]
Downloading: 12%|█▏ | 268M/2.32G [00:02<00:19, 103MB/s]
Downloading: 12%|█▏ | 279M/2.32G [00:02<00:19, 103MB/s]
Downloading: 12%|█▏ | 289M/2.32G [00:03<00:19, 103MB/s]
Downloading: 13%|█▎ | 299M/2.32G [00:03<00:19, 103MB/s]
Downloading: 13%|█▎ | 310M/2.32G [00:03<00:19, 103MB/s]
Downloading: 14%|█▍ | 320M/2.32G [00:03<00:19, 103MB/s]
Downloading: 14%|█▍ | 330M/2.32G [00:03<00:19, 104MB/s]
Downloading: 15%|█▍ | 341M/2.32G [00:03<00:19, 103MB/s]
Downloading: 15%|█▌ | 351M/2.32G [00:03<00:18, 103MB/s]
Downloading: 16%|█▌ | 362M/2.32G [00:03<00:18, 104MB/s]
Downloading: 16%|█▌ | 372M/2.32G [00:03<00:18, 103MB/s]
Downloading: 16%|█▋ | 382M/2.32G [00:03<00:18, 103MB/s]
Downloading: 17%|█▋ | 393M/2.32G [00:04<00:18, 104MB/s]
Downloading: 17%|█▋ | 403M/2.32G [00:04<00:18, 104MB/s]
Downloading: 18%|█▊ | 414M/2.32G [00:04<00:18, 103MB/s]
Downloading: 18%|█▊ | 424M/2.32G [00:04<00:18, 102MB/s]
Downloading: 19%|█▊ | 434M/2.32G [00:04<00:18, 102MB/s]
Downloading: 19%|█▉ | 445M/2.32G [00:04<00:18, 103MB/s]
Downloading: 20%|█▉ | 455M/2.32G [00:04<00:18, 103MB/s]
Downloading: 20%|██ | 465M/2.32G [00:04<00:17, 103MB/s]
Downloading: 21%|██ | 476M/2.32G [00:04<00:17, 103MB/s]
Downloading: 21%|██ | 486M/2.32G [00:04<00:17, 104MB/s]
Downloading: 21%|██▏ | 497M/2.32G [00:05<00:17, 104MB/s]
Downloading: 22%|██▏ | 507M/2.32G [00:05<00:17, 104MB/s]
Downloading: 22%|██▏ | 517M/2.32G [00:05<00:17, 103MB/s]
Downloading: 23%|██▎ | 528M/2.32G [00:05<00:17, 104MB/s]
Downloading: 23%|██▎ | 538M/2.32G [00:05<00:17, 103MB/s]
Downloading: 24%|██▎ | 549M/2.32G [00:05<00:17, 104MB/s]
Downloading: 24%|██▍ | 559M/2.32G [00:05<00:17, 100MB/s]
Downloading: 25%|██▍ | 569M/2.32G [00:05<00:17, 101MB/s]
Downloading: 25%|██▌ | 580M/2.32G [00:05<00:17, 102MB/s]
Downloading: 25%|██▌ | 590M/2.32G [00:05<00:16, 102MB/s]
Downloading: 26%|██▌ | 600M/2.32G [00:06<00:18, 93.8MB/s]
Downloading: 26%|██▋ | 610M/2.32G [00:06<00:17, 96.1MB/s]
Downloading: 27%|██▋ | 621M/2.32G [00:06<00:17, 98.0MB/s]
Downloading: 27%|██▋ | 631M/2.32G [00:06<00:16, 99.5MB/s]
Downloading: 28%|██▊ | 641M/2.32G [00:06<00:16, 99.8MB/s]
Downloading: 28%|██▊ | 651M/2.32G [00:06<00:16, 100MB/s]
Downloading: 29%|██▊ | 661M/2.32G [00:06<00:16, 101MB/s]
Downloading: 29%|██▉ | 672M/2.32G [00:06<00:16, 99.2MB/s]
Downloading: 29%|██▉ | 682M/2.32G [00:06<00:16, 100MB/s]
Downloading: 30%|██▉ | 692M/2.32G [00:06<00:16, 101MB/s]
Downloading: 30%|███ | 702M/2.32G [00:07<00:15, 101MB/s]
Downloading: 31%|███ | 713M/2.32G [00:07<00:15, 102MB/s]
Downloading: 31%|███ | 723M/2.32G [00:07<00:15, 102MB/s]
Downloading: 32%|███▏ | 733M/2.32G [00:07<00:15, 102MB/s]
Downloading: 32%|███▏ | 743M/2.32G [00:07<00:15, 102MB/s]
Downloading: 33%|███▎ | 754M/2.32G [00:07<00:15, 102MB/s]
Downloading: 33%|███▎ | 764M/2.32G [00:07<00:15, 102MB/s]
Downloading: 33%|███▎ | 774M/2.32G [00:07<00:15, 102MB/s]
Downloading: 34%|███▍ | 784M/2.32G [00:07<00:15, 102MB/s]
Downloading: 34%|███▍ | 794M/2.32G [00:07<00:15, 101MB/s]
Downloading: 35%|███▍ | 805M/2.32G [00:08<00:14, 102MB/s]
Downloading: 35%|███▌ | 815M/2.32G [00:08<00:14, 102MB/s]
Downloading: 36%|███▌ | 825M/2.32G [00:08<00:14, 102MB/s]
Downloading: 36%|███▌ | 836M/2.32G [00:08<00:14, 102MB/s]
Downloading: 37%|███▋ | 846M/2.32G [00:08<00:14, 102MB/s]
Downloading: 37%|███▋ | 856M/2.32G [00:08<00:14, 102MB/s]
Downloading: 37%|███▋ | 867M/2.32G [00:08<00:14, 103MB/s]
Downloading: 38%|███▊ | 877M/2.32G [00:08<00:14, 102MB/s]
Downloading: 38%|███▊ | 887M/2.32G [00:08<00:13, 102MB/s]
Downloading: 39%|███▊ | 897M/2.32G [00:08<00:13, 102MB/s]
Downloading: 39%|███▉ | 908M/2.32G [00:09<00:13, 102MB/s]
Downloading: 40%|███▉ | 918M/2.32G [00:09<00:13, 102MB/s]
Downloading: 40%|████ | 928M/2.32G [00:09<00:13, 102MB/s]
Downloading: 40%|████ | 938M/2.32G [00:09<00:13, 102MB/s]
Downloading: 41%|████ | 949M/2.32G [00:09<00:13, 102MB/s]
Downloading: 41%|████▏ | 959M/2.32G [00:09<00:13, 102MB/s]
Downloading: 42%|████▏ | 970M/2.32G [00:09<00:13, 103MB/s]
Downloading: 42%|████▏ | 980M/2.32G [00:09<00:12, 104MB/s]
Downloading: 43%|████▎ | 990M/2.32G [00:09<00:12, 103MB/s]
Downloading: 43%|████▎ | 1.00G/2.32G [00:09<00:12, 103MB/s]
Downloading: 44%|████▎ | 1.01G/2.32G [00:10<00:12, 103MB/s]
Downloading: 44%|████▍ | 1.02G/2.32G [00:10<00:12, 103MB/s]
Downloading: 45%|████▍ | 1.03G/2.32G [00:10<00:12, 102MB/s]
Downloading: 45%|████▍ | 1.04G/2.32G [00:10<00:12, 102MB/s]
Downloading: 45%|████▌ | 1.05G/2.32G [00:10<00:12, 102MB/s]
Downloading: 46%|████▌ | 1.06G/2.32G [00:10<00:12, 102MB/s]
Downloading: 46%|████▋ | 1.07G/2.32G [00:10<00:12, 102MB/s]
Downloading: 47%|████▋ | 1.08G/2.32G [00:10<00:12, 102MB/s]
Downloading: 47%|████▋ | 1.09G/2.32G [00:10<00:11, 102MB/s]
Downloading: 48%|████▊ | 1.10G/2.32G [00:11<00:11, 103MB/s]
Downloading: 48%|████▊ | 1.11G/2.32G [00:11<00:11, 102MB/s]
Downloading: 49%|████▊ | 1.12G/2.32G [00:11<00:11, 103MB/s]
Downloading: 49%|████▉ | 1.13G/2.32G [00:11<00:11, 102MB/s]
Downloading: 49%|████▉ | 1.14G/2.32G [00:11<00:11, 102MB/s]
Downloading: 50%|████▉ | 1.16G/2.32G [00:11<00:11, 102MB/s]
Downloading: 50%|█████ | 1.17G/2.32G [00:11<00:11, 102MB/s]
Downloading: 51%|█████ | 1.18G/2.32G [00:11<00:11, 102MB/s]
Downloading: 51%|█████ | 1.19G/2.32G [00:11<00:11, 102MB/s]
Downloading: 52%|█████▏ | 1.20G/2.32G [00:11<00:10, 103MB/s]
Downloading: 52%|█████▏ | 1.21G/2.32G [00:12<00:11, 100MB/s]
Downloading: 53%|█████▎ | 1.22G/2.32G [00:12<00:10, 101MB/s]
Downloading: 53%|█████▎ | 1.23G/2.32G [00:12<00:10, 101MB/s]
Downloading: 53%|█████▎ | 1.24G/2.32G [00:12<00:10, 102MB/s]
Downloading: 54%|█████▍ | 1.25G/2.32G [00:12<00:10, 101MB/s]
Downloading: 54%|█████▍ | 1.26G/2.32G [00:12<00:10, 101MB/s]
Downloading: 55%|█████▍ | 1.27G/2.32G [00:12<00:10, 102MB/s]
Downloading: 55%|█████▌ | 1.28G/2.32G [00:12<00:10, 102MB/s]
Downloading: 56%|█████▌ | 1.29G/2.32G [00:12<00:10, 102MB/s]
Downloading: 56%|█████▌ | 1.30G/2.32G [00:12<00:09, 102MB/s]
Downloading: 56%|█████▋ | 1.31G/2.32G [00:13<00:09, 103MB/s]
Downloading: 57%|█████▋ | 1.32G/2.32G [00:13<00:09, 103MB/s]
Downloading: 57%|█████▋ | 1.33G/2.32G [00:13<00:09, 103MB/s]
Downloading: 58%|█████▊ | 1.34G/2.32G [00:13<00:09, 103MB/s]
Downloading: 58%|█████▊ | 1.35G/2.32G [00:13<00:09, 103MB/s]
Downloading: 59%|█████▊ | 1.36G/2.32G [00:13<00:09, 103MB/s]
Downloading: 59%|█████▉ | 1.37G/2.32G [00:13<00:09, 103MB/s]
Downloading: 60%|█████▉ | 1.38G/2.32G [00:13<00:09, 103MB/s]
Downloading: 60%|██████ | 1.39G/2.32G [00:13<00:09, 102MB/s]
Downloading: 60%|██████ | 1.40G/2.32G [00:13<00:08, 102MB/s]
Downloading: 61%|██████ | 1.41G/2.32G [00:14<00:08, 102MB/s]
Downloading: 61%|██████▏ | 1.42G/2.32G [00:14<00:08, 102MB/s]
Downloading: 62%|██████▏ | 1.43G/2.32G [00:14<00:08, 102MB/s]
Downloading: 62%|██████▏ | 1.44G/2.32G [00:14<00:08, 102MB/s]
Downloading: 63%|██████▎ | 1.45G/2.32G [00:14<00:08, 102MB/s]
Downloading: 63%|██████▎ | 1.46G/2.32G [00:14<00:08, 102MB/s]
Downloading: 64%|██████▎ | 1.47G/2.32G [00:14<00:08, 99.6MB/s]
Downloading: 64%|██████▍ | 1.48G/2.32G [00:14<00:08, 95.8MB/s]
Downloading: 64%|██████▍ | 1.49G/2.32G [00:14<00:08, 97.6MB/s]
Downloading: 65%|██████▍ | 1.50G/2.32G [00:14<00:08, 99.4MB/s]
Downloading: 65%|██████▌ | 1.51G/2.32G [00:15<00:07, 101MB/s]
Downloading: 66%|██████▌ | 1.52G/2.32G [00:15<00:07, 101MB/s]
Downloading: 66%|██████▌ | 1.54G/2.32G [00:15<00:07, 102MB/s]
Downloading: 67%|██████▋ | 1.55G/2.32G [00:15<00:07, 99.2MB/s]
Downloading: 67%|██████▋ | 1.56G/2.32G [00:15<00:07, 99.7MB/s]
Downloading: 68%|██████▊ | 1.57G/2.32G [00:15<00:07, 100MB/s]
Downloading: 68%|██████▊ | 1.58G/2.32G [00:15<00:07, 101MB/s]
Downloading: 68%|██████▊ | 1.59G/2.32G [00:15<00:07, 102MB/s]
Downloading: 69%|██████▉ | 1.60G/2.32G [00:15<00:07, 102MB/s]
Downloading: 69%|██████▉ | 1.61G/2.32G [00:15<00:06, 102MB/s]
Downloading: 70%|██████▉ | 1.62G/2.32G [00:16<00:06, 103MB/s]
Downloading: 70%|███████ | 1.63G/2.32G [00:16<00:06, 103MB/s]
Downloading: 71%|███████ | 1.64G/2.32G [00:16<00:06, 103MB/s]
Downloading: 71%|███████ | 1.65G/2.32G [00:16<00:06, 103MB/s]
Downloading: 72%|███████▏ | 1.66G/2.32G [00:16<00:06, 103MB/s]
Downloading: 72%|███████▏ | 1.67G/2.32G [00:16<00:06, 103MB/s]
Downloading: 72%|███████▏ | 1.68G/2.32G [00:16<00:06, 103MB/s]
Downloading: 73%|███████▎ | 1.69G/2.32G [00:16<00:06, 103MB/s]
Downloading: 73%|███████▎ | 1.70G/2.32G [00:16<00:06, 103MB/s]
Downloading: 74%|███████▍ | 1.71G/2.32G [00:16<00:05, 103MB/s]
Downloading: 74%|███████▍ | 1.72G/2.32G [00:17<00:05, 103MB/s]
Downloading: 75%|███████▍ | 1.73G/2.32G [00:17<00:05, 103MB/s]
Downloading: 75%|███████▌ | 1.74G/2.32G [00:17<00:05, 103MB/s]
Downloading: 76%|███████▌ | 1.75G/2.32G [00:17<00:05, 104MB/s]
Downloading: 76%|███████▌ | 1.76G/2.32G [00:17<00:05, 104MB/s]
Downloading: 77%|███████▋ | 1.77G/2.32G [00:17<00:05, 104MB/s]
Downloading: 77%|███████▋ | 1.78G/2.32G [00:17<00:05, 104MB/s]
Downloading: 77%|███████▋ | 1.79G/2.32G [00:17<00:05, 104MB/s]
Downloading: 78%|███████▊ | 1.80G/2.32G [00:17<00:05, 102MB/s]
Downloading: 78%|███████▊ | 1.81G/2.32G [00:17<00:05, 99.4MB/s]
Downloading: 79%|███████▊ | 1.82G/2.32G [00:18<00:04, 100MB/s]
Downloading: 79%|███████▉ | 1.83G/2.32G [00:18<00:04, 101MB/s]
Downloading: 80%|███████▉ | 1.85G/2.32G [00:18<00:04, 102MB/s]
Downloading: 80%|████████ | 1.86G/2.32G [00:18<00:04, 102MB/s]
Downloading: 81%|████████ | 1.87G/2.32G [00:18<00:04, 103MB/s]
Downloading: 81%|████████ | 1.88G/2.32G [00:18<00:04, 103MB/s]
Downloading: 81%|████████▏ | 1.89G/2.32G [00:18<00:04, 103MB/s]
Downloading: 82%|████████▏ | 1.90G/2.32G [00:18<00:04, 103MB/s]
Downloading: 82%|████████▏ | 1.91G/2.32G [00:18<00:03, 103MB/s]
Downloading: 83%|████████▎ | 1.92G/2.32G [00:18<00:03, 104MB/s]
Downloading: 83%|████████▎ | 1.93G/2.32G [00:19<00:03, 104MB/s]
Downloading: 84%|████████▎ | 1.94G/2.32G [00:19<00:03, 104MB/s]
Downloading: 84%|████████▍ | 1.95G/2.32G [00:19<00:03, 104MB/s]
Downloading: 85%|████████▍ | 1.96G/2.32G [00:19<00:03, 103MB/s]
Downloading: 85%|████████▌ | 1.97G/2.32G [00:19<00:03, 103MB/s]
Downloading: 85%|████████▌ | 1.98G/2.32G [00:19<00:03, 88.2MB/s]
Downloading: 86%|████████▌ | 1.99G/2.32G [00:19<00:03, 92.4MB/s]
Downloading: 86%|████████▋ | 2.00G/2.32G [00:19<00:03, 85.1MB/s]
Downloading: 87%|████████▋ | 2.01G/2.32G [00:19<00:03, 90.0MB/s]
Downloading: 87%|████████▋ | 2.02G/2.32G [00:20<00:03, 93.2MB/s]
Downloading: 88%|████████▊ | 2.03G/2.32G [00:20<00:02, 96.1MB/s]
Downloading: 88%|████████▊ | 2.04G/2.32G [00:20<00:02, 94.8MB/s]
Downloading: 89%|████████▊ | 2.05G/2.32G [00:20<00:02, 97.2MB/s]
Downloading: 89%|████████▉ | 2.06G/2.32G [00:20<00:02, 98.9MB/s]
Downloading: 89%|████████▉ | 2.07G/2.32G [00:20<00:02, 98.2MB/s]
Downloading: 90%|████████▉ | 2.08G/2.32G [00:20<00:02, 99.5MB/s]
Downloading: 90%|█████████ | 2.09G/2.32G [00:20<00:02, 100MB/s]
Downloading: 91%|█████████ | 2.10G/2.32G [00:20<00:02, 101MB/s]
Downloading: 91%|█████████ | 2.11G/2.32G [00:20<00:02, 101MB/s]
Downloading: 92%|█████████▏| 2.12G/2.32G [00:21<00:01, 102MB/s]
Downloading: 92%|█████████▏| 2.13G/2.32G [00:21<00:01, 102MB/s]
Downloading: 92%|█████████▏| 2.14G/2.32G [00:21<00:01, 103MB/s]
Downloading: 93%|█████████▎| 2.15G/2.32G [00:21<00:01, 102MB/s]
Downloading: 93%|█████████▎| 2.16G/2.32G [00:21<00:01, 102MB/s]
Downloading: 94%|█████████▍| 2.17G/2.32G [00:21<00:01, 102MB/s]
Downloading: 94%|█████████▍| 2.18G/2.32G [00:21<00:01, 101MB/s]
Downloading: 95%|█████████▍| 2.19G/2.32G [00:21<00:01, 100MB/s]
Downloading: 95%|█████████▌| 2.20G/2.32G [00:21<00:01, 100MB/s]
Downloading: 96%|█████████▌| 2.21G/2.32G [00:21<00:01, 100MB/s]
Downloading: 96%|█████████▌| 2.22G/2.32G [00:22<00:00, 100MB/s]
Downloading: 96%|█████████▋| 2.23G/2.32G [00:22<00:00, 100MB/s]
Downloading: 97%|█████████▋| 2.25G/2.32G [00:22<00:00, 99.5MB/s]
Downloading: 97%|█████████▋| 2.25G/2.32G [00:22<00:00, 97.3MB/s]
Downloading: 98%|█████████▊| 2.27G/2.32G [00:22<00:00, 98.2MB/s]
Downloading: 98%|█████████▊| 2.28G/2.32G [00:22<00:00, 99.3MB/s]
Downloading: 99%|█████████▊| 2.29G/2.32G [00:22<00:00, 99.7MB/s]
Downloading: 99%|█████████▉| 2.30G/2.32G [00:22<00:00, 100MB/s]
Downloading: 99%|█████████▉| 2.31G/2.32G [00:22<00:00, 96.9MB/s]
Downloading: 100%|█████████▉| 2.32G/2.32G [00:23<00:00, 97.6MB/s]
Downloading: 100%|██████████| 2.32G/2.32G [00:23<00:00, 101MB/s]
/opt/envs/allensdk/lib/python3.10/site-packages/hdmf/utils.py:668: UserWarning: Ignoring cached namespace 'hdmf-common' version 1.1.3 because version 1.8.0 is already loaded.
return func(args[0], **pargs)
/opt/envs/allensdk/lib/python3.10/site-packages/hdmf/utils.py:668: UserWarning: Ignoring cached namespace 'core' version 2.2.2 because version 2.7.0 is already loaded.
return func(args[0], **pargs)
We can get a high-level summary of this session by accessing its metadata
attribute:
session.metadata
/opt/envs/allensdk/lib/python3.10/site-packages/hdmf/utils.py:668: UserWarning: Ignoring cached namespace 'hdmf-common' version 1.1.3 because version 1.8.0 is already loaded.
return func(args[0], **pargs)
/opt/envs/allensdk/lib/python3.10/site-packages/hdmf/utils.py:668: UserWarning: Ignoring cached namespace 'core' version 2.2.2 because version 2.7.0 is already loaded.
return func(args[0], **pargs)
/opt/envs/allensdk/lib/python3.10/site-packages/hdmf/utils.py:668: UserWarning: Ignoring cached namespace 'hdmf-common' version 1.1.3 because version 1.8.0 is already loaded.
return func(args[0], **pargs)
/opt/envs/allensdk/lib/python3.10/site-packages/hdmf/utils.py:668: UserWarning: Ignoring cached namespace 'core' version 2.2.2 because version 2.7.0 is already loaded.
return func(args[0], **pargs)
/opt/envs/allensdk/lib/python3.10/site-packages/hdmf/utils.py:668: UserWarning: Ignoring cached namespace 'hdmf-common' version 1.1.3 because version 1.8.0 is already loaded.
return func(args[0], **pargs)
/opt/envs/allensdk/lib/python3.10/site-packages/hdmf/utils.py:668: UserWarning: Ignoring cached namespace 'core' version 2.2.2 because version 2.7.0 is already loaded.
return func(args[0], **pargs)
/opt/envs/allensdk/lib/python3.10/site-packages/hdmf/utils.py:668: UserWarning: Ignoring cached namespace 'hdmf-common' version 1.1.3 because version 1.8.0 is already loaded.
return func(args[0], **pargs)
/opt/envs/allensdk/lib/python3.10/site-packages/hdmf/utils.py:668: UserWarning: Ignoring cached namespace 'core' version 2.2.2 because version 2.7.0 is already loaded.
return func(args[0], **pargs)
{'specimen_name': 'Vip-IRES-Cre;Ai32-417678',
'session_type': 'brain_observatory_1.1',
'full_genotype': 'Vip-IRES-Cre/wt;Ai32(RCL-ChR2(H134R)_EYFP)/wt',
'sex': 'M',
'age_in_days': 116.0,
'rig_equipment_name': 'NP.1',
'num_units': 555,
'num_channels': 2229,
'num_probes': 6,
'num_stimulus_presentations': 70931,
'session_start_time': datetime.datetime(2019, 1, 8, 13, 55, 1, tzinfo=tzoffset(None, -28800)),
'ecephys_session_id': 791319847,
'structure_acronyms': ['VISpm',
nan,
'LP',
'TH',
'DG',
'CA1',
'CA3',
'CA2',
'VISl',
'ZI',
'LGv',
'VISal',
'APN',
'POL',
'VISrl',
'VISam',
'LGd',
'ProS',
'SUB',
'VISp'],
'stimulus_names': ['spontaneous',
'gabors',
'flashes',
'drifting_gratings',
'natural_movie_three',
'natural_movie_one',
'static_gratings',
'natural_scenes',
'drifting_gratings_contrast']}
We can also take a look at how many units were recorded in each brain structure:
session.structurewise_unit_counts
VISp 93
CA1 85
VISrl 58
VISl 56
VISam 49
VISal 43
SUB 41
CA3 33
DG 32
VISpm 17
LGv 16
LP 9
LGd 8
TH 4
ZI 4
CA2 3
POL 3
ProS 1
Name: ecephys_structure_acronym, dtype: int64
Now that we’ve gotten spike data, we can create peristimulus time histograms.
presentations = session.get_stimulus_table("flashes")
units = session.units[session.units["ecephys_structure_acronym"] == 'VISp']
time_step = 0.01
time_bins = np.arange(-0.1, 0.5 + time_step, time_step)
histograms = session.presentationwise_spike_counts(
stimulus_presentation_ids=presentations.index.values,
bin_edges=time_bins,
unit_ids=units.index.values
)
histograms.coords
Coordinates:
* stimulus_presentation_id (stimulus_presentation_id) int64 3647 .....
* time_relative_to_stimulus_onset (time_relative_to_stimulus_onset) float64 ...
* unit_id (unit_id) int64 951061537 ... 951062679
mean_histograms = histograms.mean(dim="stimulus_presentation_id")
fig, ax = plt.subplots(figsize=(8, 8))
ax.pcolormesh(
mean_histograms["time_relative_to_stimulus_onset"],
np.arange(mean_histograms["unit_id"].size),
mean_histograms.T,
vmin=0,
vmax=1
)
ax.set_ylabel("unit", fontsize=24)
ax.set_xlabel("time relative to stimulus onset (s)", fontsize=24)
ax.set_title("peristimulus time histograms for VISp units on flash presentations", fontsize=24)
plt.show()
Decoding visual stimuli#
First, we need to extract spikes. We will do using EcephysSession.presentationwise_spike_times
, which returns spikes annotated by the unit that emitted them and the stimulus presentation during which they were emitted.
scene_presentations = session.get_stimulus_table("natural_scenes")
visp_units = session.units[session.units["ecephys_structure_acronym"] == "VISp"]
spikes = session.presentationwise_spike_times(
stimulus_presentation_ids=scene_presentations.index.values,
unit_ids=visp_units.index.values[:]
)
spikes
stimulus_presentation_id | unit_id | time_since_stimulus_presentation_onset | |
---|---|---|---|
spike_time | |||
5914.084307 | 51355 | 951061801 | 0.001665 |
5914.085207 | 51355 | 951062307 | 0.002565 |
5914.087241 | 51355 | 951062610 | 0.004598 |
5914.090107 | 51355 | 951061993 | 0.007465 |
5914.090407 | 51355 | 951062605 | 0.007765 |
... | ... | ... | ... |
8573.801313 | 68228 | 951061801 | 0.244631 |
8573.801613 | 68228 | 951062175 | 0.244931 |
8573.805379 | 68228 | 951062898 | 0.248698 |
8573.805546 | 68228 | 951061537 | 0.248865 |
8573.805846 | 68228 | 951062143 | 0.249165 |
1224681 rows × 3 columns
Next, we will convert these into a num_presentations X num_units matrix, which will serve as our input data.
spikes["count"] = np.zeros(spikes.shape[0])
spikes = spikes.groupby(["stimulus_presentation_id", "unit_id"]).count()
design = pd.pivot_table(
spikes,
values="count",
index="stimulus_presentation_id",
columns="unit_id",
fill_value=0.0,
aggfunc=np.sum
)
design
unit_id | 951061537 | 951061549 | 951061556 | 951061568 | 951061574 | 951061607 | 951061637 | 951061643 | 951061649 | 951061655 | ... | 951062587 | 951062600 | 951062605 | 951062610 | 951062647 | 951062679 | 951062808 | 951062833 | 951062854 | 951062898 |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
stimulus_presentation_id | |||||||||||||||||||||
51355 | 1 | 3 | 1 | 1 | 2 | 3 | 2 | 1 | 2 | 0 | ... | 15 | 0 | 15 | 5 | 11 | 5 | 0 | 0 | 2 | 1 |
51356 | 2 | 0 | 1 | 2 | 2 | 1 | 1 | 3 | 4 | 0 | ... | 10 | 0 | 10 | 3 | 16 | 3 | 1 | 0 | 2 | 0 |
51357 | 0 | 0 | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 0 | ... | 7 | 0 | 13 | 0 | 14 | 6 | 12 | 3 | 2 | 5 |
51358 | 0 | 2 | 1 | 3 | 4 | 1 | 0 | 2 | 1 | 2 | ... | 6 | 0 | 11 | 7 | 8 | 3 | 0 | 0 | 0 | 3 |
51359 | 1 | 0 | 1 | 2 | 2 | 1 | 0 | 1 | 0 | 2 | ... | 7 | 0 | 15 | 4 | 10 | 5 | 11 | 2 | 0 | 7 |
... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
68224 | 2 | 0 | 4 | 2 | 1 | 0 | 1 | 0 | 1 | 0 | ... | 2 | 0 | 11 | 16 | 3 | 0 | 4 | 0 | 0 | 1 |
68225 | 0 | 0 | 9 | 0 | 2 | 0 | 1 | 0 | 0 | 0 | ... | 5 | 0 | 13 | 5 | 5 | 0 | 3 | 0 | 0 | 5 |
68226 | 1 | 0 | 1 | 3 | 2 | 0 | 1 | 0 | 0 | 0 | ... | 15 | 0 | 12 | 2 | 10 | 0 | 4 | 0 | 1 | 5 |
68227 | 1 | 0 | 2 | 3 | 3 | 0 | 0 | 0 | 1 | 0 | ... | 9 | 1 | 4 | 2 | 5 | 0 | 1 | 0 | 1 | 7 |
68228 | 3 | 1 | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 0 | ... | 9 | 0 | 18 | 6 | 4 | 4 | 0 | 0 | 0 | 8 |
5950 rows × 93 columns
… with targets being the numeric identifiers of the images presented.
targets = scene_presentations.loc[design.index.values, "frame"]
targets
stimulus_presentation_id
51355 14.0
51356 0.0
51357 8.0
51358 106.0
51359 8.0
...
68224 35.0
68225 35.0
68226 73.0
68227 95.0
68228 95.0
Name: frame, Length: 5950, dtype: object
from sklearn import svm
from sklearn.model_selection import KFold
from sklearn.metrics import confusion_matrix
design_arr = design.values.astype(float)
targets_arr = targets.values.astype(int)
labels = np.unique(targets_arr)
accuracies = []
confusions = []
for train_indices, test_indices in KFold(n_splits=5).split(design_arr):
clf = svm.SVC(gamma="scale", kernel="rbf")
clf.fit(design_arr[train_indices], targets_arr[train_indices])
test_targets = targets_arr[test_indices]
test_predictions = clf.predict(design_arr[test_indices])
accuracy = 1 - (np.count_nonzero(test_predictions - test_targets) / test_predictions.size)
print(accuracy)
accuracies.append(accuracy)
confusions.append(confusion_matrix(y_true=test_targets, y_pred=test_predictions, labels=labels))
0.41428571428571426
0.4966386554621849
0.49747899159663866
0.5067226890756302
0.473109243697479
print(f"mean accuracy: {np.mean(accuracy)}")
print(f"chance: {1/labels.size}")
mean accuracy: 0.473109243697479
chance: 0.008403361344537815
imagewise performance#
mean_confusion = np.mean(confusions, axis=0)
fig, ax = plt.subplots(figsize=(8, 8))
img = ax.imshow(mean_confusion)
fig.colorbar(img)
ax.set_ylabel("actual")
ax.set_xlabel("predicted")
plt.show()
best = labels[np.argmax(np.diag(mean_confusion))]
worst = labels[np.argmin(np.diag(mean_confusion))]
fig, ax = plt.subplots(1, 2, figsize=(16, 8))
best_image = cache.get_natural_scene_template(best)
ax[0].imshow(best_image, cmap=plt.cm.gray)
ax[0].set_title("most decodable", fontsize=24)
worst_image = cache.get_natural_scene_template(worst)
ax[1].imshow(worst_image, cmap=plt.cm.gray)
ax[1].set_title("least decodable", fontsize=24)
plt.show()
WARNING:root:downloading a 1.028MiB file from http://api.brain-map.org//api/v2/well_known_file_download/950745916
Downloading: 0%| | 0.00/1.08M [00:00<?, ?B/s]
Downloading: 100%|██████████| 1.08M/1.08M [00:00<00:00, 22.4MB/s]
WARNING:root:downloading a 1.028MiB file from http://api.brain-map.org//api/v2/well_known_file_download/950745861
Downloading: 0%| | 0.00/1.08M [00:00<?, ?B/s]
Downloading: 100%|██████████| 1.08M/1.08M [00:00<00:00, 26.3MB/s]