Setting up CAVEclient

Setting up CAVEclient#

The CAVEclient is the main way to programmatically access the electron microscopy data using Python.This system works for both MICrONS and V1DD.

The Connectome Annotation Versioning Engine (CAVE) is a suite of tools developed at the Allen Institute and Seung Lab to manage large connectomics data. In particular, the CAVEclient provides an interface to query the CAVE database for annotations such as synapses, as well as to get a variety of other kinds of information.

Installation#

To install caveclient, use pip: pip install caveclient.

Once you have installed caveclient, to use it you need to set up your user token in one of the following ways:

Setting up credentials#

To access the data programmatically, you need to set up a user token. This token is assigned by the server and functions as a both a username and password to access any aspect of the data. You will need to save this token to your computer using the tools.ou can m

Scenario 1: New User, No Previous Account#

If you have never interacted with CAVE before, you will need to both create an account and get a token.

A Google account (or Google-enabled account) is required to create a CAVE account.

Run the dataset specific code block from below and follow the set up instructions; enter the token first, then enter ‘y’ when prompted

MICrONS#

# Import packages
from caveclient import CAVEclient

# Token Generation and Setup
CAVEclient.setup_token("https://global.daf-apis.com")

To check if your setup works, run the following:

# Initialize CAVEclient
client = CAVEclient(datastack_name="minnie65_public")

V1DD#

# Import packages
from caveclient import CAVEclient

# Token Generation and Setup
CAVEclient.setup_token("https://global.em.brain.allentech.org")

To check if your setup works, run the following:

# Initialize CAVEclient
client = CAVEclient(datastack_name="v1dd_public")

If you don’t get any errors, your setup has worked and you can move on

Scenario 2: Existing user, new computer#

If you have already created an account and token but not set up your computer yet, you can use the same token on a new computer.

MICrONS#

Visit https://global.daf-apis.com/sticky_auth/settings/tokens and copy an existing token or create a new one.

Copy this token, and save it to your computer using the following:

client.auth.save_token(server_address="https://global.daf-apis.com", 
                       token=YOUR_TOKEN, 
                       overwrite=True)

Note that the token must be formatted as a string.

To check if your setup works, run the following:

client = CAVEclient('minnie65_public')

V1DD#

Visit https://global.em.brain.allentech.org/sticky_auth/settings/tokens and copy an existing token or create a new one.

Copy this token, and save it to your computer using the following:

client.auth.save_token(server_address="https://global.em.brain.allentech.org", 
                       token=YOUR_TOKEN, 
                       overwrite=True)

Note that the token must be formatted as a string.

To check if your setup works, run the following:

client = CAVEclient('v1dd_public')

If you don’t get any errors, your setup has worked and you can move on!

Something went wrong?#

If you are using an older version of CAVEclient, it may not have the same onboarding module. Use the following instead:

# MICrONS
client.auth.setup_token(server_address="https://global.em.brain.allentech.org", 
                        make_new=True)

# V1DD
client.auth.setup_token(server_address="https://global.daf-apis.com", 
                        make_new=True)

Then enter the token as in Scenario 2 above

If you are still having trouble with authentication or permissions, it is probably because the token you are trying to use is not the one CAVE expects.

To find your current token, run the following code in a Jupyter notebook:

client = CAVEclient()
client.auth.token

The notebook will print the token your computer is sending to the server.

If this token is not the one you find from running the code in Scenario 2, double check that it is pointing to the datastack you want (MICrONS or V1DD) and force regenerate a token for the appropriate datastack as above.

CodeOcean Token setup#

If you want to use CAVEclient in Code Ocean, you can add your CAVE authentication token to your environment so you do not have to enter it every time you spin up a capsule.

This becomes slightly more complicated if you want TWO tokens (say, for V1DD and MICrONS). Here are step-by-step instructions for adding two tokens

Create your CAVE token secret(s)#

  1. In Code Ocean, access your Account settings from the bottom left panel

../../_images/cave_code_ocean_0.png

Fig. 37 Access Account settings#

  1. Under your Account, open your ‘User Secrets’

../../_images/cave_code_ocean_1.png

Fig. 38 name: Access ‘User Secrets’#

  1. Add a new secret, from the menu on the upper right. Select ‘API Credentials’

../../_images/cave_code_ocean_2.png

Fig. 39 Access ‘User Secrets’#

  1. Create a CAVE Token for MICrONS

../../_images/cave_code_ocean_3.png

Fig. 40 Save MICrONS secret#

  1. Create a CAVE Token for V1DD

../../_images/cave_code_ocean_4.png

Fig. 41 Save V1DD secret#

Add the secret(s) to your capsule#

  1. Open the environment tab of your Capsule

../../_images/cave_code_ocean_5.png

Fig. 42 Environment tab of capsule#

  1. Scroll to the bottom, where you can ‘Add secret to capsule’

  • Add both the CAVE and V1DD credentials

../../_images/cave_code_ocean_6.png

Fig. 43 Add secret to capsule#

  1. Rename the default ‘Environment Variable Name’: click on the three-dot menu to the right

../../_images/cave_code_ocean_8.png

Fig. 44 Select 3-dot menu to set the Environment variable names#

  1. Rename the secret variable to something distinctive. Here: ‘API_SECRET_microns’ (key does not matter)

../../_images/cave_code_ocean_9.png

Fig. 45 Rename the API secret variable for easy look up in your code#

  1. Repeat for the other token. For example: `API_SECRET_v1dd’ (key does not matter)

Initialize CAVE with dataset-specific token#

  1. In a notebook or python file within your Capsule, import CAVEclient and OS

import os
`from caveclient import CAVEclient

  1. Initialize MICrONS datastack (same pattern every time)

client = CAVEclient("minnie65_public", auth_token=os.environ["API_SECRET_microns"])

  1. Initialize V1DD datastack

client = CAVEclient("v1dd_public", server_address="https://global.em.brain.allentech.org", auth_token=os.environ["API_SECRET_v1dd"])

  • Note: the addition of a server_adddress argument. This is only necessary the first time you spin up V1DD in your capsule session. But is necessary every time you restart the capsule

Contents