Unable to fetch data from Platynereis Dataset

[kareef928]

Hey @schlegelp, I am having trouble pulling data from the Platynereis Dumerilii Larva Dataset.

Code I'm trying to run

import pymaid
rm = pymaid.CatmaidInstance(
    server="https://catmaid.jekelylab.ex.ac.uk/#",
    api_token=None,
    http_user=None,
    http_password=None,
)
skids = pymaid.get_skids_by_annotation(annotations="Bezares-Calderon_et_al_2018")

Output

INFO  : Global CATMAID instance set. Caching is ON. (pymaid)
Traceback (most recent call last):
  File "/Users/kareefullah/Desktop/NeuroData/neurodata/platy-data/scripts/get_platy_data.py", line 12, in <module>
    skids = pymaid.get_skids_by_annotation("Bezares-Calderon_et_al_2018")
  File "/Users/kareefullah/Library/Caches/pypoetry/virtualenvs/platy-data-EVeqgmAk-py3.9/lib/python3.9/site-packages/pymaid/cache.py", line 316, in wrapper
    res = function(*args, **kwargs)
  File "/Users/kareefullah/Library/Caches/pypoetry/virtualenvs/platy-data-EVeqgmAk-py3.9/lib/python3.9/site-packages/pymaid/fetch.py", line 2328, in get_skids_by_annotation
    pos_ids = get_annotation_id(pos_an,
  File "/Users/kareefullah/Library/Caches/pypoetry/virtualenvs/platy-data-EVeqgmAk-py3.9/lib/python3.9/site-packages/pymaid/cache.py", line 200, in wrapper
    res = undo_on_error(function)(*args, **kwargs)
  File "/Users/kareefullah/Library/Caches/pypoetry/virtualenvs/platy-data-EVeqgmAk-py3.9/lib/python3.9/site-packages/pymaid/cache.py", line 316, in wrapper
    res = function(*args, **kwargs)
  File "/Users/kareefullah/Library/Caches/pypoetry/virtualenvs/platy-data-EVeqgmAk-py3.9/lib/python3.9/site-packages/pymaid/fetch.py", line 1925, in get_annotation_id
    an_list = remote_instance.fetch(remote_annotation_list_url)
  File "/Users/kareefullah/Library/Caches/pypoetry/virtualenvs/platy-data-EVeqgmAk-py3.9/lib/python3.9/site-packages/pymaid/client.py", line 473, in fetch
    raise HTTPError('{} errors encountered: {}'.format(len(errors),
requests.exceptions.HTTPError: 1 errors encountered: 500 Server Error: Project matching query does not exist. for url: https://catmaid.jekelylab.ex.ac.uk/1/annotations/

I believe these queries should work because from the catmaid webviewer, I can see that annotation exists. The same 500 server error occurs for when I try to run pymaid.get_skids_by_name("PRCal1") and pymaid.find_neurons(annotations="Bezares-Calderon_et_al_2018"). Any suggestions for how to fix this? I believe something is happening with the server url but not sure if I am just using pymaid wrong.

Thanks!

[schlegelp]

Hi! Unless otherwise specified, CatmaidInstance connects to the project with id 1 which doesn't seem to exist on this server.

I should probably make this easier in pymaid but at this point, you have to figure out the project ID yourself. For example by going into CATMAID and clicking on "URL to this view" which will generate a URL that looks like this: https://catmaid.jekelylab.ex.ac.uk/?pid=11&zp=0&yp=82074.3&xp=73501.5&tool=navigator&sid0=7&s0=5. In this URL, the project ID is encoded as pid=11.

So in order to connect to your full body Platynereis Dumerili, you would need to connect like this:

import pymaid
rm = pymaid.CatmaidInstance(
    server="https://catmaid.jekelylab.ex.ac.uk/#",
    api_token=None,
    http_user=None,
    http_password=None,
    project_id=11
)

[schlegelp]

Small addendum: you can connect to a server and then figure out the project ID

>>> rm = pymaid.CatmaidInstance(
       server="https://catmaid.jekelylab.ex.ac.uk/#",
       api_token=None,
       http_user=None,
       http_password=None
      )
>>> # Look at available projects
>>> rm.available_projects
  comment  id stackgroups                                             stacks                                    title
0          11          []  [{'comment': '3-day-old larva of Platynereis d...                       HT-4_Naomi_project
5          18          []  [{'comment': '3-day-old larva of Platynereis d...      Plexus_HT-4_Naomi_project__372-4013
1          19          []  [{'comment': '3-day-old larva of Platynereis d...                Immuno_HT-4_Naomi_project
4          22          []  [{'comment': '3-day-old larva of Platynereis d...     Jump_864_HT-4_Naomi_project__853-873
3          23          []  [{'comment': '3-day-old larva of Platynereis d...  Jump_3725_HT-4_Naomi_project__3719-3728
2          24          []  [{'comment': '3-day-old larva of Platynereis d...  Jump_2800_HT-4_Naomi_project__2762-2818
>>> # Set project ID 
>>> rm.project_id = 11

[kareef928]

@schlegelp it works now! Thanks for the help.

Do you think some documentation changes to make finding the project ids easier would be beneficial? I would love to help with this.

[schlegelp]

Sure, PRs are always welcome.

[bdpedigo]

@schlegelp thanks for the info, this was super helpful for me and @kareef928!