Disable external services for embargoed data/dandisets

[yarikoptic]

AFAIK ATM all external services make sense only for public data only since we are not directing to minted URL for asset blobs, nor @magland has access to provide neurosift access to them at the level of dandisets. But ATM we keep all the "Open With" buttons enabled even though following them would just result in various errors: e.g. on https://dandiarchive.org/dandiset/001169/draft

Ideally UI should be adjusted so that if dandiset embargoed, it is greyed out (ideally with a hint that it is since it is embargoed).

• edit: inspired by Include support for neuroglancer url for umembargoed zarr and nifti #2063

[magland]

@yarikoptic ... in the case of opening an NWB file, Neurosift does support viewing embargoed files... and that's actually a valuable feature. You need to input your dandi API key into Neurosift. I need to add a feature where it tells the user what to do in that case.

Should also work for viewing dandisets... but I haven't tested that specifically.

[aaronkanzer]

@yarikoptic ... in the case of opening an NWB file, Neurosift does support viewing embargoed files... and that's actually a valuable feature. You need to input your dandi API key into Neurosift. I need to add a feature where it tells the user what to do in that case.

Should also work for viewing dandisets... but I haven't tested that specifically.

@magland -- just poking at neurosift right now as well -- somewhat related here, but also nwb.zarr presents a similar error statement: -- sample public nwb zarr -- I assume this is driven by the handful of 400 HTTP responses via calling /download endpoint in DANDI for a given zarr

[aaronkanzer]

@yarikoptic ... in the case of opening an NWB file, Neurosift does support viewing embargoed files... and that's actually a valuable feature. You need to input your dandi API key into Neurosift. I need to add a feature where it tells the user what to do in that case.

Should also work for viewing dandisets... but I haven't tested that specifically.

@magland -- included flatironinstitute/neurosift#210 here -- still tinkering around with your dev environment to replicate, but I assume from the 403's on embargoed dandisets in neurosift, this should help

[yarikoptic]

@yarikoptic ... in the case of opening an NWB file, Neurosift does support viewing embargoed files... and that's actually a valuable feature. You need to input your dandi API key into Neurosift. I need to add a feature where it tells the user what to do in that case.

Good to know, but/and neurosift specific. Indeed - should announce to the user that API key needs to be provided!

I am not that verse in web tech -- isn't there some way to provide some kind of "session continuity" so client while working on neurosift could still retain access to DANDI's session and thus /download/ to still be able to mint URL?

Overall then, while disabling we need to add annotation to neurosift that it can handle embargoed dandisets, and thus only disable services which do not announce for that. @aaronkanzer -- do you think you could draft such a generic PR for the services based on your work in the

• Include support for neuroglancer url for umembargoed zarr and nifti #2063

?

[magland]

A couple relevant updates for the specific case of neurosift

• A helpful message now appears when NWB file doesn't load properly instructing user to set the DANDI API key.
• I confirmed that Neurosift is in fact able to load the Dandiset page for embargoed dandisets.
• A helpful message now also appears for the dandiset page when unable to load.

I am not that verse in web tech -- isn't there some way to provide some kind of "session continuity" so client while working on neurosift could still retain access to DANDI's session and thus /download/ to still be able to mint URL?

It may not be the wisest thing to do, but we could have dandi pass the api key as query parameter to Neurosift. Neurosift would then accept it and then redirect to the same url without the api key. But I think it's safer and better to require that the user copies and pastes it, so it is an intentional operation.

[aaronkanzer]

A couple relevant updates for the specific case of neurosift

• A helpful message now appears when NWB file doesn't load properly instructing user to set the DANDI API key.
• I confirmed that Neurosift is in fact able to load the Dandiset page for embargoed dandisets.
• A helpful message now also appears for the dandiset page when unable to load.

I am not that verse in web tech -- isn't there some way to provide some kind of "session continuity" so client while working on neurosift could still retain access to DANDI's session and thus /download/ to still be able to mint URL?

It may not be the wisest thing to do, but we could have dandi pass the api key as query parameter to Neurosift. Neurosift would then accept it and then redirect to the same url without the api key. But I think it's safer and better to require that the user copies and pastes it, so it is an intentional operation.

@magland if you wanted to be a bit more secure here -- you could encrypt the DANDI_API_KEY in the payload, and then have logic in Neurosift that decrypts; however, in the vein of the "keep it simple" mantra, I think what you have works fine, unless users start to complain

@yarikoptic I'm not sure I fully follow what criteria would be included to invoke a presigned url minted and provided here -- might be my lack of NWB knowledge 😉