Video formats on DANDI #2732
Replies: 2 comments 3 replies
-
This link seems broken? Can you please link to the code used to perform this meta-analysis as well as (ideally) stored derived outputs corresponding to your table |
Beta Was this translation helpful? Give feedback.
-
|
@h-mayorquin Or just send me the code and I can setup a study dataset on a CRON |
Beta Was this translation helpful? Give feedback.
-
|
Sorry, yes, I was testing this: https://gist.github.com/h-mayorquin/ec574eec8b16829c12e7ef82e0ad49a9 I will edit the text above. |
Beta Was this translation helpful? Give feedback.
-
|
I will leave some additional thoughts: what is DANDI for? Pure storage? Enabling reproducibility? Being a living part of the data collection and analysis process? The YouTube of Neurophysiology? Of course, we aren't only any one of those but strive to satisfy all at some level. But that is where your first two points come at odds with the last. A scientist may not know or care about browser playability. The main concern is usually just to have a place to store their valuable scientific data as they originally processed them (as per the NIH data sharing policy). How efficient the cloud storage is (compared to their local lab server/drives), they may not care at all, especially because DANDI does not charge by the byte (unlike Zenodo). Long-term accessibility is likely also not at the forefront of most minds, and no one can claim to predict the future. At any rate this is what containers are for, right? What situation would result in an old WMV file being written with a certain version of ffmpeg+encoder library but then unreadable with that same old version (so long as a compatible image built for a typical system). The problem of reproducibility in that situation seems more about recording good provenance. In the end, browser playability and storage efficiency seem to be your biggest points in codec recommendations. But those are recommendations about how to adjust current data acquisition pipelines well before they ever reach the archive, not how to manage files previously posted to the archive. To alleviate your concerns, I would propose a different idea: how much space would it cost on the S3 bucket to have a separate store point for MP4 containers of AV1 encodings of EVERY non-AV1 FILE on the archive (including lossless -> lossy cases)? This idea is similar to chunking parameterization discussion with Oliver and Jeremy going back 3+ years, whereby no single file compression strategy is ideal for every type of streamed visualization. Note that such an idea would/could interact with a couple of design docs I'm drafting on the DANDI side: sunsetting of old unused assets and archive-controlled compression settings for lossless data (HDF5/Zarr, not external videos). The main idea there being the original 'less optimal codec' originally uploaded might fade away to cold storage if rarely used, while the smaller streamable asset stays live. |
Beta Was this translation helpful? Give feedback.
-
|
Yes, @waxlamp was proposing this in the meeting. It generalizes beyond browser friendliness as well. The general principle is to store data as it was generated for provenance (with caveats of course) and then post-upload we build copies that are optimized for specific purposes (e.g. data access, streaming, training of models, etc). |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
This is a starting point for a discussion on best practices to store media formats on Dandi.
For NWB Dandi currently accepts arbitrary video codecs and containers for
ImageSeriesexternal files. Some labs make deliberate codec choices (e.g., FFV1 for lossless preservation of high-bit-depth data), but in many cases the codec is inherited from camera SDK defaults (MJPEG from FLIR/Basler), recording software (MPEG-4 Part 2 from OpenCV or EthoVision), or older FFmpeg versions, without the lab making a conscious decision.This matters for three reasons:
I ran a script to check the current state of video on DANDI. Here are some summaries for the discussion. The script checked all 6,990 video files across 28 dandisets (March 2026):
Three dandisets (000559, 001425, 000409) account for 71% of all video files and 89% of all video storage. The full breakdown by dandiset:
Dandisets with less than 1 GiB of video omitted (5 dandisets, 28 files total). The script that was used to generate the full data is here:
https://gist.github.com/h-mayorquin/ec574eec8b16829c12e7ef82e0ad49a9
To illustrate this with more detail, I profiled 11 dandisets to understand why each codec was chosen and whether conversion is appropriate:
gray16lepixel values for MoSeq analysis. H.264 would truncate to 8/10-bit.Lavf56.1.0) in recording pipeline defaulted to MPEG-4 Part 2.pal8(8-bit grayscale) at 20 fps for whisker tracking. Deliberate choice to avoid compression artifacts.bgra(32-bit with alpha) from camera SDK/frame grabber. 10 fps defensive facial reactions.The pattern: some labs make deliberate codec choices to preserve data fidelity (FFV1 for 16-bit depth data in 000559, H.264 4:4:4 for full chroma in 000231, raw for exact pixel values in 000540), but the majority inherit their codec from camera SDKs (FLIR, Basler, Kinect), recording software (EthoVision, AnyMaze), or older FFmpeg defaults without making a conscious decision.
A proposal for using the NWB Inspector to guide codec choices going forward is at nwbinspector#669. The inspector already checks that
ImageSeriesexternal files exist and use relative paths. Adding a best practice check for codec and container would be a natural extension. The checks should start as recommendations (BEST_PRACTICE_SUGGESTION level), not blockers, since making them critical would invalidate existing dandisets.Beta Was this translation helpful? Give feedback.
All reactions