Expose embed_frames on VideoDataset ingest APIs. by horatiualmasan · Pull Request #1576 · lightly-ai/lightly-studio

horatiualmasan · 2026-07-06T09:33:15Z

What has changed and why?

Add embed_frames to add_videos_from_path and add_videos_from_youtube_vis to generate frame level embeddings, independent of video-level embed.

How has it been tested?

New tests

Did you update CHANGELOG.md?

Yes
Not needed (internal change)

Summary by CodeRabbit

Summary

New Features
- Added embed_frames support to add_videos_from_path and add_videos_from_youtube_vis, enabling image embeddings for extracted frames during ingest (default: enabled).
- Frame embeddings are controlled independently from existing video-level embeddings (embed).
Documentation
- Updated the Python SDK changelog and method documentation to describe embed_frames behavior.
Tests
- Expanded coverage for combinations of embed and embed_frames, including frame-only, video-only, and “no embeddings” scenarios.
- Updated end-to-end and subsampling tests to include embed_frames where applicable.

coderabbitai · 2026-07-06T09:33:23Z

📝 Walkthrough

Walkthrough

Video dataset ingestion adds an embed_frames option to path- and YouTube-based imports, forwards it to decoding pipelines, and tests independent video-level and frame-level embedding behavior.

Changes

Video embedding controls

Layer / File(s)	Summary
Ingest API parameter forwarding `lightly_studio/src/lightly_studio/core/video/video_dataset.py`, `CHANGELOG.md`	Both video ingestion methods expose and document `embed_frames`, forward it to their loading pipelines, and retain independent video embedding behavior.
Embedding behavior validation `lightly_studio/tests/core/video/test_video_dataset.py`, `lightly_studio/tests/core/video/test_video_frame_dataset.py`	Tests cover disabled frame embeddings, combined video and frame embeddings, and ingestion scenarios with frame embedding disabled.

Estimated code review effort: 2 (Simple) | ~10 minutes

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title is concise and accurately summarizes the main change: exposing embed_frames on VideoDataset ingest APIs.
Description check	✅ Passed	The description matches the required template and includes the change summary, testing note, and changelog confirmation.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch horatiu-lig-9970-support-calculating-embeddings-for-video-frames-python-api-3

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

Add embed_frames parameter to VideoDataset.add_videos_from_path and add_videos_from_youtube_vis to generate image embeddings for extracted video frames during ingest, independent of video-level embed.

…ngs-for-video-frames-python-api-3

horatiualmasan · 2026-07-13T09:23:43Z

/review

MalteEbner · 2026-07-13T10:54:41Z

@@ -114,6 +114,110 @@ def test_dataset_add_videos_from_path__dont_embed(
        )
        assert len(embeddings) == 0


This is not touched by you, however still wondering how it fits together

# Check that embeddings were created [...] assert len(embeddings) == 0

MalteEbner · 2026-07-13T10:55:45Z

        )
        assert len(embeddings) == 0

+    def test_dataset_add_videos_from_path__embed_frames(


Suggested change

def test_dataset_add_videos_from_path__embed_frames(

def test_dataset_add_videos_from_path__embed_only_frames(

MalteEbner

Mostly LGTM

Optional: I am wondering about the 4 tests for the cases

neither embed frames nor videos
embed only frames
embed only videos
embed frames and videos
I am fine with the current setup, however it seems quite verbose to me. Would be open to refactoring it such that there are less LOC. It would improve readability in my eyes. Could be by parametrisation and/or by putting common code into helper functions.

MalteEbner · 2026-07-13T10:56:03Z

+        )
+        assert len(video_embeddings) == 0
+
+    def test_dataset_add_videos_from_path__embed_frames_and_video(


Suggested change

def test_dataset_add_videos_from_path__embed_frames_and_video(

def test_dataset_add_videos_from_path__embed_frames_and_videos(

MalteEbner · 2026-07-13T10:57:41Z


 ### Added

+- Python SDK: `embed_frames` parameter on `VideoDataset.add_videos_from_path` and `VideoDataset.add_videos_from_youtube_vis`


Suggested change

- Python SDK: `embed_frames` parameter on `VideoDataset.add_videos_from_path` and `VideoDataset.add_videos_from_youtube_vis`

- Python SDK: Allow embedding video frames by adding the `embed_frames` parameter to `VideoDataset.add_videos_from_path` and `VideoDataset.add_videos_from_youtube_vis`

coderabbitai

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@CHANGELOG.md`:
- Line 12: Update the changelog prose around the VideoDataset entries to
capitalize the platform name as “YouTube,” while preserving the API identifier
`add_videos_from_youtube_vis` unchanged.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: e60cfc32-208c-443d-8b34-ec62950ce64f

📥 Commits

Reviewing files that changed from the base of the PR and between 966e845 and 8ad57b2.

📒 Files selected for processing (2)

CHANGELOG.md
lightly_studio/tests/core/video/test_video_dataset.py

🚧 Files skipped from review as they are similar to previous changes (1)

lightly_studio/tests/core/video/test_video_dataset.py

horatiualmasan requested a review from a team as a code owner July 6, 2026 09:33

horatiualmasan force-pushed the horatiu-lig-9970-support-calculating-embeddings-for-video-frames-python-api-2 branch from b8b01fa to 6852dff Compare July 9, 2026 09:07

Base automatically changed from horatiu-lig-9970-support-calculating-embeddings-for-video-frames-python-api-2 to main July 13, 2026 07:39

Expose embed_frames on VideoDataset ingest APIs

a2f0190

Add embed_frames parameter to VideoDataset.add_videos_from_path and add_videos_from_youtube_vis to generate image embeddings for extracted video frames during ingest, independent of video-level embed.

horatiualmasan force-pushed the horatiu-lig-9970-support-calculating-embeddings-for-video-frames-python-api-3 branch from dc92fed to a2f0190 Compare July 13, 2026 08:55

horatiualmasan and others added 2 commits July 13, 2026 12:05

format

92dcbb7

Merge branch 'main' into horatiu-lig-9970-support-calculating-embeddi…

966e845

…ngs-for-video-frames-python-api-3

MalteEbner reviewed Jul 13, 2026

View reviewed changes

MalteEbner approved these changes Jul 13, 2026

View reviewed changes

simplify

8ad57b2

coderabbitai Bot reviewed Jul 13, 2026

View reviewed changes

Comment thread CHANGELOG.md

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Expose embed_frames on VideoDataset ingest APIs.#1576

Expose embed_frames on VideoDataset ingest APIs.#1576
horatiualmasan wants to merge 4 commits into
mainfrom
horatiu-lig-9970-support-calculating-embeddings-for-video-frames-python-api-3

horatiualmasan commented Jul 6, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented Jul 6, 2026 •

edited

Loading

Walkthrough

Changes

Uh oh!

horatiualmasan commented Jul 13, 2026

Uh oh!

MalteEbner Jul 13, 2026

Uh oh!

MalteEbner Jul 13, 2026

Uh oh!

MalteEbner left a comment

Uh oh!

MalteEbner Jul 13, 2026

Uh oh!

MalteEbner Jul 13, 2026

Uh oh!

coderabbitai Bot left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		@@ -114,6 +114,110 @@ def test_dataset_add_videos_from_path__dont_embed(
		)
		assert len(embeddings) == 0

	def test_dataset_add_videos_from_path__embed_frames(
	def test_dataset_add_videos_from_path__embed_only_frames(


		### Added

		- Python SDK: `embed_frames` parameter on `VideoDataset.add_videos_from_path` and `VideoDataset.add_videos_from_youtube_vis`

	- Python SDK: `embed_frames` parameter on `VideoDataset.add_videos_from_path` and `VideoDataset.add_videos_from_youtube_vis`
	- Python SDK: Allow embedding video frames by adding the `embed_frames` parameter to `VideoDataset.add_videos_from_path` and `VideoDataset.add_videos_from_youtube_vis`

Uh oh!

Conversation

horatiualmasan commented Jul 6, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What has changed and why?

How has it been tested?

Did you update CHANGELOG.md?

Summary by CodeRabbit

Summary

Uh oh!

coderabbitai Bot commented Jul 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Uh oh!

horatiualmasan commented Jul 13, 2026

Uh oh!

MalteEbner Jul 13, 2026

Choose a reason for hiding this comment

Uh oh!

MalteEbner Jul 13, 2026

Choose a reason for hiding this comment

Uh oh!

MalteEbner left a comment

Choose a reason for hiding this comment

Uh oh!

MalteEbner Jul 13, 2026

Choose a reason for hiding this comment

Uh oh!

MalteEbner Jul 13, 2026

Choose a reason for hiding this comment

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

horatiualmasan commented Jul 6, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Jul 6, 2026 •

edited

Loading