Skip to content

chore: add sandbox memory profiling tools#3249

Open
LittleChenLiya wants to merge 4 commits into
bytedance:mainfrom
LittleChenLiya:fix/issue-3213-sandbox-memory
Open

chore: add sandbox memory profiling tools#3249
LittleChenLiya wants to merge 4 commits into
bytedance:mainfrom
LittleChenLiya:fix/issue-3213-sandbox-memory

Conversation

@LittleChenLiya
Copy link
Copy Markdown
Collaborator

@LittleChenLiya LittleChenLiya commented May 26, 2026

问题原因

AIO sandbox 在空闲时会启动多个常驻服务,Pod memory 可接近 1Gi。仅调整 Kubernetes request/limit 或限制并发不能解释单个 sandbox 的基础内存占用,需要先提供可重复的画像数据。

修改内容

  • 新增 sandbox memory profiling 脚本,用于采集 Pod memory、资源配置和进程 RSS。
  • 新增 profiling 文档,说明采样阶段、输出格式、候选 runtime 对比矩阵和 PR 取证要求。
  • 增加测试覆盖 Kubernetes memory/CPU 单位解析、进程 RSS 解析、错误报告和 Markdown 输出。

关联 issue

Related to #3213


Problem Cause

The AIO sandbox starts several long-running services while idle, and Pod memory can approach 1 GiB. Changing Kubernetes requests/limits or reducing concurrency alone cannot explain each sandbox's baseline memory, so reproducible profiling data is needed first.

Changes

  • Added a sandbox memory profiling script for collecting Pod memory, resource configuration, and process RSS.
  • Added profiling documentation covering sampling phases, output formats, candidate runtime comparison, and PR evidence expectations.
  • Added tests for Kubernetes memory/CPU unit parsing, process RSS parsing, error reporting, and Markdown output.

Related Issue

Related to #3213

@LittleChenLiya LittleChenLiya force-pushed the fix/issue-3213-sandbox-memory branch from a3bc8da to 42d43df Compare May 26, 2026 11:13
@LittleChenLiya
Copy link
Copy Markdown
Collaborator Author

@copilot review

@LittleChenLiya LittleChenLiya requested a review from Copilot May 26, 2026 12:17
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds tooling and documentation to baseline and compare Kubernetes sandbox memory usage, plus provisioner toggles to disable in-sandbox services to reduce idle memory.

Changes:

  • Introduces a kubectl-based memory profiling script that outputs JSON/Markdown and can optionally sample top RSS processes.
  • Adds provisioner environment flags to pass DISABLE_JUPYTER / DISABLE_CODE_SERVER into sandbox pods.
  • Adds docs and tests covering the new profiling workflow and provisioner behavior.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
scripts/sandbox_memory_profile.py New CLI script to snapshot pod memory/CPU and (optionally) per-process RSS into JSON/Markdown reports.
docker/provisioner/app.py Adds env-driven toggles to inject DISABLE_JUPYTER / DISABLE_CODE_SERVER into sandbox container spec.
docker/provisioner/README.md Documents new provisioner env vars for disabling Jupyter and code-server.
docker/docker-compose.yaml Wires new env vars through docker-compose deployment.
docker/docker-compose-dev.yaml Wires new env vars through dev docker-compose deployment.
backend/tests/test_sandbox_memory_profile_script.py Adds unit tests for parsing/merging/reporting logic in the new script.
backend/tests/test_provisioner_pvc_volumes.py Adds tests asserting sandbox env toggles are included/omitted as configured.
backend/docs/SANDBOX_MEMORY_PROFILING.md New guide describing what to measure and how to capture/compare memory snapshots.
backend/docs/README.md Links the new sandbox memory profiling guide from docs index.

Comment thread scripts/sandbox_memory_profile.py Outdated
Comment thread scripts/sandbox_memory_profile.py Outdated
Comment thread scripts/sandbox_memory_profile.py
Comment thread scripts/sandbox_memory_profile.py
Comment thread docker/provisioner/app.py Outdated
Comment thread docker/provisioner/app.py Outdated
@LittleChenLiya LittleChenLiya force-pushed the fix/issue-3213-sandbox-memory branch from 42d43df to a94d3b5 Compare May 26, 2026 12:29
@LittleChenLiya LittleChenLiya requested a review from Copilot May 26, 2026 13:22
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 6 comments.

Comment thread scripts/sandbox_memory_profile.py
Comment thread scripts/sandbox_memory_profile.py
Comment thread scripts/sandbox_memory_profile.py Outdated
Comment thread scripts/sandbox_memory_profile.py Outdated
Comment thread docker/provisioner/app.py Outdated
Comment thread docker/provisioner/app.py Outdated
@LittleChenLiya LittleChenLiya force-pushed the fix/issue-3213-sandbox-memory branch from fb218bd to 58a0039 Compare May 26, 2026 13:31
@LittleChenLiya LittleChenLiya requested a review from Copilot May 26, 2026 13:33
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated no new comments.

@WillemJiang WillemJiang requested a review from foreleven May 27, 2026 00:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants