Skip to content

[fix](cli): detect SGLANG_DSV4_2604_SUBMODE conflict before launch - #2025

Merged
yyj6666667 merged 2 commits into
kvcache-ai:mainfrom
devangpratap:fix/run-env-var-validation
May 30, 2026
Merged

[fix](cli): detect SGLANG_DSV4_2604_SUBMODE conflict before launch#2025
yyj6666667 merged 2 commits into
kvcache-ai:mainfrom
devangpratap:fix/run-env-var-validation

Conversation

@devangpratap

Copy link
Copy Markdown
Contributor

Fixes #2013

What does this PR do?

SGLANG_DSV4_2604_SUBMODE=2604B is a DeepSeek-V4-Flash submode flag that
injects swiglu_limit=10.0 into MoE layers. kt-kernel only supports this on
the MXFP4 backend. If it is left in the environment from a previous MXFP4
launch and the user switches to another method (e.g. AMXINT4), startup crashes
with a cryptic ValueError deep in model loading -- after 150+ seconds of weight
loading has already completed.

This PR adds two layers of detection inside the CLI:

  • kt run: after the subprocess env dict is fully assembled (shell +
    advanced.env + inference.env from kt config), check for the conflict and
    exit immediately with a clear error message before sglang is ever launched.
  • kt doctor: surface SGLANG_DSV4_2604_SUBMODE as a WARNING row in the
    diagnostics table whenever it is set to 2604B, so users can catch the
    conflict proactively.

Note: users who invoke python -m sglang.launch_server directly are not
affected by the kt run guard, but the existing error message in
experts.py already handles that path clearly.

Before submitting

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces validation and diagnostic checks for the SGLANG_DSV4_2604_SUBMODE environment variable to prevent startup crashes when the chosen kt-method is not MXFP4. In doctor.py, a diagnostic check is added to surface this environment variable. In run.py, a fail-fast check is introduced to exit early if the conflicting environment variable is set. Feedback from the review suggests: 1) In doctor.py, only surface the environment variable row if it has a non-empty value (instead of just checking is not None) to avoid noise. 2) In run.py, guard against kt_method being None or empty to prevent an AttributeError when calling .upper().

Comment thread kt-kernel/python/cli/commands/doctor.py Outdated
Comment on lines +440 to +441
dsv4_submode = os.environ.get("SGLANG_DSV4_2604_SUBMODE")
if dsv4_submode is not None:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

If SGLANG_DSV4_2604_SUBMODE is set to an empty string in the environment, checking dsv4_submode is not None will still add a row to the doctor diagnostics table with an empty value. To avoid unnecessary noise in the diagnostics output, it is better to only surface the row if the variable has a non-empty value.

Suggested change
dsv4_submode = os.environ.get("SGLANG_DSV4_2604_SUBMODE")
if dsv4_submode is not None:
dsv4_submode = os.environ.get("SGLANG_DSV4_2604_SUBMODE")
if dsv4_submode:

Comment thread kt-kernel/python/cli/commands/run.py Outdated
Comment on lines +594 to +595
dsv4_submode = env.get("SGLANG_DSV4_2604_SUBMODE", "")
if dsv4_submode == "2604B" and kt_method.upper() != "MXFP4":

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

To ensure robust defensive programming, we should guard against the possibility of kt_method being None or empty. If kt_method is None, calling kt_method.upper() will raise an AttributeError. Adding a check for not kt_method prevents this crash and correctly identifies the conflict since a missing method is not MXFP4.

Suggested change
dsv4_submode = env.get("SGLANG_DSV4_2604_SUBMODE", "")
if dsv4_submode == "2604B" and kt_method.upper() != "MXFP4":
dsv4_submode = env.get("SGLANG_DSV4_2604_SUBMODE", "")
if dsv4_submode == "2604B" and (not kt_method or kt_method.upper() != "MXFP4"):

doctor.py: skip SGLANG_DSV4_2604_SUBMODE row when value is empty string,
not just None, to avoid spurious noise in kt doctor output.

run.py: guard kt_method against None/empty before calling .upper() in
_check_conflicting_env_vars to prevent AttributeError.
@devangpratap

Copy link
Copy Markdown
Contributor Author

Applied both suggestions — empty-string guard in doctor.py and None-check before .upper() in run.py.

@yyj6666667
yyj6666667 merged commit d41f569 into kvcache-ai:main May 30, 2026
callmegaga pushed a commit to callmegaga/ktransformers that referenced this pull request Jul 10, 2026
…vcache-ai#2025)

* [fix](cli): detect SGLANG_DSV4_2604_SUBMODE conflict before launch

* [fix](cli): tighten env-var validation per review feedback

doctor.py: skip SGLANG_DSV4_2604_SUBMODE row when value is empty string,
not just None, to avoid spurious noise in kt doctor output.

run.py: guard kt_method against None/empty before calling .upper() in
_check_conflicting_env_vars to prevent AttributeError.
callmegaga pushed a commit to callmegaga/ktransformers that referenced this pull request Jul 10, 2026
…vcache-ai#2025)

* [fix](cli): detect SGLANG_DSV4_2604_SUBMODE conflict before launch

* [fix](cli): tighten env-var validation per review feedback

doctor.py: skip SGLANG_DSV4_2604_SUBMODE row when value is empty string,
not just None, to avoid spurious noise in kt doctor output.

run.py: guard kt_method against None/empty before calling .upper() in
_check_conflicting_env_vars to prevent AttributeError.
callmegaga pushed a commit to callmegaga/ktransformers that referenced this pull request Jul 10, 2026
…vcache-ai#2025)

* [fix](cli): detect SGLANG_DSV4_2604_SUBMODE conflict before launch

* [fix](cli): tighten env-var validation per review feedback

doctor.py: skip SGLANG_DSV4_2604_SUBMODE row when value is empty string,
not just None, to avoid spurious noise in kt doctor output.

run.py: guard kt_method against None/empty before calling .upper() in
_check_conflicting_env_vars to prevent AttributeError.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

swiglu_limit=10.0 is only supported on method='MXFP4'

2 participants