[fix](cli): detect SGLANG_DSV4_2604_SUBMODE conflict before launch - #2025
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces validation and diagnostic checks for the SGLANG_DSV4_2604_SUBMODE environment variable to prevent startup crashes when the chosen kt-method is not MXFP4. In doctor.py, a diagnostic check is added to surface this environment variable. In run.py, a fail-fast check is introduced to exit early if the conflicting environment variable is set. Feedback from the review suggests: 1) In doctor.py, only surface the environment variable row if it has a non-empty value (instead of just checking is not None) to avoid noise. 2) In run.py, guard against kt_method being None or empty to prevent an AttributeError when calling .upper().
| dsv4_submode = os.environ.get("SGLANG_DSV4_2604_SUBMODE") | ||
| if dsv4_submode is not None: |
There was a problem hiding this comment.
If SGLANG_DSV4_2604_SUBMODE is set to an empty string in the environment, checking dsv4_submode is not None will still add a row to the doctor diagnostics table with an empty value. To avoid unnecessary noise in the diagnostics output, it is better to only surface the row if the variable has a non-empty value.
| dsv4_submode = os.environ.get("SGLANG_DSV4_2604_SUBMODE") | |
| if dsv4_submode is not None: | |
| dsv4_submode = os.environ.get("SGLANG_DSV4_2604_SUBMODE") | |
| if dsv4_submode: |
| dsv4_submode = env.get("SGLANG_DSV4_2604_SUBMODE", "") | ||
| if dsv4_submode == "2604B" and kt_method.upper() != "MXFP4": |
There was a problem hiding this comment.
To ensure robust defensive programming, we should guard against the possibility of kt_method being None or empty. If kt_method is None, calling kt_method.upper() will raise an AttributeError. Adding a check for not kt_method prevents this crash and correctly identifies the conflict since a missing method is not MXFP4.
| dsv4_submode = env.get("SGLANG_DSV4_2604_SUBMODE", "") | |
| if dsv4_submode == "2604B" and kt_method.upper() != "MXFP4": | |
| dsv4_submode = env.get("SGLANG_DSV4_2604_SUBMODE", "") | |
| if dsv4_submode == "2604B" and (not kt_method or kt_method.upper() != "MXFP4"): |
doctor.py: skip SGLANG_DSV4_2604_SUBMODE row when value is empty string, not just None, to avoid spurious noise in kt doctor output. run.py: guard kt_method against None/empty before calling .upper() in _check_conflicting_env_vars to prevent AttributeError.
|
Applied both suggestions — empty-string guard in doctor.py and None-check before |
…vcache-ai#2025) * [fix](cli): detect SGLANG_DSV4_2604_SUBMODE conflict before launch * [fix](cli): tighten env-var validation per review feedback doctor.py: skip SGLANG_DSV4_2604_SUBMODE row when value is empty string, not just None, to avoid spurious noise in kt doctor output. run.py: guard kt_method against None/empty before calling .upper() in _check_conflicting_env_vars to prevent AttributeError.
…vcache-ai#2025) * [fix](cli): detect SGLANG_DSV4_2604_SUBMODE conflict before launch * [fix](cli): tighten env-var validation per review feedback doctor.py: skip SGLANG_DSV4_2604_SUBMODE row when value is empty string, not just None, to avoid spurious noise in kt doctor output. run.py: guard kt_method against None/empty before calling .upper() in _check_conflicting_env_vars to prevent AttributeError.
…vcache-ai#2025) * [fix](cli): detect SGLANG_DSV4_2604_SUBMODE conflict before launch * [fix](cli): tighten env-var validation per review feedback doctor.py: skip SGLANG_DSV4_2604_SUBMODE row when value is empty string, not just None, to avoid spurious noise in kt doctor output. run.py: guard kt_method against None/empty before calling .upper() in _check_conflicting_env_vars to prevent AttributeError.
Fixes #2013
What does this PR do?
SGLANG_DSV4_2604_SUBMODE=2604Bis a DeepSeek-V4-Flash submode flag thatinjects
swiglu_limit=10.0into MoE layers. kt-kernel only supports this onthe MXFP4 backend. If it is left in the environment from a previous MXFP4
launch and the user switches to another method (e.g. AMXINT4), startup crashes
with a cryptic ValueError deep in model loading -- after 150+ seconds of weight
loading has already completed.
This PR adds two layers of detection inside the CLI:
kt run: after the subprocess env dict is fully assembled (shell +advanced.env+inference.envfrom kt config), check for the conflict andexit immediately with a clear error message before sglang is ever launched.
kt doctor: surfaceSGLANG_DSV4_2604_SUBMODEas a WARNING row in thediagnostics table whenever it is set to
2604B, so users can catch theconflict proactively.
Note: users who invoke
python -m sglang.launch_serverdirectly are notaffected by the
kt runguard, but the existing error message inexperts.pyalready handles that path clearly.Before submitting