[Curiosity] Nanbeige4.1-3B as a potential base for a future acestep-5Hz-lm variant

[FeelTheFonk]

Hi,

First of all, thank you for this Amazing project and your transparency 😎💓

---

The LM lineage table in the README (Qwen3-0.6B → 1.7B → 4B) makes it clear that
base model selection is a deliberate choice, so I wanted to share something I
stumbled upon, purely as a data point, not a request.

Nanbeige4.1-3B (Apache 2.0,
arXiv:2602.13367) is a recent 3B SLM that benchmarks above Qwen3-4B on code,
math, alignment, and tool-use tasks. More interestingly, their base model
apparently outperforms Qwen3-8B-Base when both are SFT'd on identical data,
which suggests a higher post-training ceiling at the same parameter scale. It was
also specifically trained for long-horizon structured output stability, which
feels relevant to what constrained FSQ decoding demands from the LM.

The thought: if that ceiling advantage holds through your GRPO pipeline (with the
FSQ vocabulary extension), a 3B variant trained on this base might match or
exceed the current 4B in planning quality, at lower VRAM cost.

Obviously a direct swap at inference time isn't possible given the codebook
coupling. This would only matter as a training-time consideration.

Is base model selection something you revisit for future LM variants? I'm
curious whether there are hard constraints on your side (tokenizer
compatibility with vLLM, trust_remote_code, or training infra) that rule out
non-Qwen3 bases regardless of benchmark numbers.

Thank you in advance, have a great day 😎

[pratikrath126]

Nanbeige4.1-3B is definitely an interesting candidate for a base model. The 3B parameter range is becoming quite competitive. Have you had a chance to run any initial perplexity tests or benchmarks to see how it compares to the current base? Would be curious to see if it handles the 5Hz-lm constraints better!

[FeelTheFonk]

Glad it resonated :) just manual inferences on my end, nothing rigorous. The output quality felt promising but that's a weak signal without controlled comparison