How do I increase performance for Step_Audio_EditX_Inline_Tags?

[avogatro]

First thing good work on the functionality of all of this.
I really like the idea of changing emotions or speed with tags.

The problem is I have only 24GB GPU memory, and this inline tag use all of it and I often got CUDA error for out of memory.
with 0.2 it/s I need 81920 seconds for the job, that is 22.7 hours for a 30 seconds audio clip.

I see no option to change those default values:
max 8192 tokens, precision=auto, device=auto,

also I have no clue if the warning "no CUDAExecutionProvider for onnxruntime" is problematic or not.

I run on python 3.13, nightly build.

(py313) PS E:\ComfiUI_conda> .\run_nvidia_gpu.bat

E:\ComfiUI_conda>python -s .\main.py --windows-standalone-build --use-sage-attention --fast
Adding extra search path checkpoints C:\Users\jiezh\Downloads\ComfyUI\ComfyUI_windows_portable\ComfyUI\models\checkpoints
Adding extra search path text_encoders C:\Users\jiezh\Downloads\ComfyUI\ComfyUI_windows_portable\ComfyUI\models\text_encoders
Adding extra search path text_encoders C:\Users\jiezh\Downloads\ComfyUI\ComfyUI_windows_portable\ComfyUI\models\clip\ # legacy location still supported
Adding extra search path clip_vision C:\Users\jiezh\Downloads\ComfyUI\ComfyUI_windows_portable\ComfyUI\models\clip_vision
Adding extra search path configs C:\Users\jiezh\Downloads\ComfyUI\ComfyUI_windows_portable\ComfyUI\models\configs
Adding extra search path controlnet C:\Users\jiezh\Downloads\ComfyUI\ComfyUI_windows_portable\ComfyUI\models\controlnet
Adding extra search path diffusion_models C:\Users\jiezh\Downloads\ComfyUI\ComfyUI_windows_portable\ComfyUI\models\diffusion_models
Adding extra search path diffusion_models C:\Users\jiezh\Downloads\ComfyUI\ComfyUI_windows_portable\ComfyUI\models\unet
Adding extra search path embeddings C:\Users\jiezh\Downloads\ComfyUI\ComfyUI_windows_portable\ComfyUI\models\embeddings
Adding extra search path loras C:\Users\jiezh\Downloads\ComfyUI\ComfyUI_windows_portable\ComfyUI\models\loras
Adding extra search path upscale_models C:\Users\jiezh\Downloads\ComfyUI\ComfyUI_windows_portable\ComfyUI\models\upscale_models
Adding extra search path vae C:\Users\jiezh\Downloads\ComfyUI\ComfyUI_windows_portable\ComfyUI\models\vae
Adding extra search path audio_encoders C:\Users\jiezh\Downloads\ComfyUI\ComfyUI_windows_portable\ComfyUI\models\audio_encoders
Adding extra search path model_patches C:\Users\jiezh\Downloads\ComfyUI\ComfyUI_windows_portable\ComfyUI\models\model_patches
[START] Security scan
[DONE] Security scan

ComfyUI-Manager: installing dependencies done.

** ComfyUI startup time: 2025-12-29 16:00:52.660
** Platform: Windows
** Python version: 3.13.11 | packaged by conda-forge | (main, Dec 6 2025, 11:10:00) [MSC v.1944 64 bit (AMD64)]
** Python executable: D:\anaconda\envs\py313\python.exe
** ComfyUI Path: E:\ComfiUI_conda
** ComfyUI Base Folder Path: E:\ComfiUI_conda
** User directory: E:\ComfiUI_conda\user
** ComfyUI-Manager config path: E:\ComfiUI_conda\user__manager\config.ini
** Log path: E:\ComfiUI_conda\user\comfyui.log

Prestartup times for custom nodes:
0.0 seconds: E:\ComfiUI_conda\custom_nodes\rgthree-comfy
5.6 seconds: E:\ComfiUI_conda\custom_nodes\comfyui-manager

Checkpoint files will always be loaded safely.
Total VRAM 24576 MB, total RAM 32669 MB
pytorch version: 2.9.1+cu128
Enabled fp16 accumulation.
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce RTX 3090 : native
Using async weight offloading with 2 streams
Enabled pinned memory 14700.0
working around nvidia conv3d memory bug.
Using sage attention
Python version: 3.13.11 | packaged by conda-forge | (main, Dec 6 2025, 11:10:00) [MSC v.1944 64 bit (AMD64)]
ComfyUI version: 0.6.0
ComfyUI frontend version: 1.34.9
[Prompt Server] web root: D:\anaconda\envs\py313\Lib\site-packages\comfyui_frontend_package\static
Total VRAM 24576 MB, total RAM 32669 MB
pytorch version: 2.9.1+cu128
Enabled fp16 accumulation.
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce RTX 3090 : native
Using async weight offloading with 2 streams
Enabled pinned memory 14700.0
Web extensions folder found at E:\ComfiUI_conda\web\extensions\ComfyLiterals
ComfyUI-GGUF: Allowing full torch compile

Loading: ComfyUI-Impact-Pack (V8.28)

[Impact Pack] Wildcard total size (0.00 MB) is within cache limit (50.00 MB). Using full cache mode.
[Impact Pack] Wildcards loading done.

Loading: ComfyUI-Impact-Subpack (V1.3.5)

[Impact Pack/Subpack] Using folder_paths to determine whitelist path: E:\ComfiUI_conda\user\default\ComfyUI-Impact-Subpack\model-whitelist.txt
[Impact Pack/Subpack] Ensured whitelist directory exists: E:\ComfiUI_conda\user\default\ComfyUI-Impact-Subpack
[Impact Pack/Subpack] Loaded 0 model(s) from whitelist: E:\ComfiUI_conda\user\default\ComfyUI-Impact-Subpack\model-whitelist.txt
[Impact Subpack] ultralytics_bbox: E:\ComfiUI_conda\models\ultralytics\bbox
[Impact Subpack] ultralytics_segm: E:\ComfiUI_conda\models\ultralytics\segm

Loading: ComfyUI-Manager (V3.39)

[ComfyUI-Manager] network_mode: public
[ComfyUI-Manager] ComfyUI per-queue preview override detected (PR #11261). Manager's preview method feature is disabled. Use ComfyUI's --preview-method CLI option or 'Settings > Execution > Live preview method'.

ComfyUI Revision: 4431 [e4c61d75] *DETACHED | Released on '2025-12-23'

[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/alter-list.json
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/model-list.json
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/github-stats.json
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/extension-node-map.json
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/custom-node-list.json
FETCH ComfyRegistry Data: 5/116
new registry table has been added: preprocessor_classes
FETCH ComfyRegistry Data: 10/116

[StepAudio] Loaded 2 nodes:
[StepAudio] • StepAudioEditX - Clone 🎤 - Native voice cloning node
[StepAudio] • StepAudioEditX - Edit ✏️ - Native audio editing node
[StepAudio] Features: Voice Cloning | Audio Editing
[StepAudio] Powered by: Step Audio EditX (3B) + CosyVoice

FETCH ComfyRegistry Data: 15/116

Comfyroll Studio v1.76 : 175 Nodes Loaded

** For changes, please see patch notes at https://github.com/Suzie1/ComfyUI_Comfyroll_CustomNodes/blob/main/Patch_Notes.md
** For help, please see the wiki at https://github.com/Suzie1/ComfyUI_Comfyroll_CustomNodes/wiki

FETCH ComfyRegistry Data: 20/116
LOADED 6 FONTS
No OpenGL_accelerate module loaded: No module named 'OpenGL_accelerate'

███╗ ███╗ █████╗ ██████╗██╗ ██╗██╗███╗ ██╗███████╗
████╗ ████║██╔══██╗██╔════╝██║ ██║██║████╗ ██║██╔════╝
██╔████╔██║███████║██║ ███████║██║██╔██╗ ██║█████╗
██║╚██╔╝██║██╔══██║██║ ██╔══██║██║██║╚██╗██║██╔══╝
██║ ╚═╝ ██║██║ ██║╚██████╗██║ ██║██║██║ ╚████║███████╗
╚═╝ ╚═╝╚═╝ ╚═╝ ╚═════╝╚═╝ ╚═╝╚═╝╚═╝ ╚═══╝╚══════╝

██████╗ ███████╗██╗ ██╗ ██╗███████╗██╗ ██████╗ ███╗ ██╗███████╗
██╔══██╗██╔════╝██║ ██║ ██║██╔════╝██║██╔═══██╗████╗ ██║██╔════╝
██║ ██║█████╗ ██║ ██║ ██║███████╗██║██║ ██║██╔██╗ ██║███████╗
██║ ██║██╔══╝ ██║ ██║ ██║╚════██║██║██║ ██║██║╚██╗██║╚════██║
██████╔╝███████╗███████╗╚██████╔╝███████║██║╚██████╔╝██║ ╚████║███████║
╚═════╝ ╚══════╝╚══════╝ ╚═════╝ ╚══════╝╚═╝ ╚═════╝ ╚═╝ ╚═══╝╚══════╝

[rgthree-comfy] Loaded 48 extraordinary nodes. 🎉

[rgthree-comfy] ComfyUI's new Node 2.0 rendering may be incompatible with some rgthree-comfy nodes and features, breaking some rendering as well as losing the ability to access a node's properties (a vital part of many nodes). It also appears to run MUCH more slowly spiking CPU usage and causing jankiness and unresponsiveness, especially with large workflows. Personally I am not planning to use the new Nodes 2.0 and, unfortunately, am not able to invest the time to investigate and overhaul rgthree-comfy where needed. If you have issues when Nodes 2.0 is enabled, I'd urge you to switch it off as well and join me in hoping ComfyUI is not planning to deprecate the existing, stable canvas rendering all together.

FETCH ComfyRegistry Data: 25/116
🔧 torchaudio.save/load globally patched (scipy for WAV files, no TorchCodec)
🔇 Suppressed torchaudio 2.9 migration warnings
APEX FusedRMSNorm not available, using native implementation
ℹ️ Critical package versions: NumPy 2.2.6, Librosa 0.11.0, Numba 0.63.1, PyTorch 2.9.1+cu128, TorchAudio 2.9.1+cu128, Transformers 4.51.3, Accelerate 1.6.0, SoundFile 0.13.1
FETCH ComfyRegistry Data: 30/116

🚀 TTS Audio Suite v4.15.17
Universal multi-engine TTS extension for ComfyUI
🐍 Python 3.13.11 detected
✅ TTS Audio Suite v4.15.17 loaded with 29 nodes:
• ⚙️ ChatterBox Official 23-Lang Engine
• ⚙️ ChatterBox TTS Engine
• ⚙️ F5 TTS Engine
• ⚙️ Higgs Audio 2 Engine
• ⚙️ IndexTTS-2 Engine
• ⚙️ RVC Engine
• ⚙️ Step Audio EditX Engine
• ⚙️ VibeVoice Engine
• 🌈 IndexTTS-2 Emotion Vectors
• 🌈 IndexTTS-2 Text Emotion
• 🌊 Audio Wave Analyzer
• 🎙️ Voice Capture
• 🎤 TTS Text
• 🎨 Step Audio EditX - Audio Editor
• 🎭 Character Voices
• 🎭 Load RVC Character Model
• 🏷️ Multiline TTS Tag Editor
• 👄 F5-TTS Speech Editor
• 📝 Phoneme Text Normalizer
• 📺 TTS SRT
• 🔄 Voice Changer
• 🔧 Audio Analyzer Options
• 🔧 F5-TTS Edit Options
• 🔧 RVC Pitch Extraction Options
• 🔧 Viseme Mouth Shape Options
• 🗣️ Silent Speech Analyzer
• 🤐 Noise or Vocal Removal
• 🤐 Voice Fixer
• 🥪 Merge Audio

Import times for custom nodes:
0.0 seconds: E:\ComfiUI_conda\custom_nodes\websocket_image_save.py
0.0 seconds: E:\ComfiUI_conda\custom_nodes\ComfyLiterals
0.0 seconds: E:\ComfiUI_conda\custom_nodes\comfyui_ultimatesdupscale
0.0 seconds: E:\ComfiUI_conda\custom_nodes\comfyui-custom-scripts
0.0 seconds: E:\ComfiUI_conda\custom_nodes\ComfyUI-segment-anything-2
0.0 seconds: E:\ComfiUI_conda\custom_nodes\ComfyUI-GGUF
0.0 seconds: E:\ComfiUI_conda\custom_nodes\comfyui-kjnodes
0.0 seconds: E:\ComfiUI_conda\custom_nodes\rgthree-comfy
0.0 seconds: E:\ComfiUI_conda\custom_nodes\ComfyUI_Comfyroll_CustomNodes
0.1 seconds: E:\ComfiUI_conda\custom_nodes\comfyui-impact-subpack
0.1 seconds: E:\ComfiUI_conda\custom_nodes\comfyui-videohelpersuite
0.3 seconds: E:\ComfiUI_conda\custom_nodes\comfyui-tensorops
0.3 seconds: E:\ComfiUI_conda\custom_nodes\comfyui-impact-pack
0.3 seconds: E:\ComfiUI_conda\custom_nodes\ComfyUI-WanAnimatePreprocess
0.6 seconds: E:\ComfiUI_conda\custom_nodes\comfyui-manager
2.1 seconds: E:\ComfiUI_conda\custom_nodes\teacache
3.2 seconds: E:\ComfiUI_conda\custom_nodes\tts_audio_suite
5.0 seconds: E:\ComfiUI_conda\custom_nodes\comfyui_fill-nodes
5.3 seconds: E:\ComfiUI_conda\custom_nodes\ComfyUI-WanVideoWrapper
7.5 seconds: E:\ComfiUI_conda\custom_nodes\comfyui-step-audio-editx-tts

Context impl SQLiteImpl.
Will assume non-transactional DDL.
No target revision found.
[TTS Audio Suite] 🎭 Character voices: Found 26 characters, 10 aliases (cached)
[TTS Audio Suite] 🔄 Updating character discovery in background...
Starting server

To see the GUI go to: http://127.0.0.1:8188
[DEPRECATION WARNING] Detected import of deprecated legacy API: /scripts/ui.js. This is likely caused by a custom node extension using outdated APIs. Please update your extensions or contact the extension author for an updated version.
[DEPRECATION WARNING] Detected import of deprecated legacy API: /extensions/core/clipspace.js. This is likely caused by a custom node extension using outdated APIs. Please update your extensions or contact the extension author for an updated version.
[DEPRECATION WARNING] Detected import of deprecated legacy API: /extensions/core/groupNode.js. This is likely caused by a custom node extension using outdated APIs. Please update your extensions or contact the extension author for an updated version.
[DEPRECATION WARNING] Detected import of deprecated legacy API: /extensions/core/widgetInputs.js. This is likely caused by a custom node extension using outdated APIs. Please update your extensions or contact the extension author for an updated version.
FETCH ComfyRegistry Data: 35/116
Loading faiss with AVX2 support.
Successfully loaded faiss with AVX2 support.
[DEPRECATION WARNING] Detected import of deprecated legacy API: /scripts/ui/components/button.js. This is likely caused by a custom node extension using outdated APIs. Please update your extensions or contact the extension author for an updated version.
[DEPRECATION WARNING] Detected import of deprecated legacy API: /scripts/ui/components/buttonGroup.js. This is likely caused by a custom node extension using outdated APIs. Please update your extensions or contact the extension author for an updated version.
🔧 Settings endpoint called
🔧 Received settings: precision=auto, device=auto
🎨 Step Audio EditX inline tags: precision=auto, device=auto
FETCH ComfyRegistry Data: 40/116
FETCH ComfyRegistry Data: 45/116
FETCH ComfyRegistry Data: 50/116
FETCH ComfyRegistry Data: 55/116
FETCH ComfyRegistry Data: 60/116
FETCH ComfyRegistry Data: 65/116
FETCH ComfyRegistry Data: 70/116
FETCH ComfyRegistry Data: 75/116
FETCH ComfyRegistry Data: 80/116
FETCH ComfyRegistry Data: 85/116
FETCH ComfyRegistry Data: 90/116
FETCH ComfyRegistry Data: 95/116
FETCH ComfyRegistry Data: 100/116
FETCH ComfyRegistry Data: 105/116
FETCH ComfyRegistry Data: 110/116
FETCH ComfyRegistry Data: 115/116
FETCH ComfyRegistry Data [DONE]
[ComfyUI-Manager] default cache updated: https://api.comfy.org/nodes
FETCH DATA from: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/custom-node-list.json [DONE]
[ComfyUI-Manager] All startup tasks have been completed.
got prompt
🎙️ VibeVoice Engine configured:
Model: vibevoice-7B on auto
Mode: Native Multi-Speaker
CFG Scale: 3.0, Sampling: False
Attention: sage, Steps: 3
Chunking: Every 10 minutes (~7500 chars)
🎤 TTS Text: Using direct audio input (narrator)
⚠️ TTS Text: Direct audio input has no reference text - F5-TTS engines will fail
🎤 Generating Vibevoice for 'narrator' (lang: en)
🎭 VibeVoice: Character switching mode - found characters: narrator
🎙️ Using VibeVoice native multi-speaker mode for 1 speakers
🎭 Native multi-speaker: Processing 6 segments with characters: ['narrator', 'narrator', 'narrator', 'narrator', 'narrator', 'narrator']
🎤 Speaker inputs connected: ['Speaker 1']
⚠️ Priority: Speaker 1 input overrides ['narrator'] alias - using connected voice
🎭 NATIVE MULTI-SPEAKER - Complete formatted text for VibeVoice:

Speaker 1: *****************************
Speaker 1: *****************************
Speaker 1: *****************************
Speaker 1: *****************************
Speaker 1: *****************************
Speaker 1: *****************************

🎤 Using 4 voice samples for generation
📁 Using local VibeVoice model: E:\ComfiUI_conda\models\TTS\vibevoice\vibevoice-7B
🔄 Loading VibeVoice model 'vibevoice-7B' on cuda...
🧠 Using sage attention
Loading checkpoint shards: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:35<00:00, 3.53s/it]
⚠️ SageAttention not available, using SDPA
✅ VibeVoice model 'vibevoice-7B' loaded successfully
Device: cuda, Attention: sdpa
✅ VibeVoice model 'vibevoice-7B' loaded via unified interface
🔧 EditPostProcessor: Using inline tag settings - precision=auto, device=auto

🎨 EditPostProcessor: Found 1 segment(s) with edit tags

📝 Segment 1 - Applying edit tags:

---

---

---

---

---

---

============================================================
🎨 Applying emotion:angry (2 iterations)
🎨 DEBUG: Inline tag settings: precision=auto, device=auto
🔄 Loading Step Audio EditX engine (precision=auto, device=auto, quantization=None)...
🗑️ Moving 1 TTS models to CPU to free VRAM
🔄 VibeVoice: Smart CPU migration with RAM cleanup to prevent accumulation
🔍 No old VibeVoice models found in RAM
📥 Moving VibeVoice to CPU (RAM cleanup completed)
🔄 Moved tts model (vibevoice) to CPU, freed 17821MB
download models from model hub: ms
Using model path: E:\ComfiUI_conda\models\TTS\step_audio_editx\FunASR-Paraformer
Loading pretrained params from E:\ComfiUI_conda\models\TTS\step_audio_editx\FunASR-Paraformer\model.pb
ckpt: E:\ComfiUI_conda\models\TTS\step_audio_editx\FunASR-Paraformer\model.pb
Loading Transformers model from local: E:\ComfiUI_conda\models\TTS\step_audio_editx\Step-Audio-EditX
Successfully loaded model from local
CUDA Graph initialized successfully for chunk decoder
CUDA Graph initialized successfully for chunk generator
D:\anaconda\envs\py313\Lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py:123: UserWarning: Specified provider 'CUDAExecutionProvider' is not in available provider names.Available providers: 'AzureExecutionProvider, CPUExecutionProvider'
warnings.warn(
✅ Step Audio EditX model loaded via unified interface on cuda

[StepAudio] 🚀 Generation started (max 8192 tokens)...
Progress: [░░░░░░░░░░░░] 130/8192 | 0.2 it/s | 761s

[diodiogod]

vibevoice 7b is not the best model to be used with step audio editx editing because vibevoice is a model that likes to generate long segments with multiple speakers, and editing long segments is either not viable (30s is the limit), and it does not work with multiple speakers.

Step Audio EditX is unfortunately a slow model. There isn't much we can do about it.
But it should not be filling up your vram. if you have 24Gb of vram, you can use vibevoice 7b and TTS Audio Suite should be moving it to your CPU RAM after generation before loading Step editX to do the editing. You should be getting around 24-18 it/s on the editing pass.

Can you check your VRAM during all of this? VibeVoice should be unloaded and then Step EditX loaded (which shouldn't use more than 10GB of your VRAM)

[diodiogod]

Also max tokens are not the amount of tokens it will generate, it should be stopping way before that.

Test using the '🎨 Step Audio EditX - Audio Editor' by itself so you can test it on it's own to see what is happening.

[avogatro]

I do not know exactly what I did, but I think after uninstall onnxruntime onnxruntime_gpu and reinstall onnxruntime_gpu only:
using only Step Audio EditX engine, it actually stop before max token at 240/1024 with my longest segment. Max segment processing time was 216s. 6 segments in total. speed is at 5 it/s. I stopped using emotion tag here.
The problem is, voice cloning did not work, Using direct audio input for source_audio, it replaced my chosen voice with standard english male.

My best result is using prev version of TTS Audio Suite + indextts-2 or newest version of TTS Audio Suite with vibevoice over 30s to avoid post edit of emotion tag. I just have to remove noise from vibevoice somehow.

I retested vibevoice, with Step Audio EditX at post, it still use all 24GB of GPU Memory, at 0.2 it/s. in the 90th iteration. It get slower over time.

[diodiogod]

Voice cloning for Step Audio EditX is exactly like f5, it needs a ref text. It won't work with direct audio input. 5it/s is not normal I think... but I don't know what else to do.
Your message sounds like post edit emotion tags are unavoidable when using indextts2, but you don't need to use them.