Sorry for making it separate. I did not think of that yesterday.
Currently GPU-enabled services support selecting the number of GPUs (*_GPU_COUNT), but not specific GPU IDs.
This becomes a limitation on multi-GPU systems where users may want to dedicate GPUs to different workloads. For example:
GPU 0 -> InvokeAI
GPU 1 -> Ollama
GPU 2 -> Ollama
A simple solution would be to support optional GPU pinning through environment variables:
OLLAMA_GPU_DEVICES=1,2
INVOKEAI_GPU_DEVICES=0
with the default remaining:
to preserve the current behavior and full backward compatibility.
This would make GPU allocation predictable, prevent resource contention between services, and improve support for multi-GPU hosts. I currently use Ollama and InvokeAI, but the same functionality may likely be useful for ComfyUI and any future GPU-enabled services added to the stack.
I do not use comfyUI so I have no idea if this also applies.
At the moment *_GPU_COUNT only controls how many GPUs are assigned, not which GPUs are used.
Thanks a lot!
Sorry for making it separate. I did not think of that yesterday.
Currently GPU-enabled services support selecting the number of GPUs (
*_GPU_COUNT), but not specific GPU IDs.This becomes a limitation on multi-GPU systems where users may want to dedicate GPUs to different workloads. For example:
A simple solution would be to support optional GPU pinning through environment variables:
with the default remaining:
to preserve the current behavior and full backward compatibility.
This would make GPU allocation predictable, prevent resource contention between services, and improve support for multi-GPU hosts. I currently use Ollama and InvokeAI, but the same functionality may likely be useful for ComfyUI and any future GPU-enabled services added to the stack.
I do not use comfyUI so I have no idea if this also applies.
At the moment
*_GPU_COUNTonly controls how many GPUs are assigned, not which GPUs are used.Thanks a lot!