LiteLLM unnecessary memory usage when only trying to use paper metadata features

[microfauna]

Hello! I am attempting to only use paper metadata lookup - but PaperQA imports LMI / LiteLLM no matter what, incurring a ~100MB memory penalty. It's imported in init.py even when I only have from paperqa.clients import DocMetadataClient

Is there a way around this? Maybe scoping imports a little tighter.

[dosubot]

Hi @microfauna! I'm Dosu and I’m helping the paper-qa team.

Your import path from paperqa.clients import DocMetadataClient should actually work without pulling in LiteLLM, since the paperqa/clients/__init__.py module only imports gather_with_concurrency from lmi.utils—not the LLM classes themselves.

The issue is that the top-level paperqa/__init__.py imports LiteLLMModel, LiteLLMEmbeddingModel, etc. from lmi, which triggers the heavy LiteLLM load. If something in your import chain is touching that top-level package, you'll hit the ~100MB overhead.

Workaround: Ensure you're importing only from the submodule and not from paperqa itself anywhere in your code:

# Do this:
from paperqa.clients import DocMetadataClient

# Avoid this anywhere in your codebase:
from paperqa import ...
import paperqa

You're right that tighter import scoping would help here. There's been related work in the ldp repo to scope LiteLLM imports after LiteLLM streamlined its __init__.py. A similar lazy-loading approach in paperqa/__init__.py would let users who only need metadata features avoid the LLM dependency entirely.

To reply, just mention @dosu.

---

Share context across your team and agents. Try Dosu.

[jamesbraza]

Thanks @microfauna lemme see what I can do

[microfauna]

On one hand - thanks for the patch! On the other hand - could this have been caused (downstream) by the huge litellm malware breach this week which was also only caught because liteLLM was hogging memory(