[Bug]: Widen `backend` type

[AdrianSosic]

Describe the bug

I think this may be a typing issue that could potentially be addressed by refining the backend type?

Steps or code to reproduce the bug

import narwhals as nw

backend: nw.Implementation = nw.Implementation.PANDAS
nw.from_dict({}, backend=backend)

Expected results

No error is thrown.

Actual results

error: Argument "backend" to "from_dict" has incompatible type "Implementation"; expected "Literal['pandas', 'cudf', 'modin', Implementation.PANDAS, Implementation.CUDF, Implementation.MODIN, 'pyarrow', Implementation.PYARROW, 'polars', Implementation.POLARS] | Module | None"  [arg-type]
Found 1 error in 1 file (checked 1 source file)

Please run narwhals.show_versions() and enter the output below.

Python dependencies:
     narwhals: 2.20.0
        numpy: 2.3.5
       pandas: 2.3.3
        modin: 0.37.1
         cudf: 
      pyarrow: 
      pyspark: 
       polars: 1.42.1
         dask: 
       duckdb: 
         ibis: 
     sqlframe:

[FBruzzesi]

Hey @AdrianSosic that's actually something we added to be more precise in which kind of backend you can pass for the from_* functions that require an eager backend.

You might want to annotate your backend variable as:

if TYPE_CHECKING:
	from narwhals.typing import IntoBackend, EagerAllowed

backend: IntoBackend[EagerAllowed] = nw.Implementation.PANDAS

---

From the next release, plugins can also make it in the mix (see #3794):

if TYPE_CHECKING:
	from narwhals.typing import IntoBackend, EagerAllowed
	from narwhals.plugins import PluginName

backend: IntoBackend[EagerAllowed | PluginName] = nw.Implementation.PANDAS

[AdrianSosic]

Hi @FBruzzesi, thanks for the quick answer and clarification! I see, so you are basically already one step ahead 👌🏼 However, I'm not sure if IntoBackend is a good solution for my specific use case.

For context: in my minimal example, backend is just a module-level variable, but in reality it's an attribute of a Settings class that I built using attrs. Here, I'd prefer to have the rather narrow nw.Implementation enum type because changing the type would have other downstream consequences. For example, we are using cattrs for serialization of our classes, and while I can certainly come up with some additional logic and unstructuring hooks to handle IntoBackend, it seems more like a workaround than what I'd really like to have. In fact, there is a good use case for plain enums: they are dead simple 🙃

[dangotbanned]

@AdrianSosic Implementation would not work because it is too wide for from_dict

Examples

import narwhals as nw

nw.from_dict({}, backend=nw.Implementation.DUCKDB)  # Argument of type "Literal[Implementation.DUCKDB]" cannot be assigned to parameter "backend" of type "IntoBackend[EagerAllowed] | None"

ValueError: duckdb support in Narwhals is lazy-only, but `from_dict` is an eager-only function.

Hint: you may want to use an eager backend and then call `.lazy`, e.g.:

    nw.from_dict({'a': [1, 2]}, backend='pyarrow').lazy('duckdb')

import narwhals as nw

nw.from_dict({}, backend=nw.Implementation.UNKNOWN)  # Argument of type "Literal[Implementation.UNKNOWN]" cannot be assigned to parameter "backend" of type "IntoBackend[EagerAllowed] | None"

AssertionError: Cannot return native namespace from UNKNOWN Implementation

Solution

from typing import Any

import narwhals as nw
from narwhals.typing import EagerAllowed


def solution(backend: EagerAllowed) -> nw.DataFrame[Any]:
    return nw.from_dict({}, backend=backend)


# ok
solution("pandas")
solution(nw.Implementation.PANDAS)

# error
solution("duckdb")
solution(nw.Implementation.DUCKDB)