Skip to content

Integrate oak_sources with Salsa database #1234

Description

@lionel-

Part of #1212
Depends on #1190

Now that I'm almost done with the Salsa work, here is how I think this could work:

  • I no longer think a discovered_packages hook on the semantic index would be a good fit. The index is a Salsa query and it doesn't feel great to have a side effect fired from a query.

    Instead we could set up package discovery on the main loop, mirroring how diagnostics work. After initial workspace and library scans by oak_scan, we walk over all existing files and call File::dependencies(). This is a derived firewall query that lists all package uses from e.g. pkg:: or library(pkg). Then Package::dependencies() and Package::transitive_dependencies() can be used to find out the own dependencies of each discovered packages. These package queries pull from metadata and don't need populated sources.

  • We introduce a new Salsa input:

    pub struct PackageSources {
        #[returns(ref)]
        pub entries: HashMap<String, PathBuf>,
    }

    Package::files() is no longer a Salsa input but a derived query that consults PackageSources. If not there, files() returns None and the LSP operates in partial-knowledge mode. Every query that attempt to consult sources should handle the no-source case gracefully. We might never get them! The nice thing about making files() a query derived from PackageSources is that once sources come in, all these queries are invalidated. Queries that already had sources backdate somewhere close along the query tree after a cheap lookup. Queries that didn't have them recompute from scratch, now with access to sources.

  • The integration point works just like for diagnostics. The DB state is consulted by the main loop after initial library and workspace scans and at each LSP sync points (did_open, did_change, etc).

    The relevant DB state here are the File::dependencies() and Package::transitive_dependencies() methods. After the workspace root scan finishes, the main loop (which has &mut db) walks over every scripts and packages in the workspace, checks dependencies, and for each of them either populates PackageSources entries if the cache already had sources, or sends an update notification otherwise. Either way the package is entered in a hash set to keep track of which packages either already had sources, or for which a notification was sent.

  • When the PackageCache has finished generating sources for a package, it sends a notification to the main loop, which updates the relevant PackageSources entry accordingly, triggering recomputation of all downstream queries. Since that handler is effectively an LSP sync point, we'd relaunch diagnostics from there.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions