perf: add reverse index to EmbedsRepository to avoid full-map iteration#544
Open
yoshi-taka wants to merge 1 commit into
Open
perf: add reverse index to EmbedsRepository to avoid full-map iteration#544yoshi-taka wants to merge 1 commit into
yoshi-taka wants to merge 1 commit into
Conversation
refreshEmbedsForNote() previously iterated the entire embeds map (O(n)) to find entries referencing a specific note. Add a noteEmbeds reverse index (notePath -> Set<embeddedPath>) for O(k) lookup where k is the number of embeds in the affected note. noteEmbeds is a derived index rebuilt from the source-of-truth embeds map. It is not persisted to cache. rebuildNoteEmbeds() is called after renameFile() to keep the index consistent, since renameFile() updates embeds directly but does not update noteEmbeds incrementally.
Owner
|
Hello and thanks for your PR. I'm just not sure that the potential performance gain is worth the added complexity |
Author
|
Thanks, that makes sense. I agree this may not be worth the extra bookkeeping unless this path is actually hot in real-world vaults. The case I had in mind was a large vault with many embeds, where refreshing embeds for a single note repeatedly ends up scanning the global embed list. If users ever report slow refresh/reindex behavior in embed-heavy vaults, this reverse index might be worth revisiting. Happy to leave it as a future optimization rather than adding complexity now. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
refreshEmbedsForNote() previously iterated the entire embeds map to find entries referencing a specific note. Add a derived noteEmbeds reverse index (notePath -> Set) so refresh only touches embeds previously referenced by the affected note instead of scanning all embed entries.
embeds remains the source of truth. noteEmbeds is an in-memory derived index and is not persisted to cache. It is rebuilt from embeds during cache load via addEmbed(), and rebuildNoteEmbeds() is also called after renameFile() to reduce consistency risk, since renameFile() updates embeds directly but does not maintain the reverse index incrementally.
This adds some memory overhead and introduces a second in-memory index, but keeps the consistency model simple and avoids stale persisted state.