Worktree embedded browser#33
Open
suitedaces wants to merge 11 commits into
Open
Conversation
Replace Playwright-backed browser tool with an embedded Electron implementation. Each tab is a WebContentsView attached to the desktop window, and agent actions go through the webContents debugger instead of an external Chromium process. - WebContentsView-backed tabs, managed by BrowserController in main - backendNodeId-anchored refs (e1, e2...) survive reflows/re-renders - YAML snapshot format for agent-facing output - interactiveOnly mode emits a flat list of actionable roles only - webContents.printToPDF() fallback since Page.printToPDF is not exposed through Electron's debugger - BrowserView component in renderer for user-driven tab UI - headless browser-host package for agent-only use - parity test suite covering 34 behaviors (33/34 passing; press_key Enter firing submit handler still under investigation)
Input.dispatchKeyEvent without a text field dispatches keydown/keyup but chromium doesn't route character-producing keys through the path that fires default actions like implicit form submission, tab focus moves, or backspace deletion in inputs. Playwright sets text: '\r' for Enter; do the same for Enter/Tab/Space/Backspace. Completes the parity suite: 34/34 passing.
- add origin field ('user' | 'agent') to browser tab summaries so renderer can distinguish
- agent-created tabs auto-adopted into UI via new adoptBrowserTab helper
- popups inherit origin from opener
- icon-only new-browser button in tab bar with tooltip (Globe, ⌘⇧B)
- Globe icon for browser tabs in tab list
- ⌘⇧B entry in ShortcutHelp
- widen TabType exclusions for openViewTab/handleNavClick to cover 'browser'
…reopen - add invalidateAllViews() called on mainWindow show/focus/restore to sidestep Electron #42059 (inactive window pauses WebContentsView paint) - add bringToFront(pageId) to re-assert z-order; wire into BrowserView on isActive + on pageId resolve so multi-tab restore doesn't race - crash recovery: render-process-gone auto-reloads once, exposes tab-crashed/tab-load-failed to renderer, Retry UI in BrowserView - explicit #FFFFFF backgroundColor on WebContentsView to avoid vibrancy bleed-through (Electron #42335) - sync debugger detach in shutdown() so persist partition stays clean - guard toSummary against destroyed webContents
When a browser tab is focused, Chromium consumes keydown events before
they reach the renderer window — so Cmd+T, Cmd+W, Cmd+P, Cmd+Shift+Arrow,
and every other app shortcut stopped working.
- main intercepts Cmd/Ctrl keypresses on each WebContentsView via
before-input-event
- browser-native shortcuts handled in-controller:
Cmd+L focus URL bar, Cmd+R reload, Cmd+Shift+R hard reload,
Cmd+[ back, Cmd+] forward
- text-editing shortcuts (C/V/X/A/Z/Y, Cmd+F find-in-page) pass through
to Chromium
- everything else forwarded via IPC; renderer re-dispatches as a
synthetic KeyboardEvent so useKeyboardShortcuts catches it unchanged
- tab-switch leak: pushBounds gated on isActive so hidden tabs stop re-asserting bounds and resurrecting the WebContentsView over the active pane - orphan pageId: mountedRef destroys the page if create() resolves after the component unmounts - agent focus-steal on create: new tabs from agent origin adopt with focus:false so the user's active tab doesn't change - agent focus-steal on every CDP call: dropped onTabAgentActivity focus handler entirely - multi-tab smoothness: rAF-coalesced scroll/resize, observers only attach on active tabs
single-pane + agent-created browser tab now splits a new column to the right and drops the tab there, keeping the user's focus on chat. the split only happens on first agent-create per single-pane layout; after that, tabs pile into the existing group. the BrowserStream url bar in the chat transcript is now a button that dispatches dorabot:open-browser-tab — App.tsx focuses the matching tab if still open, otherwise opens a fresh browser tab at that url. this gives the user a one-click path from any agent browser action in chat to the actual tab, even after closing the pane. - useLayout.addColumnAt: opt-in activate:false for agent splits - useTabs.adoptBrowserTab: preferSplit flag; single-pane agent creates split right, multi-pane piles into active pane - App.tsx onTabCreated(origin=agent): passes preferSplit:true - App.tsx: window listener for dorabot:open-browser-tab, focuses by url or opens new tab - BrowserStream: url bar becomes button with hover + ExternalLink icon
security fixes (critical): - reject non-http(s)/about urls everywhere an untrusted url can enter: agent tool output parsed in BrowserStream, window custom events in App.tsx, tab-state entries in openBrowserTab, renderer IPC createPage in browser-controller, agent-driven Page.navigate in browser-ipc. prior to this, a prompt-injected agent emitting [page: javascript:...] or file:///... in tool output could have reached loadURL / Page.navigate when the user clicked the chat url bar. main-process checks duplicate the renderer check intentionally as a trust-boundary defense. - tighten [page: ...] regex from .+ to [^\]]+ so crafted output with extra ] can't smuggle junk into the extracted url. correctness: - atomic split-and-adopt via new addColumnWithTab: creates the new column with the tab already in its pane in one setState, so the fill-empty effect never sees an empty pane and can't spawn a ghost chat tab during an agent browser split. - App.tsx url-match uses normalizeUrl for equality so trailing-slash and port differences don't miss an open tab. - onTabCreated and open-browser-tab effects use refs to avoid re-subscribing on every tabState reference change. ui: - lock icon only shows on https. http urls no longer get the padlock. - clickable chat url bar is gated on isSafeUrl; unsafe urls render as a non-clickable about:blank placeholder. new helper desktop/src/lib/url.ts (renderer); main-process copies inline for trust-boundary reasons, kept in sync by comment.
agent input events (click, key, loadURL) pull OS keyboard focus into the WebContentsView because pages auto-focus inputs, clicks focus their targets, and navigation focuses the new document. if the user isn't on that tab, send focus back to the host window after every sendCdp() so they can keep typing in chat / the url bar.
…conciler Per-BrowserView useEffects for hide/show/setBounds/bringToFront were unreliable under fast refresh, strict mode, and orphan pageIds, leaking ghost views over other tabs. Invert control: renderer reports pane state (bounds + activeBrowserPageId + visible) and main owns show/hide/layering via an idempotent reconciler. - New BrowserTabModel in main, single source of truth. Subscribes to tab-created / tab-closed. Reconciles on every pane or tab change: any tab not claimed by a visible pane with non-zero bounds is hidden. - IPC swap: browser:set-bounds + browser:hide + browser:bring-to-front -> browser:pane-update + browser:pane-remove. - BrowserView keeps its body-div ResizeObserver but pushes through paneUpdate; only the active tab pushes. - EditorGroupPanel clears the claim when active tab is non-browser, paneRemove on unmount. A missed React cleanup can no longer produce a ghost WebContentsView.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
- add setPermissionRequestHandler + setPermissionCheckHandler on the dora-browser session, allowing publickey-credentials-get/create so webauthn (including conditional-UI autofill) works. deny all else with a log. - entitlements.mac.plist: usb + bluetooth for hardware keys and caBLE hybrid transport under hardened runtime. - favicon: carry TabSummary.favicon into BrowserTab, strip on persist, render in TabBar with graceful onError fallback. - loading: surface wc.isLoading() in TabSummary, show a 2px progress bar in BrowserView during navigation. - https chip: lock/globe indicator in the url bar based on scheme.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.