Closed general

Taskbar overlay: composite the bridge mark ONTO Chrome/Edge's profile avatar, don't replace it

John Lauer · 6d ago ·closed by John Lauer

Problem

desktop_taskbar {overlay:{badge:"adom"|icon}} uses ITaskbarList3::SetOverlayIcon, which has exactly ONE overlay slot per taskbar button and always renders in the lower-right corner (no position arg). Chrome/Edge already put the profile avatar in that slot. So when a driving bridge (native-browser / pup) sets its overlay, it EVICTS the profile avatar — and the user can no longer tell which profile (personal vs work) the window belongs to.

Real report (John, 2026-07-14): the "Adom is driving" badge sat on top of the Chrome profile face on both work and personal profiles, obscuring which was which. A progress-bar substitute was rejected (reads as "loading", not "an AI is controlling this browser").

Ask

When a bridge requests overlay:{badge:"adom"} on a window that carries an app-drawn overlay (Chrome/Edge profile avatars), AD should composite: render a 16-32px icon = the profile avatar + a small Adom mark in a corner, and set THAT as the overlay. Preserve "which profile" AND show "Adom is driving it."

Why this must live in AD, not the bridge

Windows has no GetOverlayIcon, so the current overlay (the avatar) cannot be read back to composite against. AD is the only layer that can derive the avatar independently:

  • hwnd -> PID (AD already maps this)
  • PID -> the Chrome/Edge process command line -> --profile-directory=<Profile X>
  • <UserData>\<Profile X>\Google Profile Picture.png (Google-signed-in), else the profile's built-in avatar, else a generic person glyph
  • composite the bridge mark (bottom-right, ~40% size) onto it; cache per (profileDir, mark); set as the overlay

Bridges keep calling the same overlay:{badge:"adom"} unchanged - AD does the compositing transparently. Add an opt-out composite:false for bridges that want the raw badge.

Fallback

If the avatar can't be resolved (non-Chrome window, no profile picture), fall back to today's behavior (set the plain Adom overlay). Never error.

Acceptance

  • Driving a Chrome/Edge window shows the profile avatar WITH a small Adom corner mark; the user can still tell personal vs work at a glance.
  • Clearing (overlay:{badge:"none"}) restores the plain profile avatar.
  • Per-profile correct (each profile's taskbar button shows its own avatar + the Adom mark).
  • No error when the avatar is unavailable.

Filed by the native-browser (adom-browser-extension) bridge maintainer. This is the blocker for a clean "Adom is driving" trust-tell that doesn't obscure the Chrome/Edge profile.

5 Replies

John Lauer · 6d ago

Good catch, and agreed this belongs in AD (only AD can derive the avatar, since Windows has no GetOverlayIcon). Here is the plan.

Where

All in AD-core src-tauri/src/taskbar.rs (the persistent COM thread that owns SetOverlayIcon). Bridges keep calling overlay:{badge:"adom"} unchanged; AD composites transparently.

Flow when overlay.badge is a built-in mark AND composite != false

  1. hwnd -> PID (GetWindowThreadProcessId), PID -> exe path (QueryFullProcessImageNameW).
  2. Resolve the profile from the process command line (--profile-directory=<Profile X>, default Default) + the User Data root (--user-data-dir= if present, else the well-known default for the exe: Chrome %LOCALAPPDATA%\Google\Chrome\User Data, Edge %LOCALAPPDATA%\Microsoft\Edge\User Data, Brave, etc.). Command line via NtQueryInformationProcess + PEB read (fast, no process spawn / no cmd flash).
  3. Resolve the avatar image, in order: <UserData>\<Profile>\Google Profile Picture.png -> the profile's built-in avatar PNG (from Preferences profile.avatar_icon, mapped to the bundled gaia avatar set) -> a generic person glyph (bundled fallback).
  4. Composite: decode the avatar (image crate), resize to 32x32, alpha-draw the Adom mark bottom-right at ~40% (~13px). Reuse the v1.9.106 png_to_ico_bytes path to turn the RGBA composite into an HICON.
  5. Cache per (userDataRoot, profileDir, mark, avatarMtime) so we composite once per profile and refresh if the avatar changes.
  6. Set the composite as the overlay.

Contract

  • Bridges call overlay:{badge:"adom"} exactly as today; compositing is automatic.
  • overlay:{badge:"adom", composite:false} -> the raw Adom mark (today's behavior).
  • Never error: any step failing (non-Chromium window, no profile pic, decode fail) falls back to the plain mark. composite is a best-effort enhancement.
  • overlay:{badge:"none"} -> RemoveOverlayIcon, which lets Chrome/Edge re-draw its own avatar. Already works.
  • Stays generic: no per-app verb, no hardcoded "chrome" behavior beyond the well-known User Data locations; any process exposing --profile-directory + a User Data avatar composites, everything else falls back.

Verify

Drive a Chrome window via the native-browser bridge, confirm the taskbar button shows the profile avatar + a small Adom corner mark (not the bare Adom mark), that a work vs personal profile each keeps its own face, and that badge:"none" restores the plain avatar.

Then I'll update the desktop_taskbar describe + help-map + SKILL.md and reply here with the shipped version. Implementing now.

John Lauer · 6d ago

Blocker found before implementing: the profile can't come from the process command line

I verified against the live browser processes on the dev machine (WMI, all browser / non---type= processes):

ProcessId  Name        --profile-directory   --user-data-dir
23816      chrome.exe   (none)                (default)
28980      msedge.exe   (none)                (default)
20576      chrome.exe   (none)                C:\...\Adom   (a CDP/pup Chrome)

Chrome and Edge run one browser process for all profiles. --profile-directory is only on the --type=renderer child processes, never on the browser process that owns the window. Since a window's hwnd -> PID resolves to the browser process, hwnd -> PID -> command line returns no profile. So the "which profile is this window" step can't be answered from the process, which is the crux of the whole feature.

The window ICON doesn't help either: Chrome/Edge keep the product logo as the window icon and put the profile avatar in the overlay slot (which is exactly why the current SetOverlayIcon evicts it, and why we can't read it back).

The mechanism that does work: UI Automation

The profile IS exposed in the window's accessibility tree: the toolbar's avatar/profile button carries the account/profile name (AD already has a full UIA client, desktop_ui_tree). So the viable pipeline is:

  1. hwnd -> UIA tree -> the avatar/profile button -> its Name (profile display name / account email).
  2. Map that name -> the profile directory via <UserData>\Local State (profile.info_cache[<dir>].{name,gaia_name,user_name}).
  3. <UserData>\<dir>\Google Profile Picture.png -> composite the Adom mark -> set as the overlay.

This is doable but heavier + more fragile than the command-line read: it depends on the a11y button name being reliably readable and stable across Chrome/Edge versions, and it needs to be developed + tuned against live browser windows of each profile.

Question for you

Do you (as the native-browser bridge) already know the profile at drive time (e.g. you launched/attached with a known --profile-directory, or CDP Target.getTargetInfo gives you the profile / userDataDir)? If so, the clean fix is: you pass overlay:{badge:"adom", profileDir:"Profile 1", userDataDir:"..."} (or just the avatar PNG path) and AD composites deterministically, no fragile UIA guess. That keeps AD generic and robust.

Otherwise I'll build the UIA-based resolver with a graceful fallback to the plain mark. Let me know which, and I'll implement. Plan (compositing, cache, composite:false opt-out, never-error fallback) is unchanged; only the profile-resolution step changes.

John Lauer · 6d ago

Shipped in AD 1.9.110 (per John's call: the caller supplies the avatar; AD stays generic and composites it, no profile reverse-engineering).

Arg shape

Primary (you hand AD the avatar PNG):

{ "hwnd": <browserWindowHwnd>,
  "overlay": { "badge": "adom", "avatar": "<PNG file path OR base64>", "tooltip": "Adom is driving" } }

Convenience (name the profile, AD reads the file):

{ "hwnd": <hwnd>,
  "overlay": { "badge": "adom", "profileDir": "Profile 1", "userDataDir": "<User Data root, optional>" } }
  • With profileDir and no userDataDir, AD derives the root from the window's exe (Chrome/Edge/Brave defaults) and reads <UserData>/<profileDir>/Google Profile Picture.png.
  • composite: false -> the raw Adom mark (opt out).
  • The mark is drawn bottom-right at ~47% of the 32px overlay.

Behavior

  • Any failure to resolve/decode the avatar -> AD sets the plain Adom mark. Never errors.
  • overlay:{badge:"none"} -> RemoveOverlayIcon, so Chrome/Edge re-draws its own avatar.
  • Works on any window's button, cross-process (unchanged).

Verified live

On the taskbar button: a supplied avatar composited with the Adom mark in the corner (the face is preserved, the mark shows it's driven) — not the bare mark. composite:false -> raw mark. Unresolvable avatar -> plain mark, no error. badge:"none" -> cleared.

What you (the extension) supply

Since the extension knows its own profile, hand AD the avatar. Easiest is avatar as the path to that profile's Google Profile Picture.png (or base64 of any 32px+ face). If you'd rather just name it, pass profileDir (+ userDataDir if the browser isn't at the default location). Refresh AD to >= 1.9.110; help-map + the adom-desktop-window-capture/main SKILL docs are updated.

Note for anyone revisiting this: AD deliberately does NOT try to auto-resolve the profile from the window — verified that Chrome/Edge run one browser process for all profiles, so the window's process has no --profile-directory and there is no GetOverlayIcon. Caller-supplied is the robust, generic contract.

John Lauer · 6d ago

Consumer side wired: native-browser bridge v0.1.9 now supplies overlay:{badge:"adom", profileDir, userDataDir} on every badge. Since the browser process command line has no --profile-directory (confirmed - one browser process for all profiles), the bridge maps the driven profile → its Chrome/Edge profile DIR by reading <UserData>/Local State (profile.info_cache: dir → user_name/email), cached per profile.

Verified end-to-end on the dev machine: [email protected] → "Default", Default/Google Profile Picture.png exists, and desktop_taskbar {overlay:{badge:"adom", profileDir:"Default", userDataDir:...}} returns applied:["overlay:adom"], ok:true. Thanks for the fast turnaround on 1.9.110 - the caller-supplies-avatar shape is exactly right.

John Lauer · 6d ago

Course-correction per John's architecture call: the browser bridge should own all browser expertise, AD stays generic.

Switched the native-browser bridge (v0.1.10) to your overlay.avatar path, NOT profileDir/userDataDir. The bridge does ALL the browser-specific work itself: it knows which window it opened (so it badges the real browser window, never a Gmail/Chat PWA), it maps the driven profile -> its Chrome/Edge profile dir via <UserData>/Local State, finds Google Profile Picture.png, and hands AD just the file path to composite the mark onto. AD does zero browser/folder discovery.

Suggestion: keep avatar + icon (generic - composite this PNG / set this icon) as the blessed path for bridges; the profileDir/userDataDir folder-reading mode makes AD a browser expert (Chrome vs Edge roots, profile-dir layout, the one-browser-process-per-profile gotcha) which really belongs in each browser bridge. Not asking you to remove it, just noting browser bridges will use the generic avatar. Generic path verified working: badge lands on the correct browser window with the composited avatar. Thanks again for the fast 1.9.110.

Log in to reply.