{
  "id": "tool-to-segment-ui-images",
  "name": "Tool to segment UI images",
  "description": "Blank project ready for grid editing, structure mapping, and future AI-assisted details.",
  "createdAt": "2026-04-30T13:55:25.671Z",
  "updatedAt": "2026-04-30T14:27:07.069Z",
  "nodeMetadataSchema": [
    {
      "key": "purpose",
      "label": "Purpose",
      "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
    },
    {
      "key": "implementation",
      "label": "Implementation",
      "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
    }
  ],
  "nodeTypeSchema": [
    {
      "name": "page",
      "color": "#92bedd"
    },
    {
      "name": "area",
      "color": "#92ddb1"
    },
    {
      "name": "assembly",
      "color": "#ddc692"
    },
    {
      "name": "module",
      "color": "#dd9c92"
    },
    {
      "name": "component",
      "color": "#ae92dd"
    },
    {
      "name": "note",
      "color": "#92d8dd"
    }
  ],
  "spatialMap": {
    "nodes": [
      {
        "id": "page-main-idea",
        "label": "Main idea",
        "type": "page",
        "status": "planned",
        "metadata": {},
        "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
        "rawContext": "",
        "images": [
          {
            "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
            "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
            "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
            "originalName": "image.png",
            "mimeType": "image/png",
            "size": 209602,
            "createdAt": "2026-04-30T14:27:07.069Z"
          }
        ]
      },
      {
        "id": "module-workspace",
        "label": "Workspace",
        "type": "module",
        "status": "planned",
        "metadata": {
          "purpose": "Primary canvas and layout for image segmentation work.",
          "implementation": "Hosts image view, side panels, and editing tools."
        },
        "context": "Top-level product area for day-to-day editing flow.",
        "rawContext": "",
        "images": []
      },
      {
        "id": "module-image-import",
        "label": "Image import",
        "type": "module",
        "status": "planned",
        "metadata": {
          "purpose": "Bring UI screenshots into the tool for editing.",
          "implementation": "Upload, preview, and initialize project image state."
        },
        "context": "Entry point for segmentation sessions.",
        "rawContext": "",
        "images": []
      },
      {
        "id": "module-grid-editor",
        "label": "Grid editor",
        "type": "module",
        "status": "planned",
        "metadata": {
          "purpose": "Create and adjust segmentation grids over UI images.",
          "implementation": "Overlay grid, drag handles, snapping, and edit interactions."
        },
        "context": "Primary UX surface for manual segmentation.",
        "rawContext": "",
        "images": []
      },
      {
        "id": "feature-cell-selection",
        "label": "Cell selection",
        "type": "feature",
        "status": "planned",
        "metadata": {
          "purpose": "Select, inspect, and multi-select grid cells.",
          "implementation": "Click, marquee, keyboard modifiers, and focus state."
        },
        "context": "Core interaction inside the grid editor.",
        "rawContext": "",
        "images": []
      },
      {
        "id": "feature-grid-controls",
        "label": "Grid controls",
        "type": "feature",
        "status": "planned",
        "metadata": {
          "purpose": "Change row and column layout quickly.",
          "implementation": "Add, remove, resize, and snap grid lines."
        },
        "context": "Editor controls for shaping segmentation boundaries.",
        "rawContext": "",
        "images": []
      },
      {
        "id": "module-structure-map",
        "label": "Structure map",
        "type": "module",
        "status": "planned",
        "metadata": {
          "purpose": "Organize segmented regions into UI structure.",
          "implementation": "Tree or panel for naming, grouping, and ordering regions."
        },
        "context": "Maps visual segments into a usable hierarchy.",
        "rawContext": "",
        "images": []
      },
      {
        "id": "feature-region-hierarchy",
        "label": "Region hierarchy",
        "type": "feature",
        "status": "planned",
        "metadata": {
          "purpose": "Define parent-child relationships between regions.",
          "implementation": "Nested region tree with reorder and regroup actions."
        },
        "context": "Detailed structure editing within the mapping module.",
        "rawContext": "",
        "images": []
      },
      {
        "id": "module-export",
        "label": "Export",
        "type": "module",
        "status": "planned",
        "metadata": {
          "purpose": "Output segmentation and structure data for downstream use.",
          "implementation": "Serialize grid and region data into portable formats."
        },
        "context": "Delivery surface for results.",
        "rawContext": "",
        "images": []
      },
      {
        "id": "module-ai-assist",
        "label": "AI assist",
        "type": "module",
        "status": "planned",
        "metadata": {
          "purpose": "Suggest segments and structure to speed editing.",
          "implementation": "Optional AI proposals layered onto manual workflow."
        },
        "context": "Future-facing assistive module built on editor state.",
        "rawContext": "",
        "images": []
      },
      {
        "id": "feature-upload-dropzone",
        "label": "Upload dropzone",
        "type": "feature",
        "status": "planned",
        "metadata": {
          "purpose": "Start the flow by adding a UI image.",
          "implementation": "Drag-drop area and file picker for screenshots."
        },
        "context": "Key import action for the import→segment flow.",
        "rawContext": "",
        "images": []
      },
      {
        "id": "feature-snap-toggle",
        "label": "Snap toggle",
        "type": "feature",
        "status": "planned",
        "metadata": {
          "purpose": "Align grid edits to image edges and guides.",
          "implementation": "Toolbar toggle for snap on drag and resize."
        },
        "context": "Prominent grid-edit control for fast segmentation.",
        "rawContext": "",
        "images": []
      },
      {
        "id": "feature-boundary-nudge",
        "label": "Boundary nudge",
        "type": "feature",
        "status": "planned",
        "metadata": {
          "purpose": "Refine selected segment borders manually.",
          "implementation": "Drag handles and keyboard nudges for boundaries."
        },
        "context": "Supports the manual refine flow after initial grid placement.",
        "rawContext": "",
        "images": []
      },
      {
        "id": "feature-region-naming",
        "label": "Region naming",
        "type": "feature",
        "status": "planned",
        "metadata": {
          "purpose": "Give segmented regions clear labels.",
          "implementation": "Inline rename field in the structure panel."
        },
        "context": "Connects segmentation output to usable structure mapping.",
        "rawContext": "",
        "images": []
      },
      {
        "id": "feature-export-download",
        "label": "Export download",
        "type": "feature",
        "status": "planned",
        "metadata": {
          "purpose": "Finish the flow with a downloadable result.",
          "implementation": "Export button with format choice and file save."
        },
        "context": "Concrete export action for the end of the core flow.",
        "rawContext": "",
        "images": []
      },
      {
        "id": "feature-suggestions-panel",
        "label": "Suggestions panel",
        "type": "feature",
        "status": "planned",
        "metadata": {
          "purpose": "Review AI-proposed segments and structure hints.",
          "implementation": "Side panel listing suggestions with confidence."
        },
        "context": "Primary UI entry point for AI-assisted editing.",
        "rawContext": "",
        "images": []
      },
      {
        "id": "feature-apply-suggestions",
        "label": "Apply suggestions",
        "type": "feature",
        "status": "planned",
        "metadata": {
          "purpose": "Accept or dismiss AI proposals quickly.",
          "implementation": "Per-item and batch apply or reject actions."
        },
        "context": "Turns AI output into editable grid and structure changes.",
        "rawContext": "",
        "images": []
      }
    ],
    "links": [
      {
        "source": "page-main-idea",
        "target": "module-workspace",
        "parentChild": true
      },
      {
        "source": "page-main-idea",
        "target": "module-image-import",
        "parentChild": true
      },
      {
        "source": "page-main-idea",
        "target": "module-grid-editor",
        "parentChild": true
      },
      {
        "source": "module-grid-editor",
        "target": "feature-cell-selection",
        "parentChild": true
      },
      {
        "source": "module-grid-editor",
        "target": "feature-grid-controls",
        "parentChild": true
      },
      {
        "source": "page-main-idea",
        "target": "module-structure-map",
        "parentChild": true
      },
      {
        "source": "module-structure-map",
        "target": "feature-region-hierarchy",
        "parentChild": true
      },
      {
        "source": "page-main-idea",
        "target": "module-export",
        "parentChild": true
      },
      {
        "source": "page-main-idea",
        "target": "module-ai-assist",
        "parentChild": true
      },
      {
        "source": "module-image-import",
        "target": "module-grid-editor",
        "parentChild": false,
        "relationship": "dependency"
      },
      {
        "source": "module-grid-editor",
        "target": "module-structure-map",
        "parentChild": false,
        "relationship": "dependency"
      },
      {
        "source": "module-structure-map",
        "target": "module-export",
        "parentChild": false,
        "relationship": "dependency"
      },
      {
        "source": "module-ai-assist",
        "target": "module-grid-editor",
        "parentChild": false,
        "relationship": "related"
      },
      {
        "source": "module-ai-assist",
        "target": "module-structure-map",
        "parentChild": false,
        "relationship": "reuse"
      },
      {
        "source": "module-workspace",
        "target": "module-grid-editor",
        "parentChild": false,
        "relationship": "related"
      },
      {
        "source": "module-workspace",
        "target": "module-structure-map",
        "parentChild": false,
        "relationship": "related"
      },
      {
        "source": "module-image-import",
        "target": "feature-upload-dropzone",
        "parentChild": true
      },
      {
        "source": "module-grid-editor",
        "target": "feature-snap-toggle",
        "parentChild": true
      },
      {
        "source": "module-grid-editor",
        "target": "feature-boundary-nudge",
        "parentChild": true
      },
      {
        "source": "module-structure-map",
        "target": "feature-region-naming",
        "parentChild": true
      },
      {
        "source": "module-export",
        "target": "feature-export-download",
        "parentChild": true
      },
      {
        "source": "module-ai-assist",
        "target": "feature-suggestions-panel",
        "parentChild": true
      },
      {
        "source": "feature-suggestions-panel",
        "target": "feature-apply-suggestions",
        "parentChild": true
      },
      {
        "source": "feature-upload-dropzone",
        "target": "module-grid-editor",
        "parentChild": false,
        "relationship": "dependency"
      },
      {
        "source": "feature-snap-toggle",
        "target": "module-structure-map",
        "parentChild": false,
        "relationship": "dependency"
      },
      {
        "source": "feature-boundary-nudge",
        "target": "feature-region-naming",
        "parentChild": false,
        "relationship": "related"
      },
      {
        "source": "feature-region-naming",
        "target": "module-export",
        "parentChild": false,
        "relationship": "dependency"
      },
      {
        "source": "feature-apply-suggestions",
        "target": "module-grid-editor",
        "parentChild": false,
        "relationship": "dependency"
      },
      {
        "source": "feature-apply-suggestions",
        "target": "module-structure-map",
        "parentChild": false,
        "relationship": "dependency"
      },
      {
        "source": "feature-apply-suggestions",
        "target": "feature-boundary-nudge",
        "parentChild": false,
        "relationship": "related"
      }
    ]
  },
  "operationLog": [
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-metadata-feature-apply-suggestions",
      "kind": "node.metadata.updated",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled metadata for \"Apply suggestions\".",
      "nodeIds": [
        "feature-apply-suggestions"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              },
              {
                "id": "module-workspace",
                "label": "Workspace",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Primary canvas and layout for image segmentation work.",
                  "implementation": "Hosts image view, side panels, and editing tools."
                },
                "context": "Top-level product area for day-to-day editing flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-image-import",
                "label": "Image import",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Bring UI screenshots into the tool for editing.",
                  "implementation": "Upload, preview, and initialize project image state."
                },
                "context": "Entry point for segmentation sessions.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-grid-editor",
                "label": "Grid editor",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Create and adjust segmentation grids over UI images.",
                  "implementation": "Overlay grid, drag handles, snapping, and edit interactions."
                },
                "context": "Primary UX surface for manual segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-cell-selection",
                "label": "Cell selection",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Select, inspect, and multi-select grid cells.",
                  "implementation": "Click, marquee, keyboard modifiers, and focus state."
                },
                "context": "Core interaction inside the grid editor.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-grid-controls",
                "label": "Grid controls",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Change row and column layout quickly.",
                  "implementation": "Add, remove, resize, and snap grid lines."
                },
                "context": "Editor controls for shaping segmentation boundaries.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-structure-map",
                "label": "Structure map",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Organize segmented regions into UI structure.",
                  "implementation": "Tree or panel for naming, grouping, and ordering regions."
                },
                "context": "Maps visual segments into a usable hierarchy.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-region-hierarchy",
                "label": "Region hierarchy",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Define parent-child relationships between regions.",
                  "implementation": "Nested region tree with reorder and regroup actions."
                },
                "context": "Detailed structure editing within the mapping module.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-export",
                "label": "Export",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Output segmentation and structure data for downstream use.",
                  "implementation": "Serialize grid and region data into portable formats."
                },
                "context": "Delivery surface for results.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-ai-assist",
                "label": "AI assist",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Suggest segments and structure to speed editing.",
                  "implementation": "Optional AI proposals layered onto manual workflow."
                },
                "context": "Future-facing assistive module built on editor state.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-upload-dropzone",
                "label": "Upload dropzone",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Start the flow by adding a UI image.",
                  "implementation": "Drag-drop area and file picker for screenshots."
                },
                "context": "Key import action for the import→segment flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-snap-toggle",
                "label": "Snap toggle",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Align grid edits to image edges and guides.",
                  "implementation": "Toolbar toggle for snap on drag and resize."
                },
                "context": "Prominent grid-edit control for fast segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-boundary-nudge",
                "label": "Boundary nudge",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Refine selected segment borders manually.",
                  "implementation": "Drag handles and keyboard nudges for boundaries."
                },
                "context": "Supports the manual refine flow after initial grid placement.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-region-naming",
                "label": "Region naming",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Give segmented regions clear labels.",
                  "implementation": "Inline rename field in the structure panel."
                },
                "context": "Connects segmentation output to usable structure mapping.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-export-download",
                "label": "Export download",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Finish the flow with a downloadable result.",
                  "implementation": "Export button with format choice and file save."
                },
                "context": "Concrete export action for the end of the core flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-suggestions-panel",
                "label": "Suggestions panel",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Review AI-proposed segments and structure hints.",
                  "implementation": "Side panel listing suggestions with confidence."
                },
                "context": "Primary UI entry point for AI-assisted editing.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-apply-suggestions",
                "label": "Apply suggestions",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Accept or dismiss AI proposals quickly.",
                  "implementation": "Per-item and batch apply or reject actions."
                },
                "context": "Turns AI output into editable grid and structure changes.",
                "rawContext": "",
                "images": []
              }
            ],
            "links": [
              {
                "source": "page-main-idea",
                "target": "module-workspace",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-image-import",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-grid-editor",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-cell-selection",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-grid-controls",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-structure-map",
                "parentChild": true
              },
              {
                "source": "module-structure-map",
                "target": "feature-region-hierarchy",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-export",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-ai-assist",
                "parentChild": true
              },
              {
                "source": "module-image-import",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-grid-editor",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-structure-map",
                "target": "module-export",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-ai-assist",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-ai-assist",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "reuse"
              },
              {
                "source": "module-workspace",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-workspace",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-image-import",
                "target": "feature-upload-dropzone",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-snap-toggle",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-boundary-nudge",
                "parentChild": true
              },
              {
                "source": "module-structure-map",
                "target": "feature-region-naming",
                "parentChild": true
              },
              {
                "source": "module-export",
                "target": "feature-export-download",
                "parentChild": true
              },
              {
                "source": "module-ai-assist",
                "target": "feature-suggestions-panel",
                "parentChild": true
              },
              {
                "source": "feature-suggestions-panel",
                "target": "feature-apply-suggestions",
                "parentChild": true
              },
              {
                "source": "feature-upload-dropzone",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "feature-snap-toggle",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "feature-boundary-nudge",
                "target": "feature-region-naming",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "feature-region-naming",
                "target": "module-export",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "feature-apply-suggestions",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "feature-apply-suggestions",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "feature-apply-suggestions",
                "target": "feature-boundary-nudge",
                "parentChild": false,
                "relationship": "related"
              }
            ]
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-type-feature-apply-suggestions",
      "kind": "node.type.updated",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled \"Apply suggestions\" type as feature.",
      "nodeIds": [
        "feature-apply-suggestions"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              },
              {
                "id": "module-workspace",
                "label": "Workspace",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Primary canvas and layout for image segmentation work.",
                  "implementation": "Hosts image view, side panels, and editing tools."
                },
                "context": "Top-level product area for day-to-day editing flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-image-import",
                "label": "Image import",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Bring UI screenshots into the tool for editing.",
                  "implementation": "Upload, preview, and initialize project image state."
                },
                "context": "Entry point for segmentation sessions.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-grid-editor",
                "label": "Grid editor",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Create and adjust segmentation grids over UI images.",
                  "implementation": "Overlay grid, drag handles, snapping, and edit interactions."
                },
                "context": "Primary UX surface for manual segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-cell-selection",
                "label": "Cell selection",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Select, inspect, and multi-select grid cells.",
                  "implementation": "Click, marquee, keyboard modifiers, and focus state."
                },
                "context": "Core interaction inside the grid editor.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-grid-controls",
                "label": "Grid controls",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Change row and column layout quickly.",
                  "implementation": "Add, remove, resize, and snap grid lines."
                },
                "context": "Editor controls for shaping segmentation boundaries.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-structure-map",
                "label": "Structure map",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Organize segmented regions into UI structure.",
                  "implementation": "Tree or panel for naming, grouping, and ordering regions."
                },
                "context": "Maps visual segments into a usable hierarchy.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-region-hierarchy",
                "label": "Region hierarchy",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Define parent-child relationships between regions.",
                  "implementation": "Nested region tree with reorder and regroup actions."
                },
                "context": "Detailed structure editing within the mapping module.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-export",
                "label": "Export",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Output segmentation and structure data for downstream use.",
                  "implementation": "Serialize grid and region data into portable formats."
                },
                "context": "Delivery surface for results.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-ai-assist",
                "label": "AI assist",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Suggest segments and structure to speed editing.",
                  "implementation": "Optional AI proposals layered onto manual workflow."
                },
                "context": "Future-facing assistive module built on editor state.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-upload-dropzone",
                "label": "Upload dropzone",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Start the flow by adding a UI image.",
                  "implementation": "Drag-drop area and file picker for screenshots."
                },
                "context": "Key import action for the import→segment flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-snap-toggle",
                "label": "Snap toggle",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Align grid edits to image edges and guides.",
                  "implementation": "Toolbar toggle for snap on drag and resize."
                },
                "context": "Prominent grid-edit control for fast segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-boundary-nudge",
                "label": "Boundary nudge",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Refine selected segment borders manually.",
                  "implementation": "Drag handles and keyboard nudges for boundaries."
                },
                "context": "Supports the manual refine flow after initial grid placement.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-region-naming",
                "label": "Region naming",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Give segmented regions clear labels.",
                  "implementation": "Inline rename field in the structure panel."
                },
                "context": "Connects segmentation output to usable structure mapping.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-export-download",
                "label": "Export download",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Finish the flow with a downloadable result.",
                  "implementation": "Export button with format choice and file save."
                },
                "context": "Concrete export action for the end of the core flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-suggestions-panel",
                "label": "Suggestions panel",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Review AI-proposed segments and structure hints.",
                  "implementation": "Side panel listing suggestions with confidence."
                },
                "context": "Primary UI entry point for AI-assisted editing.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-apply-suggestions",
                "label": "Apply suggestions",
                "type": "feature",
                "status": "planned",
                "metadata": {},
                "context": "Turns AI output into editable grid and structure changes.",
                "rawContext": "",
                "images": []
              }
            ],
            "links": [
              {
                "source": "page-main-idea",
                "target": "module-workspace",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-image-import",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-grid-editor",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-cell-selection",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-grid-controls",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-structure-map",
                "parentChild": true
              },
              {
                "source": "module-structure-map",
                "target": "feature-region-hierarchy",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-export",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-ai-assist",
                "parentChild": true
              },
              {
                "source": "module-image-import",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-grid-editor",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-structure-map",
                "target": "module-export",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-ai-assist",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-ai-assist",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "reuse"
              },
              {
                "source": "module-workspace",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-workspace",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-image-import",
                "target": "feature-upload-dropzone",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-snap-toggle",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-boundary-nudge",
                "parentChild": true
              },
              {
                "source": "module-structure-map",
                "target": "feature-region-naming",
                "parentChild": true
              },
              {
                "source": "module-export",
                "target": "feature-export-download",
                "parentChild": true
              },
              {
                "source": "module-ai-assist",
                "target": "feature-suggestions-panel",
                "parentChild": true
              },
              {
                "source": "feature-suggestions-panel",
                "target": "feature-apply-suggestions",
                "parentChild": true
              },
              {
                "source": "feature-upload-dropzone",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "feature-snap-toggle",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "feature-boundary-nudge",
                "target": "feature-region-naming",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "feature-region-naming",
                "target": "module-export",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "feature-apply-suggestions",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "feature-apply-suggestions",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "feature-apply-suggestions",
                "target": "feature-boundary-nudge",
                "parentChild": false,
                "relationship": "related"
              }
            ]
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-created-feature-apply-suggestions",
      "kind": "node.created",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled node \"Apply suggestions\".",
      "nodeIds": [
        "feature-apply-suggestions"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              },
              {
                "id": "module-workspace",
                "label": "Workspace",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Primary canvas and layout for image segmentation work.",
                  "implementation": "Hosts image view, side panels, and editing tools."
                },
                "context": "Top-level product area for day-to-day editing flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-image-import",
                "label": "Image import",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Bring UI screenshots into the tool for editing.",
                  "implementation": "Upload, preview, and initialize project image state."
                },
                "context": "Entry point for segmentation sessions.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-grid-editor",
                "label": "Grid editor",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Create and adjust segmentation grids over UI images.",
                  "implementation": "Overlay grid, drag handles, snapping, and edit interactions."
                },
                "context": "Primary UX surface for manual segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-cell-selection",
                "label": "Cell selection",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Select, inspect, and multi-select grid cells.",
                  "implementation": "Click, marquee, keyboard modifiers, and focus state."
                },
                "context": "Core interaction inside the grid editor.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-grid-controls",
                "label": "Grid controls",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Change row and column layout quickly.",
                  "implementation": "Add, remove, resize, and snap grid lines."
                },
                "context": "Editor controls for shaping segmentation boundaries.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-structure-map",
                "label": "Structure map",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Organize segmented regions into UI structure.",
                  "implementation": "Tree or panel for naming, grouping, and ordering regions."
                },
                "context": "Maps visual segments into a usable hierarchy.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-region-hierarchy",
                "label": "Region hierarchy",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Define parent-child relationships between regions.",
                  "implementation": "Nested region tree with reorder and regroup actions."
                },
                "context": "Detailed structure editing within the mapping module.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-export",
                "label": "Export",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Output segmentation and structure data for downstream use.",
                  "implementation": "Serialize grid and region data into portable formats."
                },
                "context": "Delivery surface for results.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-ai-assist",
                "label": "AI assist",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Suggest segments and structure to speed editing.",
                  "implementation": "Optional AI proposals layered onto manual workflow."
                },
                "context": "Future-facing assistive module built on editor state.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-upload-dropzone",
                "label": "Upload dropzone",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Start the flow by adding a UI image.",
                  "implementation": "Drag-drop area and file picker for screenshots."
                },
                "context": "Key import action for the import→segment flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-snap-toggle",
                "label": "Snap toggle",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Align grid edits to image edges and guides.",
                  "implementation": "Toolbar toggle for snap on drag and resize."
                },
                "context": "Prominent grid-edit control for fast segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-boundary-nudge",
                "label": "Boundary nudge",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Refine selected segment borders manually.",
                  "implementation": "Drag handles and keyboard nudges for boundaries."
                },
                "context": "Supports the manual refine flow after initial grid placement.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-region-naming",
                "label": "Region naming",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Give segmented regions clear labels.",
                  "implementation": "Inline rename field in the structure panel."
                },
                "context": "Connects segmentation output to usable structure mapping.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-export-download",
                "label": "Export download",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Finish the flow with a downloadable result.",
                  "implementation": "Export button with format choice and file save."
                },
                "context": "Concrete export action for the end of the core flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-suggestions-panel",
                "label": "Suggestions panel",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Review AI-proposed segments and structure hints.",
                  "implementation": "Side panel listing suggestions with confidence."
                },
                "context": "Primary UI entry point for AI-assisted editing.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-apply-suggestions",
                "label": "Apply suggestions",
                "type": "feature",
                "status": "planned",
                "metadata": {},
                "context": "Turns AI output into editable grid and structure changes.",
                "rawContext": "",
                "images": []
              }
            ],
            "links": [
              {
                "source": "page-main-idea",
                "target": "module-workspace",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-image-import",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-grid-editor",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-cell-selection",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-grid-controls",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-structure-map",
                "parentChild": true
              },
              {
                "source": "module-structure-map",
                "target": "feature-region-hierarchy",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-export",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-ai-assist",
                "parentChild": true
              },
              {
                "source": "module-image-import",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-grid-editor",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-structure-map",
                "target": "module-export",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-ai-assist",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-ai-assist",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "reuse"
              },
              {
                "source": "module-workspace",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-workspace",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-image-import",
                "target": "feature-upload-dropzone",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-snap-toggle",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-boundary-nudge",
                "parentChild": true
              },
              {
                "source": "module-structure-map",
                "target": "feature-region-naming",
                "parentChild": true
              },
              {
                "source": "module-export",
                "target": "feature-export-download",
                "parentChild": true
              },
              {
                "source": "module-ai-assist",
                "target": "feature-suggestions-panel",
                "parentChild": true
              },
              {
                "source": "feature-suggestions-panel",
                "target": "feature-apply-suggestions",
                "parentChild": true
              },
              {
                "source": "feature-upload-dropzone",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "feature-snap-toggle",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "feature-boundary-nudge",
                "target": "feature-region-naming",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "feature-region-naming",
                "target": "module-export",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "feature-apply-suggestions",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "feature-apply-suggestions",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "feature-apply-suggestions",
                "target": "feature-boundary-nudge",
                "parentChild": false,
                "relationship": "related"
              }
            ]
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-metadata-feature-suggestions-panel",
      "kind": "node.metadata.updated",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled metadata for \"Suggestions panel\".",
      "nodeIds": [
        "feature-suggestions-panel"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              },
              {
                "id": "module-workspace",
                "label": "Workspace",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Primary canvas and layout for image segmentation work.",
                  "implementation": "Hosts image view, side panels, and editing tools."
                },
                "context": "Top-level product area for day-to-day editing flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-image-import",
                "label": "Image import",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Bring UI screenshots into the tool for editing.",
                  "implementation": "Upload, preview, and initialize project image state."
                },
                "context": "Entry point for segmentation sessions.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-grid-editor",
                "label": "Grid editor",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Create and adjust segmentation grids over UI images.",
                  "implementation": "Overlay grid, drag handles, snapping, and edit interactions."
                },
                "context": "Primary UX surface for manual segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-cell-selection",
                "label": "Cell selection",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Select, inspect, and multi-select grid cells.",
                  "implementation": "Click, marquee, keyboard modifiers, and focus state."
                },
                "context": "Core interaction inside the grid editor.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-grid-controls",
                "label": "Grid controls",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Change row and column layout quickly.",
                  "implementation": "Add, remove, resize, and snap grid lines."
                },
                "context": "Editor controls for shaping segmentation boundaries.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-structure-map",
                "label": "Structure map",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Organize segmented regions into UI structure.",
                  "implementation": "Tree or panel for naming, grouping, and ordering regions."
                },
                "context": "Maps visual segments into a usable hierarchy.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-region-hierarchy",
                "label": "Region hierarchy",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Define parent-child relationships between regions.",
                  "implementation": "Nested region tree with reorder and regroup actions."
                },
                "context": "Detailed structure editing within the mapping module.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-export",
                "label": "Export",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Output segmentation and structure data for downstream use.",
                  "implementation": "Serialize grid and region data into portable formats."
                },
                "context": "Delivery surface for results.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-ai-assist",
                "label": "AI assist",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Suggest segments and structure to speed editing.",
                  "implementation": "Optional AI proposals layered onto manual workflow."
                },
                "context": "Future-facing assistive module built on editor state.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-upload-dropzone",
                "label": "Upload dropzone",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Start the flow by adding a UI image.",
                  "implementation": "Drag-drop area and file picker for screenshots."
                },
                "context": "Key import action for the import→segment flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-snap-toggle",
                "label": "Snap toggle",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Align grid edits to image edges and guides.",
                  "implementation": "Toolbar toggle for snap on drag and resize."
                },
                "context": "Prominent grid-edit control for fast segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-boundary-nudge",
                "label": "Boundary nudge",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Refine selected segment borders manually.",
                  "implementation": "Drag handles and keyboard nudges for boundaries."
                },
                "context": "Supports the manual refine flow after initial grid placement.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-region-naming",
                "label": "Region naming",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Give segmented regions clear labels.",
                  "implementation": "Inline rename field in the structure panel."
                },
                "context": "Connects segmentation output to usable structure mapping.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-export-download",
                "label": "Export download",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Finish the flow with a downloadable result.",
                  "implementation": "Export button with format choice and file save."
                },
                "context": "Concrete export action for the end of the core flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-suggestions-panel",
                "label": "Suggestions panel",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Review AI-proposed segments and structure hints.",
                  "implementation": "Side panel listing suggestions with confidence."
                },
                "context": "Primary UI entry point for AI-assisted editing.",
                "rawContext": "",
                "images": []
              }
            ],
            "links": [
              {
                "source": "page-main-idea",
                "target": "module-workspace",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-image-import",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-grid-editor",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-cell-selection",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-grid-controls",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-structure-map",
                "parentChild": true
              },
              {
                "source": "module-structure-map",
                "target": "feature-region-hierarchy",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-export",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-ai-assist",
                "parentChild": true
              },
              {
                "source": "module-image-import",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-grid-editor",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-structure-map",
                "target": "module-export",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-ai-assist",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-ai-assist",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "reuse"
              },
              {
                "source": "module-workspace",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-workspace",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-image-import",
                "target": "feature-upload-dropzone",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-snap-toggle",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-boundary-nudge",
                "parentChild": true
              },
              {
                "source": "module-structure-map",
                "target": "feature-region-naming",
                "parentChild": true
              },
              {
                "source": "module-export",
                "target": "feature-export-download",
                "parentChild": true
              },
              {
                "source": "module-ai-assist",
                "target": "feature-suggestions-panel",
                "parentChild": true
              },
              {
                "source": "feature-upload-dropzone",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "feature-snap-toggle",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "feature-boundary-nudge",
                "target": "feature-region-naming",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "feature-region-naming",
                "target": "module-export",
                "parentChild": false,
                "relationship": "dependency"
              }
            ]
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-type-feature-suggestions-panel",
      "kind": "node.type.updated",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled \"Suggestions panel\" type as feature.",
      "nodeIds": [
        "feature-suggestions-panel"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              },
              {
                "id": "module-workspace",
                "label": "Workspace",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Primary canvas and layout for image segmentation work.",
                  "implementation": "Hosts image view, side panels, and editing tools."
                },
                "context": "Top-level product area for day-to-day editing flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-image-import",
                "label": "Image import",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Bring UI screenshots into the tool for editing.",
                  "implementation": "Upload, preview, and initialize project image state."
                },
                "context": "Entry point for segmentation sessions.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-grid-editor",
                "label": "Grid editor",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Create and adjust segmentation grids over UI images.",
                  "implementation": "Overlay grid, drag handles, snapping, and edit interactions."
                },
                "context": "Primary UX surface for manual segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-cell-selection",
                "label": "Cell selection",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Select, inspect, and multi-select grid cells.",
                  "implementation": "Click, marquee, keyboard modifiers, and focus state."
                },
                "context": "Core interaction inside the grid editor.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-grid-controls",
                "label": "Grid controls",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Change row and column layout quickly.",
                  "implementation": "Add, remove, resize, and snap grid lines."
                },
                "context": "Editor controls for shaping segmentation boundaries.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-structure-map",
                "label": "Structure map",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Organize segmented regions into UI structure.",
                  "implementation": "Tree or panel for naming, grouping, and ordering regions."
                },
                "context": "Maps visual segments into a usable hierarchy.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-region-hierarchy",
                "label": "Region hierarchy",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Define parent-child relationships between regions.",
                  "implementation": "Nested region tree with reorder and regroup actions."
                },
                "context": "Detailed structure editing within the mapping module.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-export",
                "label": "Export",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Output segmentation and structure data for downstream use.",
                  "implementation": "Serialize grid and region data into portable formats."
                },
                "context": "Delivery surface for results.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-ai-assist",
                "label": "AI assist",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Suggest segments and structure to speed editing.",
                  "implementation": "Optional AI proposals layered onto manual workflow."
                },
                "context": "Future-facing assistive module built on editor state.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-upload-dropzone",
                "label": "Upload dropzone",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Start the flow by adding a UI image.",
                  "implementation": "Drag-drop area and file picker for screenshots."
                },
                "context": "Key import action for the import→segment flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-snap-toggle",
                "label": "Snap toggle",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Align grid edits to image edges and guides.",
                  "implementation": "Toolbar toggle for snap on drag and resize."
                },
                "context": "Prominent grid-edit control for fast segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-boundary-nudge",
                "label": "Boundary nudge",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Refine selected segment borders manually.",
                  "implementation": "Drag handles and keyboard nudges for boundaries."
                },
                "context": "Supports the manual refine flow after initial grid placement.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-region-naming",
                "label": "Region naming",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Give segmented regions clear labels.",
                  "implementation": "Inline rename field in the structure panel."
                },
                "context": "Connects segmentation output to usable structure mapping.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-export-download",
                "label": "Export download",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Finish the flow with a downloadable result.",
                  "implementation": "Export button with format choice and file save."
                },
                "context": "Concrete export action for the end of the core flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-suggestions-panel",
                "label": "Suggestions panel",
                "type": "feature",
                "status": "planned",
                "metadata": {},
                "context": "Primary UI entry point for AI-assisted editing.",
                "rawContext": "",
                "images": []
              }
            ],
            "links": [
              {
                "source": "page-main-idea",
                "target": "module-workspace",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-image-import",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-grid-editor",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-cell-selection",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-grid-controls",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-structure-map",
                "parentChild": true
              },
              {
                "source": "module-structure-map",
                "target": "feature-region-hierarchy",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-export",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-ai-assist",
                "parentChild": true
              },
              {
                "source": "module-image-import",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-grid-editor",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-structure-map",
                "target": "module-export",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-ai-assist",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-ai-assist",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "reuse"
              },
              {
                "source": "module-workspace",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-workspace",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-image-import",
                "target": "feature-upload-dropzone",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-snap-toggle",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-boundary-nudge",
                "parentChild": true
              },
              {
                "source": "module-structure-map",
                "target": "feature-region-naming",
                "parentChild": true
              },
              {
                "source": "module-export",
                "target": "feature-export-download",
                "parentChild": true
              },
              {
                "source": "module-ai-assist",
                "target": "feature-suggestions-panel",
                "parentChild": true
              },
              {
                "source": "feature-upload-dropzone",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "feature-snap-toggle",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "feature-boundary-nudge",
                "target": "feature-region-naming",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "feature-region-naming",
                "target": "module-export",
                "parentChild": false,
                "relationship": "dependency"
              }
            ]
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-created-feature-suggestions-panel",
      "kind": "node.created",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled node \"Suggestions panel\".",
      "nodeIds": [
        "feature-suggestions-panel"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              },
              {
                "id": "module-workspace",
                "label": "Workspace",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Primary canvas and layout for image segmentation work.",
                  "implementation": "Hosts image view, side panels, and editing tools."
                },
                "context": "Top-level product area for day-to-day editing flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-image-import",
                "label": "Image import",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Bring UI screenshots into the tool for editing.",
                  "implementation": "Upload, preview, and initialize project image state."
                },
                "context": "Entry point for segmentation sessions.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-grid-editor",
                "label": "Grid editor",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Create and adjust segmentation grids over UI images.",
                  "implementation": "Overlay grid, drag handles, snapping, and edit interactions."
                },
                "context": "Primary UX surface for manual segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-cell-selection",
                "label": "Cell selection",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Select, inspect, and multi-select grid cells.",
                  "implementation": "Click, marquee, keyboard modifiers, and focus state."
                },
                "context": "Core interaction inside the grid editor.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-grid-controls",
                "label": "Grid controls",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Change row and column layout quickly.",
                  "implementation": "Add, remove, resize, and snap grid lines."
                },
                "context": "Editor controls for shaping segmentation boundaries.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-structure-map",
                "label": "Structure map",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Organize segmented regions into UI structure.",
                  "implementation": "Tree or panel for naming, grouping, and ordering regions."
                },
                "context": "Maps visual segments into a usable hierarchy.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-region-hierarchy",
                "label": "Region hierarchy",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Define parent-child relationships between regions.",
                  "implementation": "Nested region tree with reorder and regroup actions."
                },
                "context": "Detailed structure editing within the mapping module.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-export",
                "label": "Export",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Output segmentation and structure data for downstream use.",
                  "implementation": "Serialize grid and region data into portable formats."
                },
                "context": "Delivery surface for results.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-ai-assist",
                "label": "AI assist",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Suggest segments and structure to speed editing.",
                  "implementation": "Optional AI proposals layered onto manual workflow."
                },
                "context": "Future-facing assistive module built on editor state.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-upload-dropzone",
                "label": "Upload dropzone",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Start the flow by adding a UI image.",
                  "implementation": "Drag-drop area and file picker for screenshots."
                },
                "context": "Key import action for the import→segment flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-snap-toggle",
                "label": "Snap toggle",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Align grid edits to image edges and guides.",
                  "implementation": "Toolbar toggle for snap on drag and resize."
                },
                "context": "Prominent grid-edit control for fast segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-boundary-nudge",
                "label": "Boundary nudge",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Refine selected segment borders manually.",
                  "implementation": "Drag handles and keyboard nudges for boundaries."
                },
                "context": "Supports the manual refine flow after initial grid placement.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-region-naming",
                "label": "Region naming",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Give segmented regions clear labels.",
                  "implementation": "Inline rename field in the structure panel."
                },
                "context": "Connects segmentation output to usable structure mapping.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-export-download",
                "label": "Export download",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Finish the flow with a downloadable result.",
                  "implementation": "Export button with format choice and file save."
                },
                "context": "Concrete export action for the end of the core flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-suggestions-panel",
                "label": "Suggestions panel",
                "type": "feature",
                "status": "planned",
                "metadata": {},
                "context": "Primary UI entry point for AI-assisted editing.",
                "rawContext": "",
                "images": []
              }
            ],
            "links": [
              {
                "source": "page-main-idea",
                "target": "module-workspace",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-image-import",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-grid-editor",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-cell-selection",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-grid-controls",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-structure-map",
                "parentChild": true
              },
              {
                "source": "module-structure-map",
                "target": "feature-region-hierarchy",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-export",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-ai-assist",
                "parentChild": true
              },
              {
                "source": "module-image-import",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-grid-editor",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-structure-map",
                "target": "module-export",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-ai-assist",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-ai-assist",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "reuse"
              },
              {
                "source": "module-workspace",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-workspace",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-image-import",
                "target": "feature-upload-dropzone",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-snap-toggle",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-boundary-nudge",
                "parentChild": true
              },
              {
                "source": "module-structure-map",
                "target": "feature-region-naming",
                "parentChild": true
              },
              {
                "source": "module-export",
                "target": "feature-export-download",
                "parentChild": true
              },
              {
                "source": "module-ai-assist",
                "target": "feature-suggestions-panel",
                "parentChild": true
              },
              {
                "source": "feature-upload-dropzone",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "feature-snap-toggle",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "feature-boundary-nudge",
                "target": "feature-region-naming",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "feature-region-naming",
                "target": "module-export",
                "parentChild": false,
                "relationship": "dependency"
              }
            ]
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-metadata-feature-export-download",
      "kind": "node.metadata.updated",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled metadata for \"Export download\".",
      "nodeIds": [
        "feature-export-download"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              },
              {
                "id": "module-workspace",
                "label": "Workspace",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Primary canvas and layout for image segmentation work.",
                  "implementation": "Hosts image view, side panels, and editing tools."
                },
                "context": "Top-level product area for day-to-day editing flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-image-import",
                "label": "Image import",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Bring UI screenshots into the tool for editing.",
                  "implementation": "Upload, preview, and initialize project image state."
                },
                "context": "Entry point for segmentation sessions.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-grid-editor",
                "label": "Grid editor",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Create and adjust segmentation grids over UI images.",
                  "implementation": "Overlay grid, drag handles, snapping, and edit interactions."
                },
                "context": "Primary UX surface for manual segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-cell-selection",
                "label": "Cell selection",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Select, inspect, and multi-select grid cells.",
                  "implementation": "Click, marquee, keyboard modifiers, and focus state."
                },
                "context": "Core interaction inside the grid editor.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-grid-controls",
                "label": "Grid controls",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Change row and column layout quickly.",
                  "implementation": "Add, remove, resize, and snap grid lines."
                },
                "context": "Editor controls for shaping segmentation boundaries.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-structure-map",
                "label": "Structure map",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Organize segmented regions into UI structure.",
                  "implementation": "Tree or panel for naming, grouping, and ordering regions."
                },
                "context": "Maps visual segments into a usable hierarchy.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-region-hierarchy",
                "label": "Region hierarchy",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Define parent-child relationships between regions.",
                  "implementation": "Nested region tree with reorder and regroup actions."
                },
                "context": "Detailed structure editing within the mapping module.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-export",
                "label": "Export",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Output segmentation and structure data for downstream use.",
                  "implementation": "Serialize grid and region data into portable formats."
                },
                "context": "Delivery surface for results.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-ai-assist",
                "label": "AI assist",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Suggest segments and structure to speed editing.",
                  "implementation": "Optional AI proposals layered onto manual workflow."
                },
                "context": "Future-facing assistive module built on editor state.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-upload-dropzone",
                "label": "Upload dropzone",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Start the flow by adding a UI image.",
                  "implementation": "Drag-drop area and file picker for screenshots."
                },
                "context": "Key import action for the import→segment flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-snap-toggle",
                "label": "Snap toggle",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Align grid edits to image edges and guides.",
                  "implementation": "Toolbar toggle for snap on drag and resize."
                },
                "context": "Prominent grid-edit control for fast segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-boundary-nudge",
                "label": "Boundary nudge",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Refine selected segment borders manually.",
                  "implementation": "Drag handles and keyboard nudges for boundaries."
                },
                "context": "Supports the manual refine flow after initial grid placement.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-region-naming",
                "label": "Region naming",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Give segmented regions clear labels.",
                  "implementation": "Inline rename field in the structure panel."
                },
                "context": "Connects segmentation output to usable structure mapping.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-export-download",
                "label": "Export download",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Finish the flow with a downloadable result.",
                  "implementation": "Export button with format choice and file save."
                },
                "context": "Concrete export action for the end of the core flow.",
                "rawContext": "",
                "images": []
              }
            ],
            "links": [
              {
                "source": "page-main-idea",
                "target": "module-workspace",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-image-import",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-grid-editor",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-cell-selection",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-grid-controls",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-structure-map",
                "parentChild": true
              },
              {
                "source": "module-structure-map",
                "target": "feature-region-hierarchy",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-export",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-ai-assist",
                "parentChild": true
              },
              {
                "source": "module-image-import",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-grid-editor",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-structure-map",
                "target": "module-export",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-ai-assist",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-ai-assist",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "reuse"
              },
              {
                "source": "module-workspace",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-workspace",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-image-import",
                "target": "feature-upload-dropzone",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-snap-toggle",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-boundary-nudge",
                "parentChild": true
              },
              {
                "source": "module-structure-map",
                "target": "feature-region-naming",
                "parentChild": true
              },
              {
                "source": "module-export",
                "target": "feature-export-download",
                "parentChild": true
              },
              {
                "source": "feature-upload-dropzone",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "feature-snap-toggle",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "feature-boundary-nudge",
                "target": "feature-region-naming",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "feature-region-naming",
                "target": "module-export",
                "parentChild": false,
                "relationship": "dependency"
              }
            ]
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-type-feature-export-download",
      "kind": "node.type.updated",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled \"Export download\" type as feature.",
      "nodeIds": [
        "feature-export-download"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              },
              {
                "id": "module-workspace",
                "label": "Workspace",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Primary canvas and layout for image segmentation work.",
                  "implementation": "Hosts image view, side panels, and editing tools."
                },
                "context": "Top-level product area for day-to-day editing flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-image-import",
                "label": "Image import",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Bring UI screenshots into the tool for editing.",
                  "implementation": "Upload, preview, and initialize project image state."
                },
                "context": "Entry point for segmentation sessions.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-grid-editor",
                "label": "Grid editor",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Create and adjust segmentation grids over UI images.",
                  "implementation": "Overlay grid, drag handles, snapping, and edit interactions."
                },
                "context": "Primary UX surface for manual segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-cell-selection",
                "label": "Cell selection",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Select, inspect, and multi-select grid cells.",
                  "implementation": "Click, marquee, keyboard modifiers, and focus state."
                },
                "context": "Core interaction inside the grid editor.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-grid-controls",
                "label": "Grid controls",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Change row and column layout quickly.",
                  "implementation": "Add, remove, resize, and snap grid lines."
                },
                "context": "Editor controls for shaping segmentation boundaries.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-structure-map",
                "label": "Structure map",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Organize segmented regions into UI structure.",
                  "implementation": "Tree or panel for naming, grouping, and ordering regions."
                },
                "context": "Maps visual segments into a usable hierarchy.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-region-hierarchy",
                "label": "Region hierarchy",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Define parent-child relationships between regions.",
                  "implementation": "Nested region tree with reorder and regroup actions."
                },
                "context": "Detailed structure editing within the mapping module.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-export",
                "label": "Export",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Output segmentation and structure data for downstream use.",
                  "implementation": "Serialize grid and region data into portable formats."
                },
                "context": "Delivery surface for results.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-ai-assist",
                "label": "AI assist",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Suggest segments and structure to speed editing.",
                  "implementation": "Optional AI proposals layered onto manual workflow."
                },
                "context": "Future-facing assistive module built on editor state.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-upload-dropzone",
                "label": "Upload dropzone",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Start the flow by adding a UI image.",
                  "implementation": "Drag-drop area and file picker for screenshots."
                },
                "context": "Key import action for the import→segment flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-snap-toggle",
                "label": "Snap toggle",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Align grid edits to image edges and guides.",
                  "implementation": "Toolbar toggle for snap on drag and resize."
                },
                "context": "Prominent grid-edit control for fast segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-boundary-nudge",
                "label": "Boundary nudge",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Refine selected segment borders manually.",
                  "implementation": "Drag handles and keyboard nudges for boundaries."
                },
                "context": "Supports the manual refine flow after initial grid placement.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-region-naming",
                "label": "Region naming",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Give segmented regions clear labels.",
                  "implementation": "Inline rename field in the structure panel."
                },
                "context": "Connects segmentation output to usable structure mapping.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-export-download",
                "label": "Export download",
                "type": "feature",
                "status": "planned",
                "metadata": {},
                "context": "Concrete export action for the end of the core flow.",
                "rawContext": "",
                "images": []
              }
            ],
            "links": [
              {
                "source": "page-main-idea",
                "target": "module-workspace",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-image-import",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-grid-editor",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-cell-selection",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-grid-controls",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-structure-map",
                "parentChild": true
              },
              {
                "source": "module-structure-map",
                "target": "feature-region-hierarchy",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-export",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-ai-assist",
                "parentChild": true
              },
              {
                "source": "module-image-import",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-grid-editor",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-structure-map",
                "target": "module-export",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-ai-assist",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-ai-assist",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "reuse"
              },
              {
                "source": "module-workspace",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-workspace",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-image-import",
                "target": "feature-upload-dropzone",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-snap-toggle",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-boundary-nudge",
                "parentChild": true
              },
              {
                "source": "module-structure-map",
                "target": "feature-region-naming",
                "parentChild": true
              },
              {
                "source": "module-export",
                "target": "feature-export-download",
                "parentChild": true
              },
              {
                "source": "feature-upload-dropzone",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "feature-snap-toggle",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "feature-boundary-nudge",
                "target": "feature-region-naming",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "feature-region-naming",
                "target": "module-export",
                "parentChild": false,
                "relationship": "dependency"
              }
            ]
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-created-feature-export-download",
      "kind": "node.created",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled node \"Export download\".",
      "nodeIds": [
        "feature-export-download"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              },
              {
                "id": "module-workspace",
                "label": "Workspace",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Primary canvas and layout for image segmentation work.",
                  "implementation": "Hosts image view, side panels, and editing tools."
                },
                "context": "Top-level product area for day-to-day editing flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-image-import",
                "label": "Image import",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Bring UI screenshots into the tool for editing.",
                  "implementation": "Upload, preview, and initialize project image state."
                },
                "context": "Entry point for segmentation sessions.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-grid-editor",
                "label": "Grid editor",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Create and adjust segmentation grids over UI images.",
                  "implementation": "Overlay grid, drag handles, snapping, and edit interactions."
                },
                "context": "Primary UX surface for manual segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-cell-selection",
                "label": "Cell selection",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Select, inspect, and multi-select grid cells.",
                  "implementation": "Click, marquee, keyboard modifiers, and focus state."
                },
                "context": "Core interaction inside the grid editor.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-grid-controls",
                "label": "Grid controls",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Change row and column layout quickly.",
                  "implementation": "Add, remove, resize, and snap grid lines."
                },
                "context": "Editor controls for shaping segmentation boundaries.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-structure-map",
                "label": "Structure map",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Organize segmented regions into UI structure.",
                  "implementation": "Tree or panel for naming, grouping, and ordering regions."
                },
                "context": "Maps visual segments into a usable hierarchy.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-region-hierarchy",
                "label": "Region hierarchy",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Define parent-child relationships between regions.",
                  "implementation": "Nested region tree with reorder and regroup actions."
                },
                "context": "Detailed structure editing within the mapping module.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-export",
                "label": "Export",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Output segmentation and structure data for downstream use.",
                  "implementation": "Serialize grid and region data into portable formats."
                },
                "context": "Delivery surface for results.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-ai-assist",
                "label": "AI assist",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Suggest segments and structure to speed editing.",
                  "implementation": "Optional AI proposals layered onto manual workflow."
                },
                "context": "Future-facing assistive module built on editor state.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-upload-dropzone",
                "label": "Upload dropzone",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Start the flow by adding a UI image.",
                  "implementation": "Drag-drop area and file picker for screenshots."
                },
                "context": "Key import action for the import→segment flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-snap-toggle",
                "label": "Snap toggle",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Align grid edits to image edges and guides.",
                  "implementation": "Toolbar toggle for snap on drag and resize."
                },
                "context": "Prominent grid-edit control for fast segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-boundary-nudge",
                "label": "Boundary nudge",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Refine selected segment borders manually.",
                  "implementation": "Drag handles and keyboard nudges for boundaries."
                },
                "context": "Supports the manual refine flow after initial grid placement.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-region-naming",
                "label": "Region naming",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Give segmented regions clear labels.",
                  "implementation": "Inline rename field in the structure panel."
                },
                "context": "Connects segmentation output to usable structure mapping.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-export-download",
                "label": "Export download",
                "type": "feature",
                "status": "planned",
                "metadata": {},
                "context": "Concrete export action for the end of the core flow.",
                "rawContext": "",
                "images": []
              }
            ],
            "links": [
              {
                "source": "page-main-idea",
                "target": "module-workspace",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-image-import",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-grid-editor",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-cell-selection",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-grid-controls",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-structure-map",
                "parentChild": true
              },
              {
                "source": "module-structure-map",
                "target": "feature-region-hierarchy",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-export",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-ai-assist",
                "parentChild": true
              },
              {
                "source": "module-image-import",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-grid-editor",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-structure-map",
                "target": "module-export",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-ai-assist",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-ai-assist",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "reuse"
              },
              {
                "source": "module-workspace",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-workspace",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-image-import",
                "target": "feature-upload-dropzone",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-snap-toggle",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-boundary-nudge",
                "parentChild": true
              },
              {
                "source": "module-structure-map",
                "target": "feature-region-naming",
                "parentChild": true
              },
              {
                "source": "module-export",
                "target": "feature-export-download",
                "parentChild": true
              },
              {
                "source": "feature-upload-dropzone",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "feature-snap-toggle",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "feature-boundary-nudge",
                "target": "feature-region-naming",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "feature-region-naming",
                "target": "module-export",
                "parentChild": false,
                "relationship": "dependency"
              }
            ]
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-metadata-feature-region-naming",
      "kind": "node.metadata.updated",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled metadata for \"Region naming\".",
      "nodeIds": [
        "feature-region-naming"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              },
              {
                "id": "module-workspace",
                "label": "Workspace",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Primary canvas and layout for image segmentation work.",
                  "implementation": "Hosts image view, side panels, and editing tools."
                },
                "context": "Top-level product area for day-to-day editing flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-image-import",
                "label": "Image import",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Bring UI screenshots into the tool for editing.",
                  "implementation": "Upload, preview, and initialize project image state."
                },
                "context": "Entry point for segmentation sessions.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-grid-editor",
                "label": "Grid editor",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Create and adjust segmentation grids over UI images.",
                  "implementation": "Overlay grid, drag handles, snapping, and edit interactions."
                },
                "context": "Primary UX surface for manual segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-cell-selection",
                "label": "Cell selection",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Select, inspect, and multi-select grid cells.",
                  "implementation": "Click, marquee, keyboard modifiers, and focus state."
                },
                "context": "Core interaction inside the grid editor.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-grid-controls",
                "label": "Grid controls",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Change row and column layout quickly.",
                  "implementation": "Add, remove, resize, and snap grid lines."
                },
                "context": "Editor controls for shaping segmentation boundaries.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-structure-map",
                "label": "Structure map",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Organize segmented regions into UI structure.",
                  "implementation": "Tree or panel for naming, grouping, and ordering regions."
                },
                "context": "Maps visual segments into a usable hierarchy.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-region-hierarchy",
                "label": "Region hierarchy",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Define parent-child relationships between regions.",
                  "implementation": "Nested region tree with reorder and regroup actions."
                },
                "context": "Detailed structure editing within the mapping module.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-export",
                "label": "Export",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Output segmentation and structure data for downstream use.",
                  "implementation": "Serialize grid and region data into portable formats."
                },
                "context": "Delivery surface for results.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-ai-assist",
                "label": "AI assist",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Suggest segments and structure to speed editing.",
                  "implementation": "Optional AI proposals layered onto manual workflow."
                },
                "context": "Future-facing assistive module built on editor state.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-upload-dropzone",
                "label": "Upload dropzone",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Start the flow by adding a UI image.",
                  "implementation": "Drag-drop area and file picker for screenshots."
                },
                "context": "Key import action for the import→segment flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-snap-toggle",
                "label": "Snap toggle",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Align grid edits to image edges and guides.",
                  "implementation": "Toolbar toggle for snap on drag and resize."
                },
                "context": "Prominent grid-edit control for fast segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-boundary-nudge",
                "label": "Boundary nudge",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Refine selected segment borders manually.",
                  "implementation": "Drag handles and keyboard nudges for boundaries."
                },
                "context": "Supports the manual refine flow after initial grid placement.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-region-naming",
                "label": "Region naming",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Give segmented regions clear labels.",
                  "implementation": "Inline rename field in the structure panel."
                },
                "context": "Connects segmentation output to usable structure mapping.",
                "rawContext": "",
                "images": []
              }
            ],
            "links": [
              {
                "source": "page-main-idea",
                "target": "module-workspace",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-image-import",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-grid-editor",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-cell-selection",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-grid-controls",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-structure-map",
                "parentChild": true
              },
              {
                "source": "module-structure-map",
                "target": "feature-region-hierarchy",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-export",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-ai-assist",
                "parentChild": true
              },
              {
                "source": "module-image-import",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-grid-editor",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-structure-map",
                "target": "module-export",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-ai-assist",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-ai-assist",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "reuse"
              },
              {
                "source": "module-workspace",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-workspace",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-image-import",
                "target": "feature-upload-dropzone",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-snap-toggle",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-boundary-nudge",
                "parentChild": true
              },
              {
                "source": "module-structure-map",
                "target": "feature-region-naming",
                "parentChild": true
              },
              {
                "source": "feature-upload-dropzone",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "feature-snap-toggle",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "feature-boundary-nudge",
                "target": "feature-region-naming",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "feature-region-naming",
                "target": "module-export",
                "parentChild": false,
                "relationship": "dependency"
              }
            ]
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-type-feature-region-naming",
      "kind": "node.type.updated",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled \"Region naming\" type as feature.",
      "nodeIds": [
        "feature-region-naming"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              },
              {
                "id": "module-workspace",
                "label": "Workspace",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Primary canvas and layout for image segmentation work.",
                  "implementation": "Hosts image view, side panels, and editing tools."
                },
                "context": "Top-level product area for day-to-day editing flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-image-import",
                "label": "Image import",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Bring UI screenshots into the tool for editing.",
                  "implementation": "Upload, preview, and initialize project image state."
                },
                "context": "Entry point for segmentation sessions.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-grid-editor",
                "label": "Grid editor",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Create and adjust segmentation grids over UI images.",
                  "implementation": "Overlay grid, drag handles, snapping, and edit interactions."
                },
                "context": "Primary UX surface for manual segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-cell-selection",
                "label": "Cell selection",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Select, inspect, and multi-select grid cells.",
                  "implementation": "Click, marquee, keyboard modifiers, and focus state."
                },
                "context": "Core interaction inside the grid editor.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-grid-controls",
                "label": "Grid controls",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Change row and column layout quickly.",
                  "implementation": "Add, remove, resize, and snap grid lines."
                },
                "context": "Editor controls for shaping segmentation boundaries.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-structure-map",
                "label": "Structure map",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Organize segmented regions into UI structure.",
                  "implementation": "Tree or panel for naming, grouping, and ordering regions."
                },
                "context": "Maps visual segments into a usable hierarchy.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-region-hierarchy",
                "label": "Region hierarchy",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Define parent-child relationships between regions.",
                  "implementation": "Nested region tree with reorder and regroup actions."
                },
                "context": "Detailed structure editing within the mapping module.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-export",
                "label": "Export",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Output segmentation and structure data for downstream use.",
                  "implementation": "Serialize grid and region data into portable formats."
                },
                "context": "Delivery surface for results.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-ai-assist",
                "label": "AI assist",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Suggest segments and structure to speed editing.",
                  "implementation": "Optional AI proposals layered onto manual workflow."
                },
                "context": "Future-facing assistive module built on editor state.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-upload-dropzone",
                "label": "Upload dropzone",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Start the flow by adding a UI image.",
                  "implementation": "Drag-drop area and file picker for screenshots."
                },
                "context": "Key import action for the import→segment flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-snap-toggle",
                "label": "Snap toggle",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Align grid edits to image edges and guides.",
                  "implementation": "Toolbar toggle for snap on drag and resize."
                },
                "context": "Prominent grid-edit control for fast segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-boundary-nudge",
                "label": "Boundary nudge",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Refine selected segment borders manually.",
                  "implementation": "Drag handles and keyboard nudges for boundaries."
                },
                "context": "Supports the manual refine flow after initial grid placement.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-region-naming",
                "label": "Region naming",
                "type": "feature",
                "status": "planned",
                "metadata": {},
                "context": "Connects segmentation output to usable structure mapping.",
                "rawContext": "",
                "images": []
              }
            ],
            "links": [
              {
                "source": "page-main-idea",
                "target": "module-workspace",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-image-import",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-grid-editor",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-cell-selection",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-grid-controls",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-structure-map",
                "parentChild": true
              },
              {
                "source": "module-structure-map",
                "target": "feature-region-hierarchy",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-export",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-ai-assist",
                "parentChild": true
              },
              {
                "source": "module-image-import",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-grid-editor",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-structure-map",
                "target": "module-export",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-ai-assist",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-ai-assist",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "reuse"
              },
              {
                "source": "module-workspace",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-workspace",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-image-import",
                "target": "feature-upload-dropzone",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-snap-toggle",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-boundary-nudge",
                "parentChild": true
              },
              {
                "source": "module-structure-map",
                "target": "feature-region-naming",
                "parentChild": true
              },
              {
                "source": "feature-upload-dropzone",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "feature-snap-toggle",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "feature-boundary-nudge",
                "target": "feature-region-naming",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "feature-region-naming",
                "target": "module-export",
                "parentChild": false,
                "relationship": "dependency"
              }
            ]
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-created-feature-region-naming",
      "kind": "node.created",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled node \"Region naming\".",
      "nodeIds": [
        "feature-region-naming"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              },
              {
                "id": "module-workspace",
                "label": "Workspace",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Primary canvas and layout for image segmentation work.",
                  "implementation": "Hosts image view, side panels, and editing tools."
                },
                "context": "Top-level product area for day-to-day editing flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-image-import",
                "label": "Image import",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Bring UI screenshots into the tool for editing.",
                  "implementation": "Upload, preview, and initialize project image state."
                },
                "context": "Entry point for segmentation sessions.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-grid-editor",
                "label": "Grid editor",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Create and adjust segmentation grids over UI images.",
                  "implementation": "Overlay grid, drag handles, snapping, and edit interactions."
                },
                "context": "Primary UX surface for manual segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-cell-selection",
                "label": "Cell selection",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Select, inspect, and multi-select grid cells.",
                  "implementation": "Click, marquee, keyboard modifiers, and focus state."
                },
                "context": "Core interaction inside the grid editor.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-grid-controls",
                "label": "Grid controls",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Change row and column layout quickly.",
                  "implementation": "Add, remove, resize, and snap grid lines."
                },
                "context": "Editor controls for shaping segmentation boundaries.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-structure-map",
                "label": "Structure map",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Organize segmented regions into UI structure.",
                  "implementation": "Tree or panel for naming, grouping, and ordering regions."
                },
                "context": "Maps visual segments into a usable hierarchy.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-region-hierarchy",
                "label": "Region hierarchy",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Define parent-child relationships between regions.",
                  "implementation": "Nested region tree with reorder and regroup actions."
                },
                "context": "Detailed structure editing within the mapping module.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-export",
                "label": "Export",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Output segmentation and structure data for downstream use.",
                  "implementation": "Serialize grid and region data into portable formats."
                },
                "context": "Delivery surface for results.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-ai-assist",
                "label": "AI assist",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Suggest segments and structure to speed editing.",
                  "implementation": "Optional AI proposals layered onto manual workflow."
                },
                "context": "Future-facing assistive module built on editor state.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-upload-dropzone",
                "label": "Upload dropzone",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Start the flow by adding a UI image.",
                  "implementation": "Drag-drop area and file picker for screenshots."
                },
                "context": "Key import action for the import→segment flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-snap-toggle",
                "label": "Snap toggle",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Align grid edits to image edges and guides.",
                  "implementation": "Toolbar toggle for snap on drag and resize."
                },
                "context": "Prominent grid-edit control for fast segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-boundary-nudge",
                "label": "Boundary nudge",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Refine selected segment borders manually.",
                  "implementation": "Drag handles and keyboard nudges for boundaries."
                },
                "context": "Supports the manual refine flow after initial grid placement.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-region-naming",
                "label": "Region naming",
                "type": "feature",
                "status": "planned",
                "metadata": {},
                "context": "Connects segmentation output to usable structure mapping.",
                "rawContext": "",
                "images": []
              }
            ],
            "links": [
              {
                "source": "page-main-idea",
                "target": "module-workspace",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-image-import",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-grid-editor",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-cell-selection",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-grid-controls",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-structure-map",
                "parentChild": true
              },
              {
                "source": "module-structure-map",
                "target": "feature-region-hierarchy",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-export",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-ai-assist",
                "parentChild": true
              },
              {
                "source": "module-image-import",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-grid-editor",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-structure-map",
                "target": "module-export",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-ai-assist",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-ai-assist",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "reuse"
              },
              {
                "source": "module-workspace",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-workspace",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-image-import",
                "target": "feature-upload-dropzone",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-snap-toggle",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-boundary-nudge",
                "parentChild": true
              },
              {
                "source": "module-structure-map",
                "target": "feature-region-naming",
                "parentChild": true
              },
              {
                "source": "feature-upload-dropzone",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "feature-snap-toggle",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "feature-boundary-nudge",
                "target": "feature-region-naming",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "feature-region-naming",
                "target": "module-export",
                "parentChild": false,
                "relationship": "dependency"
              }
            ]
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-metadata-feature-boundary-nudge",
      "kind": "node.metadata.updated",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled metadata for \"Boundary nudge\".",
      "nodeIds": [
        "feature-boundary-nudge"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              },
              {
                "id": "module-workspace",
                "label": "Workspace",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Primary canvas and layout for image segmentation work.",
                  "implementation": "Hosts image view, side panels, and editing tools."
                },
                "context": "Top-level product area for day-to-day editing flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-image-import",
                "label": "Image import",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Bring UI screenshots into the tool for editing.",
                  "implementation": "Upload, preview, and initialize project image state."
                },
                "context": "Entry point for segmentation sessions.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-grid-editor",
                "label": "Grid editor",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Create and adjust segmentation grids over UI images.",
                  "implementation": "Overlay grid, drag handles, snapping, and edit interactions."
                },
                "context": "Primary UX surface for manual segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-cell-selection",
                "label": "Cell selection",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Select, inspect, and multi-select grid cells.",
                  "implementation": "Click, marquee, keyboard modifiers, and focus state."
                },
                "context": "Core interaction inside the grid editor.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-grid-controls",
                "label": "Grid controls",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Change row and column layout quickly.",
                  "implementation": "Add, remove, resize, and snap grid lines."
                },
                "context": "Editor controls for shaping segmentation boundaries.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-structure-map",
                "label": "Structure map",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Organize segmented regions into UI structure.",
                  "implementation": "Tree or panel for naming, grouping, and ordering regions."
                },
                "context": "Maps visual segments into a usable hierarchy.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-region-hierarchy",
                "label": "Region hierarchy",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Define parent-child relationships between regions.",
                  "implementation": "Nested region tree with reorder and regroup actions."
                },
                "context": "Detailed structure editing within the mapping module.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-export",
                "label": "Export",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Output segmentation and structure data for downstream use.",
                  "implementation": "Serialize grid and region data into portable formats."
                },
                "context": "Delivery surface for results.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-ai-assist",
                "label": "AI assist",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Suggest segments and structure to speed editing.",
                  "implementation": "Optional AI proposals layered onto manual workflow."
                },
                "context": "Future-facing assistive module built on editor state.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-upload-dropzone",
                "label": "Upload dropzone",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Start the flow by adding a UI image.",
                  "implementation": "Drag-drop area and file picker for screenshots."
                },
                "context": "Key import action for the import→segment flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-snap-toggle",
                "label": "Snap toggle",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Align grid edits to image edges and guides.",
                  "implementation": "Toolbar toggle for snap on drag and resize."
                },
                "context": "Prominent grid-edit control for fast segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-boundary-nudge",
                "label": "Boundary nudge",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Refine selected segment borders manually.",
                  "implementation": "Drag handles and keyboard nudges for boundaries."
                },
                "context": "Supports the manual refine flow after initial grid placement.",
                "rawContext": "",
                "images": []
              }
            ],
            "links": [
              {
                "source": "page-main-idea",
                "target": "module-workspace",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-image-import",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-grid-editor",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-cell-selection",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-grid-controls",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-structure-map",
                "parentChild": true
              },
              {
                "source": "module-structure-map",
                "target": "feature-region-hierarchy",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-export",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-ai-assist",
                "parentChild": true
              },
              {
                "source": "module-image-import",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-grid-editor",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-structure-map",
                "target": "module-export",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-ai-assist",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-ai-assist",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "reuse"
              },
              {
                "source": "module-workspace",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-workspace",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-image-import",
                "target": "feature-upload-dropzone",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-snap-toggle",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-boundary-nudge",
                "parentChild": true
              },
              {
                "source": "feature-upload-dropzone",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "feature-snap-toggle",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              }
            ]
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-type-feature-boundary-nudge",
      "kind": "node.type.updated",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled \"Boundary nudge\" type as feature.",
      "nodeIds": [
        "feature-boundary-nudge"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              },
              {
                "id": "module-workspace",
                "label": "Workspace",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Primary canvas and layout for image segmentation work.",
                  "implementation": "Hosts image view, side panels, and editing tools."
                },
                "context": "Top-level product area for day-to-day editing flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-image-import",
                "label": "Image import",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Bring UI screenshots into the tool for editing.",
                  "implementation": "Upload, preview, and initialize project image state."
                },
                "context": "Entry point for segmentation sessions.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-grid-editor",
                "label": "Grid editor",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Create and adjust segmentation grids over UI images.",
                  "implementation": "Overlay grid, drag handles, snapping, and edit interactions."
                },
                "context": "Primary UX surface for manual segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-cell-selection",
                "label": "Cell selection",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Select, inspect, and multi-select grid cells.",
                  "implementation": "Click, marquee, keyboard modifiers, and focus state."
                },
                "context": "Core interaction inside the grid editor.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-grid-controls",
                "label": "Grid controls",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Change row and column layout quickly.",
                  "implementation": "Add, remove, resize, and snap grid lines."
                },
                "context": "Editor controls for shaping segmentation boundaries.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-structure-map",
                "label": "Structure map",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Organize segmented regions into UI structure.",
                  "implementation": "Tree or panel for naming, grouping, and ordering regions."
                },
                "context": "Maps visual segments into a usable hierarchy.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-region-hierarchy",
                "label": "Region hierarchy",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Define parent-child relationships between regions.",
                  "implementation": "Nested region tree with reorder and regroup actions."
                },
                "context": "Detailed structure editing within the mapping module.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-export",
                "label": "Export",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Output segmentation and structure data for downstream use.",
                  "implementation": "Serialize grid and region data into portable formats."
                },
                "context": "Delivery surface for results.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-ai-assist",
                "label": "AI assist",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Suggest segments and structure to speed editing.",
                  "implementation": "Optional AI proposals layered onto manual workflow."
                },
                "context": "Future-facing assistive module built on editor state.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-upload-dropzone",
                "label": "Upload dropzone",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Start the flow by adding a UI image.",
                  "implementation": "Drag-drop area and file picker for screenshots."
                },
                "context": "Key import action for the import→segment flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-snap-toggle",
                "label": "Snap toggle",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Align grid edits to image edges and guides.",
                  "implementation": "Toolbar toggle for snap on drag and resize."
                },
                "context": "Prominent grid-edit control for fast segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-boundary-nudge",
                "label": "Boundary nudge",
                "type": "feature",
                "status": "planned",
                "metadata": {},
                "context": "Supports the manual refine flow after initial grid placement.",
                "rawContext": "",
                "images": []
              }
            ],
            "links": [
              {
                "source": "page-main-idea",
                "target": "module-workspace",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-image-import",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-grid-editor",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-cell-selection",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-grid-controls",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-structure-map",
                "parentChild": true
              },
              {
                "source": "module-structure-map",
                "target": "feature-region-hierarchy",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-export",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-ai-assist",
                "parentChild": true
              },
              {
                "source": "module-image-import",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-grid-editor",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-structure-map",
                "target": "module-export",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-ai-assist",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-ai-assist",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "reuse"
              },
              {
                "source": "module-workspace",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-workspace",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-image-import",
                "target": "feature-upload-dropzone",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-snap-toggle",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-boundary-nudge",
                "parentChild": true
              },
              {
                "source": "feature-upload-dropzone",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "feature-snap-toggle",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              }
            ]
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-created-feature-boundary-nudge",
      "kind": "node.created",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled node \"Boundary nudge\".",
      "nodeIds": [
        "feature-boundary-nudge"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              },
              {
                "id": "module-workspace",
                "label": "Workspace",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Primary canvas and layout for image segmentation work.",
                  "implementation": "Hosts image view, side panels, and editing tools."
                },
                "context": "Top-level product area for day-to-day editing flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-image-import",
                "label": "Image import",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Bring UI screenshots into the tool for editing.",
                  "implementation": "Upload, preview, and initialize project image state."
                },
                "context": "Entry point for segmentation sessions.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-grid-editor",
                "label": "Grid editor",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Create and adjust segmentation grids over UI images.",
                  "implementation": "Overlay grid, drag handles, snapping, and edit interactions."
                },
                "context": "Primary UX surface for manual segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-cell-selection",
                "label": "Cell selection",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Select, inspect, and multi-select grid cells.",
                  "implementation": "Click, marquee, keyboard modifiers, and focus state."
                },
                "context": "Core interaction inside the grid editor.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-grid-controls",
                "label": "Grid controls",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Change row and column layout quickly.",
                  "implementation": "Add, remove, resize, and snap grid lines."
                },
                "context": "Editor controls for shaping segmentation boundaries.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-structure-map",
                "label": "Structure map",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Organize segmented regions into UI structure.",
                  "implementation": "Tree or panel for naming, grouping, and ordering regions."
                },
                "context": "Maps visual segments into a usable hierarchy.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-region-hierarchy",
                "label": "Region hierarchy",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Define parent-child relationships between regions.",
                  "implementation": "Nested region tree with reorder and regroup actions."
                },
                "context": "Detailed structure editing within the mapping module.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-export",
                "label": "Export",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Output segmentation and structure data for downstream use.",
                  "implementation": "Serialize grid and region data into portable formats."
                },
                "context": "Delivery surface for results.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-ai-assist",
                "label": "AI assist",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Suggest segments and structure to speed editing.",
                  "implementation": "Optional AI proposals layered onto manual workflow."
                },
                "context": "Future-facing assistive module built on editor state.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-upload-dropzone",
                "label": "Upload dropzone",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Start the flow by adding a UI image.",
                  "implementation": "Drag-drop area and file picker for screenshots."
                },
                "context": "Key import action for the import→segment flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-snap-toggle",
                "label": "Snap toggle",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Align grid edits to image edges and guides.",
                  "implementation": "Toolbar toggle for snap on drag and resize."
                },
                "context": "Prominent grid-edit control for fast segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-boundary-nudge",
                "label": "Boundary nudge",
                "type": "feature",
                "status": "planned",
                "metadata": {},
                "context": "Supports the manual refine flow after initial grid placement.",
                "rawContext": "",
                "images": []
              }
            ],
            "links": [
              {
                "source": "page-main-idea",
                "target": "module-workspace",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-image-import",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-grid-editor",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-cell-selection",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-grid-controls",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-structure-map",
                "parentChild": true
              },
              {
                "source": "module-structure-map",
                "target": "feature-region-hierarchy",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-export",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-ai-assist",
                "parentChild": true
              },
              {
                "source": "module-image-import",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-grid-editor",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-structure-map",
                "target": "module-export",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-ai-assist",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-ai-assist",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "reuse"
              },
              {
                "source": "module-workspace",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-workspace",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-image-import",
                "target": "feature-upload-dropzone",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-snap-toggle",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-boundary-nudge",
                "parentChild": true
              },
              {
                "source": "feature-upload-dropzone",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "feature-snap-toggle",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              }
            ]
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-metadata-feature-snap-toggle",
      "kind": "node.metadata.updated",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled metadata for \"Snap toggle\".",
      "nodeIds": [
        "feature-snap-toggle"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              },
              {
                "id": "module-workspace",
                "label": "Workspace",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Primary canvas and layout for image segmentation work.",
                  "implementation": "Hosts image view, side panels, and editing tools."
                },
                "context": "Top-level product area for day-to-day editing flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-image-import",
                "label": "Image import",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Bring UI screenshots into the tool for editing.",
                  "implementation": "Upload, preview, and initialize project image state."
                },
                "context": "Entry point for segmentation sessions.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-grid-editor",
                "label": "Grid editor",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Create and adjust segmentation grids over UI images.",
                  "implementation": "Overlay grid, drag handles, snapping, and edit interactions."
                },
                "context": "Primary UX surface for manual segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-cell-selection",
                "label": "Cell selection",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Select, inspect, and multi-select grid cells.",
                  "implementation": "Click, marquee, keyboard modifiers, and focus state."
                },
                "context": "Core interaction inside the grid editor.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-grid-controls",
                "label": "Grid controls",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Change row and column layout quickly.",
                  "implementation": "Add, remove, resize, and snap grid lines."
                },
                "context": "Editor controls for shaping segmentation boundaries.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-structure-map",
                "label": "Structure map",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Organize segmented regions into UI structure.",
                  "implementation": "Tree or panel for naming, grouping, and ordering regions."
                },
                "context": "Maps visual segments into a usable hierarchy.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-region-hierarchy",
                "label": "Region hierarchy",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Define parent-child relationships between regions.",
                  "implementation": "Nested region tree with reorder and regroup actions."
                },
                "context": "Detailed structure editing within the mapping module.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-export",
                "label": "Export",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Output segmentation and structure data for downstream use.",
                  "implementation": "Serialize grid and region data into portable formats."
                },
                "context": "Delivery surface for results.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-ai-assist",
                "label": "AI assist",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Suggest segments and structure to speed editing.",
                  "implementation": "Optional AI proposals layered onto manual workflow."
                },
                "context": "Future-facing assistive module built on editor state.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-upload-dropzone",
                "label": "Upload dropzone",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Start the flow by adding a UI image.",
                  "implementation": "Drag-drop area and file picker for screenshots."
                },
                "context": "Key import action for the import→segment flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-snap-toggle",
                "label": "Snap toggle",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Align grid edits to image edges and guides.",
                  "implementation": "Toolbar toggle for snap on drag and resize."
                },
                "context": "Prominent grid-edit control for fast segmentation.",
                "rawContext": "",
                "images": []
              }
            ],
            "links": [
              {
                "source": "page-main-idea",
                "target": "module-workspace",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-image-import",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-grid-editor",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-cell-selection",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-grid-controls",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-structure-map",
                "parentChild": true
              },
              {
                "source": "module-structure-map",
                "target": "feature-region-hierarchy",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-export",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-ai-assist",
                "parentChild": true
              },
              {
                "source": "module-image-import",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-grid-editor",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-structure-map",
                "target": "module-export",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-ai-assist",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-ai-assist",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "reuse"
              },
              {
                "source": "module-workspace",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-workspace",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-image-import",
                "target": "feature-upload-dropzone",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-snap-toggle",
                "parentChild": true
              },
              {
                "source": "feature-upload-dropzone",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "feature-snap-toggle",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              }
            ]
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-type-feature-snap-toggle",
      "kind": "node.type.updated",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled \"Snap toggle\" type as feature.",
      "nodeIds": [
        "feature-snap-toggle"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              },
              {
                "id": "module-workspace",
                "label": "Workspace",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Primary canvas and layout for image segmentation work.",
                  "implementation": "Hosts image view, side panels, and editing tools."
                },
                "context": "Top-level product area for day-to-day editing flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-image-import",
                "label": "Image import",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Bring UI screenshots into the tool for editing.",
                  "implementation": "Upload, preview, and initialize project image state."
                },
                "context": "Entry point for segmentation sessions.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-grid-editor",
                "label": "Grid editor",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Create and adjust segmentation grids over UI images.",
                  "implementation": "Overlay grid, drag handles, snapping, and edit interactions."
                },
                "context": "Primary UX surface for manual segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-cell-selection",
                "label": "Cell selection",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Select, inspect, and multi-select grid cells.",
                  "implementation": "Click, marquee, keyboard modifiers, and focus state."
                },
                "context": "Core interaction inside the grid editor.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-grid-controls",
                "label": "Grid controls",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Change row and column layout quickly.",
                  "implementation": "Add, remove, resize, and snap grid lines."
                },
                "context": "Editor controls for shaping segmentation boundaries.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-structure-map",
                "label": "Structure map",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Organize segmented regions into UI structure.",
                  "implementation": "Tree or panel for naming, grouping, and ordering regions."
                },
                "context": "Maps visual segments into a usable hierarchy.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-region-hierarchy",
                "label": "Region hierarchy",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Define parent-child relationships between regions.",
                  "implementation": "Nested region tree with reorder and regroup actions."
                },
                "context": "Detailed structure editing within the mapping module.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-export",
                "label": "Export",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Output segmentation and structure data for downstream use.",
                  "implementation": "Serialize grid and region data into portable formats."
                },
                "context": "Delivery surface for results.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-ai-assist",
                "label": "AI assist",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Suggest segments and structure to speed editing.",
                  "implementation": "Optional AI proposals layered onto manual workflow."
                },
                "context": "Future-facing assistive module built on editor state.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-upload-dropzone",
                "label": "Upload dropzone",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Start the flow by adding a UI image.",
                  "implementation": "Drag-drop area and file picker for screenshots."
                },
                "context": "Key import action for the import→segment flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-snap-toggle",
                "label": "Snap toggle",
                "type": "feature",
                "status": "planned",
                "metadata": {},
                "context": "Prominent grid-edit control for fast segmentation.",
                "rawContext": "",
                "images": []
              }
            ],
            "links": [
              {
                "source": "page-main-idea",
                "target": "module-workspace",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-image-import",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-grid-editor",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-cell-selection",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-grid-controls",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-structure-map",
                "parentChild": true
              },
              {
                "source": "module-structure-map",
                "target": "feature-region-hierarchy",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-export",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-ai-assist",
                "parentChild": true
              },
              {
                "source": "module-image-import",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-grid-editor",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-structure-map",
                "target": "module-export",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-ai-assist",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-ai-assist",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "reuse"
              },
              {
                "source": "module-workspace",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-workspace",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-image-import",
                "target": "feature-upload-dropzone",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-snap-toggle",
                "parentChild": true
              },
              {
                "source": "feature-upload-dropzone",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "feature-snap-toggle",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              }
            ]
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-created-feature-snap-toggle",
      "kind": "node.created",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled node \"Snap toggle\".",
      "nodeIds": [
        "feature-snap-toggle"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              },
              {
                "id": "module-workspace",
                "label": "Workspace",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Primary canvas and layout for image segmentation work.",
                  "implementation": "Hosts image view, side panels, and editing tools."
                },
                "context": "Top-level product area for day-to-day editing flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-image-import",
                "label": "Image import",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Bring UI screenshots into the tool for editing.",
                  "implementation": "Upload, preview, and initialize project image state."
                },
                "context": "Entry point for segmentation sessions.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-grid-editor",
                "label": "Grid editor",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Create and adjust segmentation grids over UI images.",
                  "implementation": "Overlay grid, drag handles, snapping, and edit interactions."
                },
                "context": "Primary UX surface for manual segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-cell-selection",
                "label": "Cell selection",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Select, inspect, and multi-select grid cells.",
                  "implementation": "Click, marquee, keyboard modifiers, and focus state."
                },
                "context": "Core interaction inside the grid editor.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-grid-controls",
                "label": "Grid controls",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Change row and column layout quickly.",
                  "implementation": "Add, remove, resize, and snap grid lines."
                },
                "context": "Editor controls for shaping segmentation boundaries.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-structure-map",
                "label": "Structure map",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Organize segmented regions into UI structure.",
                  "implementation": "Tree or panel for naming, grouping, and ordering regions."
                },
                "context": "Maps visual segments into a usable hierarchy.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-region-hierarchy",
                "label": "Region hierarchy",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Define parent-child relationships between regions.",
                  "implementation": "Nested region tree with reorder and regroup actions."
                },
                "context": "Detailed structure editing within the mapping module.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-export",
                "label": "Export",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Output segmentation and structure data for downstream use.",
                  "implementation": "Serialize grid and region data into portable formats."
                },
                "context": "Delivery surface for results.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-ai-assist",
                "label": "AI assist",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Suggest segments and structure to speed editing.",
                  "implementation": "Optional AI proposals layered onto manual workflow."
                },
                "context": "Future-facing assistive module built on editor state.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-upload-dropzone",
                "label": "Upload dropzone",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Start the flow by adding a UI image.",
                  "implementation": "Drag-drop area and file picker for screenshots."
                },
                "context": "Key import action for the import→segment flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-snap-toggle",
                "label": "Snap toggle",
                "type": "feature",
                "status": "planned",
                "metadata": {},
                "context": "Prominent grid-edit control for fast segmentation.",
                "rawContext": "",
                "images": []
              }
            ],
            "links": [
              {
                "source": "page-main-idea",
                "target": "module-workspace",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-image-import",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-grid-editor",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-cell-selection",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-grid-controls",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-structure-map",
                "parentChild": true
              },
              {
                "source": "module-structure-map",
                "target": "feature-region-hierarchy",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-export",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-ai-assist",
                "parentChild": true
              },
              {
                "source": "module-image-import",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-grid-editor",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-structure-map",
                "target": "module-export",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-ai-assist",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-ai-assist",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "reuse"
              },
              {
                "source": "module-workspace",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-workspace",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-image-import",
                "target": "feature-upload-dropzone",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-snap-toggle",
                "parentChild": true
              },
              {
                "source": "feature-upload-dropzone",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "feature-snap-toggle",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              }
            ]
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-metadata-feature-upload-dropzone",
      "kind": "node.metadata.updated",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled metadata for \"Upload dropzone\".",
      "nodeIds": [
        "feature-upload-dropzone"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              },
              {
                "id": "module-workspace",
                "label": "Workspace",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Primary canvas and layout for image segmentation work.",
                  "implementation": "Hosts image view, side panels, and editing tools."
                },
                "context": "Top-level product area for day-to-day editing flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-image-import",
                "label": "Image import",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Bring UI screenshots into the tool for editing.",
                  "implementation": "Upload, preview, and initialize project image state."
                },
                "context": "Entry point for segmentation sessions.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-grid-editor",
                "label": "Grid editor",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Create and adjust segmentation grids over UI images.",
                  "implementation": "Overlay grid, drag handles, snapping, and edit interactions."
                },
                "context": "Primary UX surface for manual segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-cell-selection",
                "label": "Cell selection",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Select, inspect, and multi-select grid cells.",
                  "implementation": "Click, marquee, keyboard modifiers, and focus state."
                },
                "context": "Core interaction inside the grid editor.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-grid-controls",
                "label": "Grid controls",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Change row and column layout quickly.",
                  "implementation": "Add, remove, resize, and snap grid lines."
                },
                "context": "Editor controls for shaping segmentation boundaries.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-structure-map",
                "label": "Structure map",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Organize segmented regions into UI structure.",
                  "implementation": "Tree or panel for naming, grouping, and ordering regions."
                },
                "context": "Maps visual segments into a usable hierarchy.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-region-hierarchy",
                "label": "Region hierarchy",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Define parent-child relationships between regions.",
                  "implementation": "Nested region tree with reorder and regroup actions."
                },
                "context": "Detailed structure editing within the mapping module.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-export",
                "label": "Export",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Output segmentation and structure data for downstream use.",
                  "implementation": "Serialize grid and region data into portable formats."
                },
                "context": "Delivery surface for results.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-ai-assist",
                "label": "AI assist",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Suggest segments and structure to speed editing.",
                  "implementation": "Optional AI proposals layered onto manual workflow."
                },
                "context": "Future-facing assistive module built on editor state.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-upload-dropzone",
                "label": "Upload dropzone",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Start the flow by adding a UI image.",
                  "implementation": "Drag-drop area and file picker for screenshots."
                },
                "context": "Key import action for the import→segment flow.",
                "rawContext": "",
                "images": []
              }
            ],
            "links": [
              {
                "source": "page-main-idea",
                "target": "module-workspace",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-image-import",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-grid-editor",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-cell-selection",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-grid-controls",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-structure-map",
                "parentChild": true
              },
              {
                "source": "module-structure-map",
                "target": "feature-region-hierarchy",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-export",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-ai-assist",
                "parentChild": true
              },
              {
                "source": "module-image-import",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-grid-editor",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-structure-map",
                "target": "module-export",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-ai-assist",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-ai-assist",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "reuse"
              },
              {
                "source": "module-workspace",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-workspace",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-image-import",
                "target": "feature-upload-dropzone",
                "parentChild": true
              },
              {
                "source": "feature-upload-dropzone",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              }
            ]
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-type-feature-upload-dropzone",
      "kind": "node.type.updated",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled \"Upload dropzone\" type as feature.",
      "nodeIds": [
        "feature-upload-dropzone"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              },
              {
                "id": "module-workspace",
                "label": "Workspace",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Primary canvas and layout for image segmentation work.",
                  "implementation": "Hosts image view, side panels, and editing tools."
                },
                "context": "Top-level product area for day-to-day editing flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-image-import",
                "label": "Image import",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Bring UI screenshots into the tool for editing.",
                  "implementation": "Upload, preview, and initialize project image state."
                },
                "context": "Entry point for segmentation sessions.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-grid-editor",
                "label": "Grid editor",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Create and adjust segmentation grids over UI images.",
                  "implementation": "Overlay grid, drag handles, snapping, and edit interactions."
                },
                "context": "Primary UX surface for manual segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-cell-selection",
                "label": "Cell selection",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Select, inspect, and multi-select grid cells.",
                  "implementation": "Click, marquee, keyboard modifiers, and focus state."
                },
                "context": "Core interaction inside the grid editor.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-grid-controls",
                "label": "Grid controls",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Change row and column layout quickly.",
                  "implementation": "Add, remove, resize, and snap grid lines."
                },
                "context": "Editor controls for shaping segmentation boundaries.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-structure-map",
                "label": "Structure map",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Organize segmented regions into UI structure.",
                  "implementation": "Tree or panel for naming, grouping, and ordering regions."
                },
                "context": "Maps visual segments into a usable hierarchy.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-region-hierarchy",
                "label": "Region hierarchy",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Define parent-child relationships between regions.",
                  "implementation": "Nested region tree with reorder and regroup actions."
                },
                "context": "Detailed structure editing within the mapping module.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-export",
                "label": "Export",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Output segmentation and structure data for downstream use.",
                  "implementation": "Serialize grid and region data into portable formats."
                },
                "context": "Delivery surface for results.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-ai-assist",
                "label": "AI assist",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Suggest segments and structure to speed editing.",
                  "implementation": "Optional AI proposals layered onto manual workflow."
                },
                "context": "Future-facing assistive module built on editor state.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-upload-dropzone",
                "label": "Upload dropzone",
                "type": "feature",
                "status": "planned",
                "metadata": {},
                "context": "Key import action for the import→segment flow.",
                "rawContext": "",
                "images": []
              }
            ],
            "links": [
              {
                "source": "page-main-idea",
                "target": "module-workspace",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-image-import",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-grid-editor",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-cell-selection",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-grid-controls",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-structure-map",
                "parentChild": true
              },
              {
                "source": "module-structure-map",
                "target": "feature-region-hierarchy",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-export",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-ai-assist",
                "parentChild": true
              },
              {
                "source": "module-image-import",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-grid-editor",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-structure-map",
                "target": "module-export",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-ai-assist",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-ai-assist",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "reuse"
              },
              {
                "source": "module-workspace",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-workspace",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-image-import",
                "target": "feature-upload-dropzone",
                "parentChild": true
              },
              {
                "source": "feature-upload-dropzone",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              }
            ]
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-created-feature-upload-dropzone",
      "kind": "node.created",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled node \"Upload dropzone\".",
      "nodeIds": [
        "feature-upload-dropzone"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              },
              {
                "id": "module-workspace",
                "label": "Workspace",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Primary canvas and layout for image segmentation work.",
                  "implementation": "Hosts image view, side panels, and editing tools."
                },
                "context": "Top-level product area for day-to-day editing flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-image-import",
                "label": "Image import",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Bring UI screenshots into the tool for editing.",
                  "implementation": "Upload, preview, and initialize project image state."
                },
                "context": "Entry point for segmentation sessions.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-grid-editor",
                "label": "Grid editor",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Create and adjust segmentation grids over UI images.",
                  "implementation": "Overlay grid, drag handles, snapping, and edit interactions."
                },
                "context": "Primary UX surface for manual segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-cell-selection",
                "label": "Cell selection",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Select, inspect, and multi-select grid cells.",
                  "implementation": "Click, marquee, keyboard modifiers, and focus state."
                },
                "context": "Core interaction inside the grid editor.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-grid-controls",
                "label": "Grid controls",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Change row and column layout quickly.",
                  "implementation": "Add, remove, resize, and snap grid lines."
                },
                "context": "Editor controls for shaping segmentation boundaries.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-structure-map",
                "label": "Structure map",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Organize segmented regions into UI structure.",
                  "implementation": "Tree or panel for naming, grouping, and ordering regions."
                },
                "context": "Maps visual segments into a usable hierarchy.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-region-hierarchy",
                "label": "Region hierarchy",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Define parent-child relationships between regions.",
                  "implementation": "Nested region tree with reorder and regroup actions."
                },
                "context": "Detailed structure editing within the mapping module.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-export",
                "label": "Export",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Output segmentation and structure data for downstream use.",
                  "implementation": "Serialize grid and region data into portable formats."
                },
                "context": "Delivery surface for results.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-ai-assist",
                "label": "AI assist",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Suggest segments and structure to speed editing.",
                  "implementation": "Optional AI proposals layered onto manual workflow."
                },
                "context": "Future-facing assistive module built on editor state.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-upload-dropzone",
                "label": "Upload dropzone",
                "type": "feature",
                "status": "planned",
                "metadata": {},
                "context": "Key import action for the import→segment flow.",
                "rawContext": "",
                "images": []
              }
            ],
            "links": [
              {
                "source": "page-main-idea",
                "target": "module-workspace",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-image-import",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-grid-editor",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-cell-selection",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-grid-controls",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-structure-map",
                "parentChild": true
              },
              {
                "source": "module-structure-map",
                "target": "feature-region-hierarchy",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-export",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-ai-assist",
                "parentChild": true
              },
              {
                "source": "module-image-import",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-grid-editor",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-structure-map",
                "target": "module-export",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-ai-assist",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-ai-assist",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "reuse"
              },
              {
                "source": "module-workspace",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-workspace",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-image-import",
                "target": "feature-upload-dropzone",
                "parentChild": true
              },
              {
                "source": "feature-upload-dropzone",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              }
            ]
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-metadata-module-ai-assist",
      "kind": "node.metadata.updated",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled metadata for \"AI assist\".",
      "nodeIds": [
        "module-ai-assist"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              },
              {
                "id": "module-workspace",
                "label": "Workspace",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Primary canvas and layout for image segmentation work.",
                  "implementation": "Hosts image view, side panels, and editing tools."
                },
                "context": "Top-level product area for day-to-day editing flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-image-import",
                "label": "Image import",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Bring UI screenshots into the tool for editing.",
                  "implementation": "Upload, preview, and initialize project image state."
                },
                "context": "Entry point for segmentation sessions.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-grid-editor",
                "label": "Grid editor",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Create and adjust segmentation grids over UI images.",
                  "implementation": "Overlay grid, drag handles, snapping, and edit interactions."
                },
                "context": "Primary UX surface for manual segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-cell-selection",
                "label": "Cell selection",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Select, inspect, and multi-select grid cells.",
                  "implementation": "Click, marquee, keyboard modifiers, and focus state."
                },
                "context": "Core interaction inside the grid editor.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-grid-controls",
                "label": "Grid controls",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Change row and column layout quickly.",
                  "implementation": "Add, remove, resize, and snap grid lines."
                },
                "context": "Editor controls for shaping segmentation boundaries.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-structure-map",
                "label": "Structure map",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Organize segmented regions into UI structure.",
                  "implementation": "Tree or panel for naming, grouping, and ordering regions."
                },
                "context": "Maps visual segments into a usable hierarchy.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-region-hierarchy",
                "label": "Region hierarchy",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Define parent-child relationships between regions.",
                  "implementation": "Nested region tree with reorder and regroup actions."
                },
                "context": "Detailed structure editing within the mapping module.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-export",
                "label": "Export",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Output segmentation and structure data for downstream use.",
                  "implementation": "Serialize grid and region data into portable formats."
                },
                "context": "Delivery surface for results.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-ai-assist",
                "label": "AI assist",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Suggest segments and structure to speed editing.",
                  "implementation": "Optional AI proposals layered onto manual workflow."
                },
                "context": "Future-facing assistive module built on editor state.",
                "rawContext": "",
                "images": []
              }
            ],
            "links": [
              {
                "source": "page-main-idea",
                "target": "module-workspace",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-image-import",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-grid-editor",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-cell-selection",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-grid-controls",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-structure-map",
                "parentChild": true
              },
              {
                "source": "module-structure-map",
                "target": "feature-region-hierarchy",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-export",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-ai-assist",
                "parentChild": true
              },
              {
                "source": "module-image-import",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-grid-editor",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-structure-map",
                "target": "module-export",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-ai-assist",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-ai-assist",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "reuse"
              },
              {
                "source": "module-workspace",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-workspace",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "related"
              }
            ]
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-type-module-ai-assist",
      "kind": "node.type.updated",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled \"AI assist\" type as module.",
      "nodeIds": [
        "module-ai-assist"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              },
              {
                "id": "module-workspace",
                "label": "Workspace",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Primary canvas and layout for image segmentation work.",
                  "implementation": "Hosts image view, side panels, and editing tools."
                },
                "context": "Top-level product area for day-to-day editing flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-image-import",
                "label": "Image import",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Bring UI screenshots into the tool for editing.",
                  "implementation": "Upload, preview, and initialize project image state."
                },
                "context": "Entry point for segmentation sessions.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-grid-editor",
                "label": "Grid editor",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Create and adjust segmentation grids over UI images.",
                  "implementation": "Overlay grid, drag handles, snapping, and edit interactions."
                },
                "context": "Primary UX surface for manual segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-cell-selection",
                "label": "Cell selection",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Select, inspect, and multi-select grid cells.",
                  "implementation": "Click, marquee, keyboard modifiers, and focus state."
                },
                "context": "Core interaction inside the grid editor.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-grid-controls",
                "label": "Grid controls",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Change row and column layout quickly.",
                  "implementation": "Add, remove, resize, and snap grid lines."
                },
                "context": "Editor controls for shaping segmentation boundaries.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-structure-map",
                "label": "Structure map",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Organize segmented regions into UI structure.",
                  "implementation": "Tree or panel for naming, grouping, and ordering regions."
                },
                "context": "Maps visual segments into a usable hierarchy.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-region-hierarchy",
                "label": "Region hierarchy",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Define parent-child relationships between regions.",
                  "implementation": "Nested region tree with reorder and regroup actions."
                },
                "context": "Detailed structure editing within the mapping module.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-export",
                "label": "Export",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Output segmentation and structure data for downstream use.",
                  "implementation": "Serialize grid and region data into portable formats."
                },
                "context": "Delivery surface for results.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-ai-assist",
                "label": "AI assist",
                "type": "module",
                "status": "planned",
                "metadata": {},
                "context": "Future-facing assistive module built on editor state.",
                "rawContext": "",
                "images": []
              }
            ],
            "links": [
              {
                "source": "page-main-idea",
                "target": "module-workspace",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-image-import",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-grid-editor",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-cell-selection",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-grid-controls",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-structure-map",
                "parentChild": true
              },
              {
                "source": "module-structure-map",
                "target": "feature-region-hierarchy",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-export",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-ai-assist",
                "parentChild": true
              },
              {
                "source": "module-image-import",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-grid-editor",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-structure-map",
                "target": "module-export",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-ai-assist",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-ai-assist",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "reuse"
              },
              {
                "source": "module-workspace",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-workspace",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "related"
              }
            ]
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-created-module-ai-assist",
      "kind": "node.created",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled node \"AI assist\".",
      "nodeIds": [
        "module-ai-assist"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              },
              {
                "id": "module-workspace",
                "label": "Workspace",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Primary canvas and layout for image segmentation work.",
                  "implementation": "Hosts image view, side panels, and editing tools."
                },
                "context": "Top-level product area for day-to-day editing flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-image-import",
                "label": "Image import",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Bring UI screenshots into the tool for editing.",
                  "implementation": "Upload, preview, and initialize project image state."
                },
                "context": "Entry point for segmentation sessions.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-grid-editor",
                "label": "Grid editor",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Create and adjust segmentation grids over UI images.",
                  "implementation": "Overlay grid, drag handles, snapping, and edit interactions."
                },
                "context": "Primary UX surface for manual segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-cell-selection",
                "label": "Cell selection",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Select, inspect, and multi-select grid cells.",
                  "implementation": "Click, marquee, keyboard modifiers, and focus state."
                },
                "context": "Core interaction inside the grid editor.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-grid-controls",
                "label": "Grid controls",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Change row and column layout quickly.",
                  "implementation": "Add, remove, resize, and snap grid lines."
                },
                "context": "Editor controls for shaping segmentation boundaries.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-structure-map",
                "label": "Structure map",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Organize segmented regions into UI structure.",
                  "implementation": "Tree or panel for naming, grouping, and ordering regions."
                },
                "context": "Maps visual segments into a usable hierarchy.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-region-hierarchy",
                "label": "Region hierarchy",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Define parent-child relationships between regions.",
                  "implementation": "Nested region tree with reorder and regroup actions."
                },
                "context": "Detailed structure editing within the mapping module.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-export",
                "label": "Export",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Output segmentation and structure data for downstream use.",
                  "implementation": "Serialize grid and region data into portable formats."
                },
                "context": "Delivery surface for results.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-ai-assist",
                "label": "AI assist",
                "type": "module",
                "status": "planned",
                "metadata": {},
                "context": "Future-facing assistive module built on editor state.",
                "rawContext": "",
                "images": []
              }
            ],
            "links": [
              {
                "source": "page-main-idea",
                "target": "module-workspace",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-image-import",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-grid-editor",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-cell-selection",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-grid-controls",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-structure-map",
                "parentChild": true
              },
              {
                "source": "module-structure-map",
                "target": "feature-region-hierarchy",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-export",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-ai-assist",
                "parentChild": true
              },
              {
                "source": "module-image-import",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-grid-editor",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-structure-map",
                "target": "module-export",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-ai-assist",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-ai-assist",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "reuse"
              },
              {
                "source": "module-workspace",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-workspace",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "related"
              }
            ]
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-metadata-module-export",
      "kind": "node.metadata.updated",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled metadata for \"Export\".",
      "nodeIds": [
        "module-export"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              },
              {
                "id": "module-workspace",
                "label": "Workspace",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Primary canvas and layout for image segmentation work.",
                  "implementation": "Hosts image view, side panels, and editing tools."
                },
                "context": "Top-level product area for day-to-day editing flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-image-import",
                "label": "Image import",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Bring UI screenshots into the tool for editing.",
                  "implementation": "Upload, preview, and initialize project image state."
                },
                "context": "Entry point for segmentation sessions.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-grid-editor",
                "label": "Grid editor",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Create and adjust segmentation grids over UI images.",
                  "implementation": "Overlay grid, drag handles, snapping, and edit interactions."
                },
                "context": "Primary UX surface for manual segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-cell-selection",
                "label": "Cell selection",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Select, inspect, and multi-select grid cells.",
                  "implementation": "Click, marquee, keyboard modifiers, and focus state."
                },
                "context": "Core interaction inside the grid editor.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-grid-controls",
                "label": "Grid controls",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Change row and column layout quickly.",
                  "implementation": "Add, remove, resize, and snap grid lines."
                },
                "context": "Editor controls for shaping segmentation boundaries.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-structure-map",
                "label": "Structure map",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Organize segmented regions into UI structure.",
                  "implementation": "Tree or panel for naming, grouping, and ordering regions."
                },
                "context": "Maps visual segments into a usable hierarchy.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-region-hierarchy",
                "label": "Region hierarchy",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Define parent-child relationships between regions.",
                  "implementation": "Nested region tree with reorder and regroup actions."
                },
                "context": "Detailed structure editing within the mapping module.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-export",
                "label": "Export",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Output segmentation and structure data for downstream use.",
                  "implementation": "Serialize grid and region data into portable formats."
                },
                "context": "Delivery surface for results.",
                "rawContext": "",
                "images": []
              }
            ],
            "links": [
              {
                "source": "page-main-idea",
                "target": "module-workspace",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-image-import",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-grid-editor",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-cell-selection",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-grid-controls",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-structure-map",
                "parentChild": true
              },
              {
                "source": "module-structure-map",
                "target": "feature-region-hierarchy",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-export",
                "parentChild": true
              },
              {
                "source": "module-image-import",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-grid-editor",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-structure-map",
                "target": "module-export",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-workspace",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-workspace",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "related"
              }
            ]
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-type-module-export",
      "kind": "node.type.updated",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled \"Export\" type as module.",
      "nodeIds": [
        "module-export"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              },
              {
                "id": "module-workspace",
                "label": "Workspace",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Primary canvas and layout for image segmentation work.",
                  "implementation": "Hosts image view, side panels, and editing tools."
                },
                "context": "Top-level product area for day-to-day editing flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-image-import",
                "label": "Image import",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Bring UI screenshots into the tool for editing.",
                  "implementation": "Upload, preview, and initialize project image state."
                },
                "context": "Entry point for segmentation sessions.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-grid-editor",
                "label": "Grid editor",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Create and adjust segmentation grids over UI images.",
                  "implementation": "Overlay grid, drag handles, snapping, and edit interactions."
                },
                "context": "Primary UX surface for manual segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-cell-selection",
                "label": "Cell selection",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Select, inspect, and multi-select grid cells.",
                  "implementation": "Click, marquee, keyboard modifiers, and focus state."
                },
                "context": "Core interaction inside the grid editor.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-grid-controls",
                "label": "Grid controls",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Change row and column layout quickly.",
                  "implementation": "Add, remove, resize, and snap grid lines."
                },
                "context": "Editor controls for shaping segmentation boundaries.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-structure-map",
                "label": "Structure map",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Organize segmented regions into UI structure.",
                  "implementation": "Tree or panel for naming, grouping, and ordering regions."
                },
                "context": "Maps visual segments into a usable hierarchy.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-region-hierarchy",
                "label": "Region hierarchy",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Define parent-child relationships between regions.",
                  "implementation": "Nested region tree with reorder and regroup actions."
                },
                "context": "Detailed structure editing within the mapping module.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-export",
                "label": "Export",
                "type": "module",
                "status": "planned",
                "metadata": {},
                "context": "Delivery surface for results.",
                "rawContext": "",
                "images": []
              }
            ],
            "links": [
              {
                "source": "page-main-idea",
                "target": "module-workspace",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-image-import",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-grid-editor",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-cell-selection",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-grid-controls",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-structure-map",
                "parentChild": true
              },
              {
                "source": "module-structure-map",
                "target": "feature-region-hierarchy",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-export",
                "parentChild": true
              },
              {
                "source": "module-image-import",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-grid-editor",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-structure-map",
                "target": "module-export",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-workspace",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-workspace",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "related"
              }
            ]
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-created-module-export",
      "kind": "node.created",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled node \"Export\".",
      "nodeIds": [
        "module-export"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              },
              {
                "id": "module-workspace",
                "label": "Workspace",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Primary canvas and layout for image segmentation work.",
                  "implementation": "Hosts image view, side panels, and editing tools."
                },
                "context": "Top-level product area for day-to-day editing flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-image-import",
                "label": "Image import",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Bring UI screenshots into the tool for editing.",
                  "implementation": "Upload, preview, and initialize project image state."
                },
                "context": "Entry point for segmentation sessions.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-grid-editor",
                "label": "Grid editor",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Create and adjust segmentation grids over UI images.",
                  "implementation": "Overlay grid, drag handles, snapping, and edit interactions."
                },
                "context": "Primary UX surface for manual segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-cell-selection",
                "label": "Cell selection",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Select, inspect, and multi-select grid cells.",
                  "implementation": "Click, marquee, keyboard modifiers, and focus state."
                },
                "context": "Core interaction inside the grid editor.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-grid-controls",
                "label": "Grid controls",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Change row and column layout quickly.",
                  "implementation": "Add, remove, resize, and snap grid lines."
                },
                "context": "Editor controls for shaping segmentation boundaries.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-structure-map",
                "label": "Structure map",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Organize segmented regions into UI structure.",
                  "implementation": "Tree or panel for naming, grouping, and ordering regions."
                },
                "context": "Maps visual segments into a usable hierarchy.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-region-hierarchy",
                "label": "Region hierarchy",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Define parent-child relationships between regions.",
                  "implementation": "Nested region tree with reorder and regroup actions."
                },
                "context": "Detailed structure editing within the mapping module.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-export",
                "label": "Export",
                "type": "module",
                "status": "planned",
                "metadata": {},
                "context": "Delivery surface for results.",
                "rawContext": "",
                "images": []
              }
            ],
            "links": [
              {
                "source": "page-main-idea",
                "target": "module-workspace",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-image-import",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-grid-editor",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-cell-selection",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-grid-controls",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-structure-map",
                "parentChild": true
              },
              {
                "source": "module-structure-map",
                "target": "feature-region-hierarchy",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-export",
                "parentChild": true
              },
              {
                "source": "module-image-import",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-grid-editor",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-structure-map",
                "target": "module-export",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-workspace",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-workspace",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "related"
              }
            ]
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-metadata-feature-region-hierarchy",
      "kind": "node.metadata.updated",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled metadata for \"Region hierarchy\".",
      "nodeIds": [
        "feature-region-hierarchy"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              },
              {
                "id": "module-workspace",
                "label": "Workspace",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Primary canvas and layout for image segmentation work.",
                  "implementation": "Hosts image view, side panels, and editing tools."
                },
                "context": "Top-level product area for day-to-day editing flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-image-import",
                "label": "Image import",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Bring UI screenshots into the tool for editing.",
                  "implementation": "Upload, preview, and initialize project image state."
                },
                "context": "Entry point for segmentation sessions.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-grid-editor",
                "label": "Grid editor",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Create and adjust segmentation grids over UI images.",
                  "implementation": "Overlay grid, drag handles, snapping, and edit interactions."
                },
                "context": "Primary UX surface for manual segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-cell-selection",
                "label": "Cell selection",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Select, inspect, and multi-select grid cells.",
                  "implementation": "Click, marquee, keyboard modifiers, and focus state."
                },
                "context": "Core interaction inside the grid editor.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-grid-controls",
                "label": "Grid controls",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Change row and column layout quickly.",
                  "implementation": "Add, remove, resize, and snap grid lines."
                },
                "context": "Editor controls for shaping segmentation boundaries.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-structure-map",
                "label": "Structure map",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Organize segmented regions into UI structure.",
                  "implementation": "Tree or panel for naming, grouping, and ordering regions."
                },
                "context": "Maps visual segments into a usable hierarchy.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-region-hierarchy",
                "label": "Region hierarchy",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Define parent-child relationships between regions.",
                  "implementation": "Nested region tree with reorder and regroup actions."
                },
                "context": "Detailed structure editing within the mapping module.",
                "rawContext": "",
                "images": []
              }
            ],
            "links": [
              {
                "source": "page-main-idea",
                "target": "module-workspace",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-image-import",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-grid-editor",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-cell-selection",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-grid-controls",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-structure-map",
                "parentChild": true
              },
              {
                "source": "module-structure-map",
                "target": "feature-region-hierarchy",
                "parentChild": true
              },
              {
                "source": "module-image-import",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-grid-editor",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-workspace",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-workspace",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "related"
              }
            ]
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-type-feature-region-hierarchy",
      "kind": "node.type.updated",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled \"Region hierarchy\" type as feature.",
      "nodeIds": [
        "feature-region-hierarchy"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              },
              {
                "id": "module-workspace",
                "label": "Workspace",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Primary canvas and layout for image segmentation work.",
                  "implementation": "Hosts image view, side panels, and editing tools."
                },
                "context": "Top-level product area for day-to-day editing flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-image-import",
                "label": "Image import",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Bring UI screenshots into the tool for editing.",
                  "implementation": "Upload, preview, and initialize project image state."
                },
                "context": "Entry point for segmentation sessions.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-grid-editor",
                "label": "Grid editor",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Create and adjust segmentation grids over UI images.",
                  "implementation": "Overlay grid, drag handles, snapping, and edit interactions."
                },
                "context": "Primary UX surface for manual segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-cell-selection",
                "label": "Cell selection",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Select, inspect, and multi-select grid cells.",
                  "implementation": "Click, marquee, keyboard modifiers, and focus state."
                },
                "context": "Core interaction inside the grid editor.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-grid-controls",
                "label": "Grid controls",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Change row and column layout quickly.",
                  "implementation": "Add, remove, resize, and snap grid lines."
                },
                "context": "Editor controls for shaping segmentation boundaries.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-structure-map",
                "label": "Structure map",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Organize segmented regions into UI structure.",
                  "implementation": "Tree or panel for naming, grouping, and ordering regions."
                },
                "context": "Maps visual segments into a usable hierarchy.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-region-hierarchy",
                "label": "Region hierarchy",
                "type": "feature",
                "status": "planned",
                "metadata": {},
                "context": "Detailed structure editing within the mapping module.",
                "rawContext": "",
                "images": []
              }
            ],
            "links": [
              {
                "source": "page-main-idea",
                "target": "module-workspace",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-image-import",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-grid-editor",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-cell-selection",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-grid-controls",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-structure-map",
                "parentChild": true
              },
              {
                "source": "module-structure-map",
                "target": "feature-region-hierarchy",
                "parentChild": true
              },
              {
                "source": "module-image-import",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-grid-editor",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-workspace",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-workspace",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "related"
              }
            ]
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-created-feature-region-hierarchy",
      "kind": "node.created",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled node \"Region hierarchy\".",
      "nodeIds": [
        "feature-region-hierarchy"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              },
              {
                "id": "module-workspace",
                "label": "Workspace",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Primary canvas and layout for image segmentation work.",
                  "implementation": "Hosts image view, side panels, and editing tools."
                },
                "context": "Top-level product area for day-to-day editing flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-image-import",
                "label": "Image import",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Bring UI screenshots into the tool for editing.",
                  "implementation": "Upload, preview, and initialize project image state."
                },
                "context": "Entry point for segmentation sessions.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-grid-editor",
                "label": "Grid editor",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Create and adjust segmentation grids over UI images.",
                  "implementation": "Overlay grid, drag handles, snapping, and edit interactions."
                },
                "context": "Primary UX surface for manual segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-cell-selection",
                "label": "Cell selection",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Select, inspect, and multi-select grid cells.",
                  "implementation": "Click, marquee, keyboard modifiers, and focus state."
                },
                "context": "Core interaction inside the grid editor.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-grid-controls",
                "label": "Grid controls",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Change row and column layout quickly.",
                  "implementation": "Add, remove, resize, and snap grid lines."
                },
                "context": "Editor controls for shaping segmentation boundaries.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-structure-map",
                "label": "Structure map",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Organize segmented regions into UI structure.",
                  "implementation": "Tree or panel for naming, grouping, and ordering regions."
                },
                "context": "Maps visual segments into a usable hierarchy.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-region-hierarchy",
                "label": "Region hierarchy",
                "type": "feature",
                "status": "planned",
                "metadata": {},
                "context": "Detailed structure editing within the mapping module.",
                "rawContext": "",
                "images": []
              }
            ],
            "links": [
              {
                "source": "page-main-idea",
                "target": "module-workspace",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-image-import",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-grid-editor",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-cell-selection",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-grid-controls",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-structure-map",
                "parentChild": true
              },
              {
                "source": "module-structure-map",
                "target": "feature-region-hierarchy",
                "parentChild": true
              },
              {
                "source": "module-image-import",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-grid-editor",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-workspace",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-workspace",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "related"
              }
            ]
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-metadata-module-structure-map",
      "kind": "node.metadata.updated",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled metadata for \"Structure map\".",
      "nodeIds": [
        "module-structure-map"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              },
              {
                "id": "module-workspace",
                "label": "Workspace",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Primary canvas and layout for image segmentation work.",
                  "implementation": "Hosts image view, side panels, and editing tools."
                },
                "context": "Top-level product area for day-to-day editing flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-image-import",
                "label": "Image import",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Bring UI screenshots into the tool for editing.",
                  "implementation": "Upload, preview, and initialize project image state."
                },
                "context": "Entry point for segmentation sessions.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-grid-editor",
                "label": "Grid editor",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Create and adjust segmentation grids over UI images.",
                  "implementation": "Overlay grid, drag handles, snapping, and edit interactions."
                },
                "context": "Primary UX surface for manual segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-cell-selection",
                "label": "Cell selection",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Select, inspect, and multi-select grid cells.",
                  "implementation": "Click, marquee, keyboard modifiers, and focus state."
                },
                "context": "Core interaction inside the grid editor.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-grid-controls",
                "label": "Grid controls",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Change row and column layout quickly.",
                  "implementation": "Add, remove, resize, and snap grid lines."
                },
                "context": "Editor controls for shaping segmentation boundaries.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-structure-map",
                "label": "Structure map",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Organize segmented regions into UI structure.",
                  "implementation": "Tree or panel for naming, grouping, and ordering regions."
                },
                "context": "Maps visual segments into a usable hierarchy.",
                "rawContext": "",
                "images": []
              }
            ],
            "links": [
              {
                "source": "page-main-idea",
                "target": "module-workspace",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-image-import",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-grid-editor",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-cell-selection",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-grid-controls",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-structure-map",
                "parentChild": true
              },
              {
                "source": "module-image-import",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-grid-editor",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-workspace",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-workspace",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "related"
              }
            ]
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-type-module-structure-map",
      "kind": "node.type.updated",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled \"Structure map\" type as module.",
      "nodeIds": [
        "module-structure-map"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              },
              {
                "id": "module-workspace",
                "label": "Workspace",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Primary canvas and layout for image segmentation work.",
                  "implementation": "Hosts image view, side panels, and editing tools."
                },
                "context": "Top-level product area for day-to-day editing flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-image-import",
                "label": "Image import",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Bring UI screenshots into the tool for editing.",
                  "implementation": "Upload, preview, and initialize project image state."
                },
                "context": "Entry point for segmentation sessions.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-grid-editor",
                "label": "Grid editor",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Create and adjust segmentation grids over UI images.",
                  "implementation": "Overlay grid, drag handles, snapping, and edit interactions."
                },
                "context": "Primary UX surface for manual segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-cell-selection",
                "label": "Cell selection",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Select, inspect, and multi-select grid cells.",
                  "implementation": "Click, marquee, keyboard modifiers, and focus state."
                },
                "context": "Core interaction inside the grid editor.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-grid-controls",
                "label": "Grid controls",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Change row and column layout quickly.",
                  "implementation": "Add, remove, resize, and snap grid lines."
                },
                "context": "Editor controls for shaping segmentation boundaries.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-structure-map",
                "label": "Structure map",
                "type": "module",
                "status": "planned",
                "metadata": {},
                "context": "Maps visual segments into a usable hierarchy.",
                "rawContext": "",
                "images": []
              }
            ],
            "links": [
              {
                "source": "page-main-idea",
                "target": "module-workspace",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-image-import",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-grid-editor",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-cell-selection",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-grid-controls",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-structure-map",
                "parentChild": true
              },
              {
                "source": "module-image-import",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-grid-editor",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-workspace",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-workspace",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "related"
              }
            ]
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-created-module-structure-map",
      "kind": "node.created",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled node \"Structure map\".",
      "nodeIds": [
        "module-structure-map"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              },
              {
                "id": "module-workspace",
                "label": "Workspace",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Primary canvas and layout for image segmentation work.",
                  "implementation": "Hosts image view, side panels, and editing tools."
                },
                "context": "Top-level product area for day-to-day editing flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-image-import",
                "label": "Image import",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Bring UI screenshots into the tool for editing.",
                  "implementation": "Upload, preview, and initialize project image state."
                },
                "context": "Entry point for segmentation sessions.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-grid-editor",
                "label": "Grid editor",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Create and adjust segmentation grids over UI images.",
                  "implementation": "Overlay grid, drag handles, snapping, and edit interactions."
                },
                "context": "Primary UX surface for manual segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-cell-selection",
                "label": "Cell selection",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Select, inspect, and multi-select grid cells.",
                  "implementation": "Click, marquee, keyboard modifiers, and focus state."
                },
                "context": "Core interaction inside the grid editor.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-grid-controls",
                "label": "Grid controls",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Change row and column layout quickly.",
                  "implementation": "Add, remove, resize, and snap grid lines."
                },
                "context": "Editor controls for shaping segmentation boundaries.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-structure-map",
                "label": "Structure map",
                "type": "module",
                "status": "planned",
                "metadata": {},
                "context": "Maps visual segments into a usable hierarchy.",
                "rawContext": "",
                "images": []
              }
            ],
            "links": [
              {
                "source": "page-main-idea",
                "target": "module-workspace",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-image-import",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-grid-editor",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-cell-selection",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-grid-controls",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-structure-map",
                "parentChild": true
              },
              {
                "source": "module-image-import",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-grid-editor",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-workspace",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              },
              {
                "source": "module-workspace",
                "target": "module-structure-map",
                "parentChild": false,
                "relationship": "related"
              }
            ]
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-metadata-feature-grid-controls",
      "kind": "node.metadata.updated",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled metadata for \"Grid controls\".",
      "nodeIds": [
        "feature-grid-controls"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              },
              {
                "id": "module-workspace",
                "label": "Workspace",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Primary canvas and layout for image segmentation work.",
                  "implementation": "Hosts image view, side panels, and editing tools."
                },
                "context": "Top-level product area for day-to-day editing flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-image-import",
                "label": "Image import",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Bring UI screenshots into the tool for editing.",
                  "implementation": "Upload, preview, and initialize project image state."
                },
                "context": "Entry point for segmentation sessions.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-grid-editor",
                "label": "Grid editor",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Create and adjust segmentation grids over UI images.",
                  "implementation": "Overlay grid, drag handles, snapping, and edit interactions."
                },
                "context": "Primary UX surface for manual segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-cell-selection",
                "label": "Cell selection",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Select, inspect, and multi-select grid cells.",
                  "implementation": "Click, marquee, keyboard modifiers, and focus state."
                },
                "context": "Core interaction inside the grid editor.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-grid-controls",
                "label": "Grid controls",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Change row and column layout quickly.",
                  "implementation": "Add, remove, resize, and snap grid lines."
                },
                "context": "Editor controls for shaping segmentation boundaries.",
                "rawContext": "",
                "images": []
              }
            ],
            "links": [
              {
                "source": "page-main-idea",
                "target": "module-workspace",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-image-import",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-grid-editor",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-cell-selection",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-grid-controls",
                "parentChild": true
              },
              {
                "source": "module-image-import",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-workspace",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              }
            ]
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-type-feature-grid-controls",
      "kind": "node.type.updated",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled \"Grid controls\" type as feature.",
      "nodeIds": [
        "feature-grid-controls"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              },
              {
                "id": "module-workspace",
                "label": "Workspace",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Primary canvas and layout for image segmentation work.",
                  "implementation": "Hosts image view, side panels, and editing tools."
                },
                "context": "Top-level product area for day-to-day editing flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-image-import",
                "label": "Image import",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Bring UI screenshots into the tool for editing.",
                  "implementation": "Upload, preview, and initialize project image state."
                },
                "context": "Entry point for segmentation sessions.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-grid-editor",
                "label": "Grid editor",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Create and adjust segmentation grids over UI images.",
                  "implementation": "Overlay grid, drag handles, snapping, and edit interactions."
                },
                "context": "Primary UX surface for manual segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-cell-selection",
                "label": "Cell selection",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Select, inspect, and multi-select grid cells.",
                  "implementation": "Click, marquee, keyboard modifiers, and focus state."
                },
                "context": "Core interaction inside the grid editor.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-grid-controls",
                "label": "Grid controls",
                "type": "feature",
                "status": "planned",
                "metadata": {},
                "context": "Editor controls for shaping segmentation boundaries.",
                "rawContext": "",
                "images": []
              }
            ],
            "links": [
              {
                "source": "page-main-idea",
                "target": "module-workspace",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-image-import",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-grid-editor",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-cell-selection",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-grid-controls",
                "parentChild": true
              },
              {
                "source": "module-image-import",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-workspace",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              }
            ]
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-created-feature-grid-controls",
      "kind": "node.created",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled node \"Grid controls\".",
      "nodeIds": [
        "feature-grid-controls"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              },
              {
                "id": "module-workspace",
                "label": "Workspace",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Primary canvas and layout for image segmentation work.",
                  "implementation": "Hosts image view, side panels, and editing tools."
                },
                "context": "Top-level product area for day-to-day editing flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-image-import",
                "label": "Image import",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Bring UI screenshots into the tool for editing.",
                  "implementation": "Upload, preview, and initialize project image state."
                },
                "context": "Entry point for segmentation sessions.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-grid-editor",
                "label": "Grid editor",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Create and adjust segmentation grids over UI images.",
                  "implementation": "Overlay grid, drag handles, snapping, and edit interactions."
                },
                "context": "Primary UX surface for manual segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-cell-selection",
                "label": "Cell selection",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Select, inspect, and multi-select grid cells.",
                  "implementation": "Click, marquee, keyboard modifiers, and focus state."
                },
                "context": "Core interaction inside the grid editor.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-grid-controls",
                "label": "Grid controls",
                "type": "feature",
                "status": "planned",
                "metadata": {},
                "context": "Editor controls for shaping segmentation boundaries.",
                "rawContext": "",
                "images": []
              }
            ],
            "links": [
              {
                "source": "page-main-idea",
                "target": "module-workspace",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-image-import",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-grid-editor",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-cell-selection",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-grid-controls",
                "parentChild": true
              },
              {
                "source": "module-image-import",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-workspace",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              }
            ]
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-metadata-feature-cell-selection",
      "kind": "node.metadata.updated",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled metadata for \"Cell selection\".",
      "nodeIds": [
        "feature-cell-selection"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              },
              {
                "id": "module-workspace",
                "label": "Workspace",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Primary canvas and layout for image segmentation work.",
                  "implementation": "Hosts image view, side panels, and editing tools."
                },
                "context": "Top-level product area for day-to-day editing flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-image-import",
                "label": "Image import",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Bring UI screenshots into the tool for editing.",
                  "implementation": "Upload, preview, and initialize project image state."
                },
                "context": "Entry point for segmentation sessions.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-grid-editor",
                "label": "Grid editor",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Create and adjust segmentation grids over UI images.",
                  "implementation": "Overlay grid, drag handles, snapping, and edit interactions."
                },
                "context": "Primary UX surface for manual segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-cell-selection",
                "label": "Cell selection",
                "type": "feature",
                "status": "planned",
                "metadata": {
                  "purpose": "Select, inspect, and multi-select grid cells.",
                  "implementation": "Click, marquee, keyboard modifiers, and focus state."
                },
                "context": "Core interaction inside the grid editor.",
                "rawContext": "",
                "images": []
              }
            ],
            "links": [
              {
                "source": "page-main-idea",
                "target": "module-workspace",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-image-import",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-grid-editor",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-cell-selection",
                "parentChild": true
              },
              {
                "source": "module-image-import",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-workspace",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              }
            ]
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-type-feature-cell-selection",
      "kind": "node.type.updated",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled \"Cell selection\" type as feature.",
      "nodeIds": [
        "feature-cell-selection"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              },
              {
                "id": "module-workspace",
                "label": "Workspace",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Primary canvas and layout for image segmentation work.",
                  "implementation": "Hosts image view, side panels, and editing tools."
                },
                "context": "Top-level product area for day-to-day editing flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-image-import",
                "label": "Image import",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Bring UI screenshots into the tool for editing.",
                  "implementation": "Upload, preview, and initialize project image state."
                },
                "context": "Entry point for segmentation sessions.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-grid-editor",
                "label": "Grid editor",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Create and adjust segmentation grids over UI images.",
                  "implementation": "Overlay grid, drag handles, snapping, and edit interactions."
                },
                "context": "Primary UX surface for manual segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-cell-selection",
                "label": "Cell selection",
                "type": "feature",
                "status": "planned",
                "metadata": {},
                "context": "Core interaction inside the grid editor.",
                "rawContext": "",
                "images": []
              }
            ],
            "links": [
              {
                "source": "page-main-idea",
                "target": "module-workspace",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-image-import",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-grid-editor",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-cell-selection",
                "parentChild": true
              },
              {
                "source": "module-image-import",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-workspace",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              }
            ]
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-created-feature-cell-selection",
      "kind": "node.created",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled node \"Cell selection\".",
      "nodeIds": [
        "feature-cell-selection"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              },
              {
                "id": "module-workspace",
                "label": "Workspace",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Primary canvas and layout for image segmentation work.",
                  "implementation": "Hosts image view, side panels, and editing tools."
                },
                "context": "Top-level product area for day-to-day editing flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-image-import",
                "label": "Image import",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Bring UI screenshots into the tool for editing.",
                  "implementation": "Upload, preview, and initialize project image state."
                },
                "context": "Entry point for segmentation sessions.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-grid-editor",
                "label": "Grid editor",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Create and adjust segmentation grids over UI images.",
                  "implementation": "Overlay grid, drag handles, snapping, and edit interactions."
                },
                "context": "Primary UX surface for manual segmentation.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "feature-cell-selection",
                "label": "Cell selection",
                "type": "feature",
                "status": "planned",
                "metadata": {},
                "context": "Core interaction inside the grid editor.",
                "rawContext": "",
                "images": []
              }
            ],
            "links": [
              {
                "source": "page-main-idea",
                "target": "module-workspace",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-image-import",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-grid-editor",
                "parentChild": true
              },
              {
                "source": "module-grid-editor",
                "target": "feature-cell-selection",
                "parentChild": true
              },
              {
                "source": "module-image-import",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-workspace",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              }
            ]
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-metadata-module-grid-editor",
      "kind": "node.metadata.updated",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled metadata for \"Grid editor\".",
      "nodeIds": [
        "module-grid-editor"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              },
              {
                "id": "module-workspace",
                "label": "Workspace",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Primary canvas and layout for image segmentation work.",
                  "implementation": "Hosts image view, side panels, and editing tools."
                },
                "context": "Top-level product area for day-to-day editing flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-image-import",
                "label": "Image import",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Bring UI screenshots into the tool for editing.",
                  "implementation": "Upload, preview, and initialize project image state."
                },
                "context": "Entry point for segmentation sessions.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-grid-editor",
                "label": "Grid editor",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Create and adjust segmentation grids over UI images.",
                  "implementation": "Overlay grid, drag handles, snapping, and edit interactions."
                },
                "context": "Primary UX surface for manual segmentation.",
                "rawContext": "",
                "images": []
              }
            ],
            "links": [
              {
                "source": "page-main-idea",
                "target": "module-workspace",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-image-import",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-grid-editor",
                "parentChild": true
              },
              {
                "source": "module-image-import",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-workspace",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              }
            ]
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-type-module-grid-editor",
      "kind": "node.type.updated",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled \"Grid editor\" type as module.",
      "nodeIds": [
        "module-grid-editor"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              },
              {
                "id": "module-workspace",
                "label": "Workspace",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Primary canvas and layout for image segmentation work.",
                  "implementation": "Hosts image view, side panels, and editing tools."
                },
                "context": "Top-level product area for day-to-day editing flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-image-import",
                "label": "Image import",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Bring UI screenshots into the tool for editing.",
                  "implementation": "Upload, preview, and initialize project image state."
                },
                "context": "Entry point for segmentation sessions.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-grid-editor",
                "label": "Grid editor",
                "type": "module",
                "status": "planned",
                "metadata": {},
                "context": "Primary UX surface for manual segmentation.",
                "rawContext": "",
                "images": []
              }
            ],
            "links": [
              {
                "source": "page-main-idea",
                "target": "module-workspace",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-image-import",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-grid-editor",
                "parentChild": true
              },
              {
                "source": "module-image-import",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-workspace",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              }
            ]
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-created-module-grid-editor",
      "kind": "node.created",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled node \"Grid editor\".",
      "nodeIds": [
        "module-grid-editor"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              },
              {
                "id": "module-workspace",
                "label": "Workspace",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Primary canvas and layout for image segmentation work.",
                  "implementation": "Hosts image view, side panels, and editing tools."
                },
                "context": "Top-level product area for day-to-day editing flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-image-import",
                "label": "Image import",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Bring UI screenshots into the tool for editing.",
                  "implementation": "Upload, preview, and initialize project image state."
                },
                "context": "Entry point for segmentation sessions.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-grid-editor",
                "label": "Grid editor",
                "type": "module",
                "status": "planned",
                "metadata": {},
                "context": "Primary UX surface for manual segmentation.",
                "rawContext": "",
                "images": []
              }
            ],
            "links": [
              {
                "source": "page-main-idea",
                "target": "module-workspace",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-image-import",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-grid-editor",
                "parentChild": true
              },
              {
                "source": "module-image-import",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "dependency"
              },
              {
                "source": "module-workspace",
                "target": "module-grid-editor",
                "parentChild": false,
                "relationship": "related"
              }
            ]
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-metadata-module-image-import",
      "kind": "node.metadata.updated",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled metadata for \"Image import\".",
      "nodeIds": [
        "module-image-import"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              },
              {
                "id": "module-workspace",
                "label": "Workspace",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Primary canvas and layout for image segmentation work.",
                  "implementation": "Hosts image view, side panels, and editing tools."
                },
                "context": "Top-level product area for day-to-day editing flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-image-import",
                "label": "Image import",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Bring UI screenshots into the tool for editing.",
                  "implementation": "Upload, preview, and initialize project image state."
                },
                "context": "Entry point for segmentation sessions.",
                "rawContext": "",
                "images": []
              }
            ],
            "links": [
              {
                "source": "page-main-idea",
                "target": "module-workspace",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-image-import",
                "parentChild": true
              }
            ]
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-type-module-image-import",
      "kind": "node.type.updated",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled \"Image import\" type as module.",
      "nodeIds": [
        "module-image-import"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              },
              {
                "id": "module-workspace",
                "label": "Workspace",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Primary canvas and layout for image segmentation work.",
                  "implementation": "Hosts image view, side panels, and editing tools."
                },
                "context": "Top-level product area for day-to-day editing flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-image-import",
                "label": "Image import",
                "type": "module",
                "status": "planned",
                "metadata": {},
                "context": "Entry point for segmentation sessions.",
                "rawContext": "",
                "images": []
              }
            ],
            "links": [
              {
                "source": "page-main-idea",
                "target": "module-workspace",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-image-import",
                "parentChild": true
              }
            ]
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-created-module-image-import",
      "kind": "node.created",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled node \"Image import\".",
      "nodeIds": [
        "module-image-import"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              },
              {
                "id": "module-workspace",
                "label": "Workspace",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Primary canvas and layout for image segmentation work.",
                  "implementation": "Hosts image view, side panels, and editing tools."
                },
                "context": "Top-level product area for day-to-day editing flow.",
                "rawContext": "",
                "images": []
              },
              {
                "id": "module-image-import",
                "label": "Image import",
                "type": "module",
                "status": "planned",
                "metadata": {},
                "context": "Entry point for segmentation sessions.",
                "rawContext": "",
                "images": []
              }
            ],
            "links": [
              {
                "source": "page-main-idea",
                "target": "module-workspace",
                "parentChild": true
              },
              {
                "source": "page-main-idea",
                "target": "module-image-import",
                "parentChild": true
              }
            ]
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-metadata-module-workspace",
      "kind": "node.metadata.updated",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled metadata for \"Workspace\".",
      "nodeIds": [
        "module-workspace"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              },
              {
                "id": "module-workspace",
                "label": "Workspace",
                "type": "module",
                "status": "planned",
                "metadata": {
                  "purpose": "Primary canvas and layout for image segmentation work.",
                  "implementation": "Hosts image view, side panels, and editing tools."
                },
                "context": "Top-level product area for day-to-day editing flow.",
                "rawContext": "",
                "images": []
              }
            ],
            "links": [
              {
                "source": "page-main-idea",
                "target": "module-workspace",
                "parentChild": true
              }
            ]
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-type-module-workspace",
      "kind": "node.type.updated",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled \"Workspace\" type as module.",
      "nodeIds": [
        "module-workspace"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              },
              {
                "id": "module-workspace",
                "label": "Workspace",
                "type": "module",
                "status": "planned",
                "metadata": {},
                "context": "Top-level product area for day-to-day editing flow.",
                "rawContext": "",
                "images": []
              }
            ],
            "links": [
              {
                "source": "page-main-idea",
                "target": "module-workspace",
                "parentChild": true
              }
            ]
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-created-module-workspace",
      "kind": "node.created",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled node \"Workspace\".",
      "nodeIds": [
        "module-workspace"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              },
              {
                "id": "module-workspace",
                "label": "Workspace",
                "type": "module",
                "status": "planned",
                "metadata": {},
                "context": "Top-level product area for day-to-day editing flow.",
                "rawContext": "",
                "images": []
              }
            ],
            "links": [
              {
                "source": "page-main-idea",
                "target": "module-workspace",
                "parentChild": true
              }
            ]
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-type-page-main-idea",
      "kind": "node.type.updated",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled \"Main idea\" type as page.",
      "nodeIds": [
        "page-main-idea"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              }
            ],
            "links": []
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-node-created-page-main-idea",
      "kind": "node.created",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled node \"Main idea\".",
      "nodeIds": [
        "page-main-idea"
      ],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [
              {
                "id": "page-main-idea",
                "label": "Main idea",
                "type": "page",
                "status": "planned",
                "metadata": {},
                "context": "# Model Introspection Mock Review Tool\n\nDate: 2026-04-28\n\n## Scope\n\nThis document describes the current surface area of the one-off mock review UI implemented in:\n\n- [/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts)\n- [/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js)\n- [/tmp/20260422-mock-review-ui.css](/tmp/20260422-mock-review-ui.css)\n- [/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py)\n\nIt is not a production feature today. It is a local review tool for decomposing screenshots into nested regions, attaching review notes to those regions, and overriding the machine-detected region tree with user-authored structure.\n\nThe most important idea to preserve in a rebuilt tool is this:\n\n> A screenshot can be treated as a hierarchical assembly. Reviewers can progressively carve the image into nested regions, inspect each region in isolation, comment on it, reject auto-detected structure, and replace it with a better manual decomposition.\n\nThat is the real product primitive. The current implementation calls everything a generic node, but the behavior already supports a pieces-and-assemblies workflow.\n\n## Bottom Line\n\nThe mock review tool already proves five useful product concepts:\n\n1. A screenshot can be represented as a persistent region tree rooted at the full page.\n2. Auto-detected regions and user-authored regions can coexist in one merged review tree.\n3. A reviewer can zoom into any region and treat it as a temporary local canvas for more detailed decomposition.\n4. Review comments can attach to any node in that tree rather than only to the page as a whole.\n5. Reviewers can rework a machine-generated decomposition without mutating the original extraction output by layering manual additions and deletion overrides in a separate ledger.\n\nThat combination is what makes the prototype more interesting than a simple image annotation tool.\n\n## Mental Model\n\nThe current tool is easiest to understand as five nested concepts.\n\n### 1. Image\n\nAn image is a review document, usually a screenshot or mock.\n\nEach image has:\n\n- a stable `imageId`\n- a source file path\n- pixel dimensions\n- a full-page root node\n- a set of detected child nodes\n\n### 2. Root Region\n\nEvery image has a synthetic `root` node that represents the full screenshot. This means every review begins from a full-page assembly even if no child regions were detected.\n\n### 3. Region Tree\n\nEach image contains a hierarchy of rectangular regions. In the prototype these are called nodes. Conceptually they are better understood as assemblies and pieces:\n\n- high-level assemblies are large parent regions that contain other regions\n- pieces are smaller child regions, often leaves\n- any region can become the local working viewport for deeper review\n\nThe tree structure matters more than the raw rectangles. The tool is built around parent-child containment, not around a flat list of bounding boxes.\n\n### 4. Review Ledger\n\nThe extraction output and the reviewer edits are deliberately separated:\n\n- `manifest.json` stores the extracted image and detected node tree\n- `annotations.json` stores comments, user-created nodes, and deleted detected nodes\n\nThis separation is one of the strongest design choices in the prototype because it lets review edits remain reversible and inspectable.\n\n### 5. Viewport\n\nThe user can promote any node into the current viewport. Once that happens, the tool temporarily treats that node as the full canvas for inspection and further subdivision.\n\nThis is not just zoom. It is semantic zoom into a subassembly.\n\n## Current Architecture\n\nThe tool has a simple four-stage pipeline.\n\n### 1. Extraction\n\n[/tmp/20260422-extract-mock-tree.py](/tmp/20260422-extract-mock-tree.py) scans PNGs, detects likely rectangular regions using OpenCV-based mask building and segmentation, assigns tree structure by containment, and writes a manifest.\n\nInputs:\n\n- a source directory of PNG screenshots, defaulting to `/tmp/mocks`\n\nOutputs:\n\n- `/tmp/mock-review/manifest.json`\n- `/tmp/mock-review/overlay/*.png`\n- optional `/tmp/mock-review/mask/*.png`\n\nKey behavior:\n\n- detection is recursive up to a bounded depth\n- children are assigned by geometric containment\n- all extracted nodes are labeled generically as `Node 01`, `Node 02`, and so on\n- a `root` node is synthesized for the full page\n\n### 2. Review Server\n\n[/tmp/20260422-mock-review-ui.ts](/tmp/20260422-mock-review-ui.ts) is a Bun server that:\n\n- serves the HTML shell\n- serves the client script and stylesheet\n- exposes the dataset API\n- persists comments and manual nodes\n- serves image files from repo-relative paths\n\n### 3. Browser Client\n\n[/tmp/20260422-mock-review-ui.client.js](/tmp/20260422-mock-review-ui.client.js) handles:\n\n- client-side merged tree assembly\n- selection state\n- viewport state\n- draw mode for manual regions\n- autosaving comments\n- focus modal behavior\n- deletion of nodes and subtrees\n\n### 4. Persistent Review Data\n\nThe live review state is reconstructed by combining:\n\n- extracted nodes from the manifest\n- user-authored nodes from the annotations file\n- deleted detected node ids from the annotations file\n\nThe effective visible tree is therefore:\n\n`visible tree = extracted nodes + manual nodes - deleted nodes`\n\nThat merged-tree behavior is a core product feature, not an implementation detail.\n\n## User-Facing Surface Area\n\n## 1. Image Rail\n\nThe top rail presents each screenshot as a thumbnail card.\n\nCurrent behavior:\n\n- horizontal thumbnail list\n- one selected image at a time\n- switching images resets selection and viewport back to that image's root\n- each image acts as an independent review tree\n\nWhat this enables:\n\n- batch review of many screenshots\n- per-image decomposition instead of one giant canvas\n\nCurrent limitations:\n\n- no search or filter UI\n- no grouping, tagging, status, or reviewer ownership on images\n- image cards show only the screenshot, not review progress beyond aggregate counts elsewhere\n\n## 2. Node Tree Sidebar\n\nThe left sidebar renders the current image's region hierarchy as a nested tree.\n\nCurrent behavior:\n\n- shows root plus all visible descendants\n- visually distinguishes root, extracted, and user-created nodes\n- indicates whether a node has a saved comment\n- supports click to select a node\n- supports double click to make a node the current viewport\n\nImportant semantics:\n\n- this is a real parent-child region tree, not a flat region list\n- hidden or deleted extracted nodes do not appear\n- user-authored nodes are merged into the same tree and treated like first-class children\n\nWhy this matters:\n\n- the tree is the assembly browser\n- it gives reviewers a durable structure for reasoning about nested layout, not just raw coordinates\n\n## 3. Viewer Canvas\n\nThe center viewer is the primary review workspace.\n\nCurrent behavior:\n\n- displays the current viewport as a cropped canvas\n- draws clickable overlay rectangles for descendant nodes visible within that viewport\n- highlights the selected node\n- supports click to select a node\n- supports double click to promote a node into the viewport\n- supports a button to reset the viewport back to the full page\n\nImportant semantics:\n\n- overlays are drawn relative to the current viewport, not the original page origin\n- the viewer only shows nodes that are descendants of the current viewport\n- the current viewport acts like a temporary local full page\n\nThis is the clearest expression of the pieces-and-assemblies idea in the prototype.\n\nThe reviewer is not just zooming in visually. They are temporarily saying, \"treat this assembly as the thing I am decomposing right now.\"\n\n## 4. Browse Mode vs Draw Mode\n\nThe tool has two editor modes.\n\n### Browse Mode\n\nBrowse mode is for navigating the current tree.\n\nBehavior:\n\n- click overlays to select nodes\n- double click overlays or tree items to use them as the viewport\n- inspect a selected node without changing structure\n\n### Draw Mode\n\nDraw mode is for creating new nested regions.\n\nBehavior:\n\n- drag a rectangle inside the current viewport\n- if the rectangle is large enough, persist it as a new user-authored node\n- newly created nodes are attached to the current viewport as their parent\n- the new node immediately appears in the tree and becomes selected\n\nImportant semantics:\n\n- new manual regions are scoped to the viewport, not arbitrarily placed in the global tree\n- this makes viewport selection the primary act of choosing the assembly you want to refine\n- the reviewer can iteratively zoom into a parent assembly, then sketch its child pieces\n\nThis is the feature most worth carrying forward into a proper tool.\n\nIt turns screenshot review into structural authoring.\n\n## 5. Selection Inspector\n\nThe right inspector is the detail panel for the currently selected node.\n\nCurrent behavior:\n\n- shows node label, id, source, and image context\n- shows a cropped preview of the selected region\n- shows stats such as depth, child count, area share, pixel size, and source type\n- shows the comment editor for the selected node\n- exposes node deletion for any non-root node\n\nWhat it does well:\n\n- makes every node inspectable as its own visual artifact\n- reinforces that review happens at the region level, not only at the full-image level\n\nCurrent limitations:\n\n- node labels are generic and not editable\n- there is no node kind beyond generic `node`\n- there is no status model like approved, needs rework, draft, or obsolete\n\n## 6. Focus View\n\nThe focus modal is a second inspection mode for the current node.\n\nCurrent behavior:\n\n- opens a larger crop of the selected node\n- keeps the same comment record as the main inspector\n- supports double click to promote the selection into the viewport\n\nWhy it matters:\n\n- it separates detailed review of a region from the crowded full-canvas view\n- it shows that a future tool could support side-by-side inspection and editing without changing the underlying record model\n\n## 7. Commenting\n\nComments are attached to a specific `(imageId, nodeId)` pair.\n\nCurrent behavior:\n\n- a reviewer can type a freeform comment for any selected node\n- comments autosave with a short debounce\n- blur forces an immediate save\n- clearing the text deletes the annotation record for that node\n- the tree shows a marker for nodes with comments\n- both the main inspector and focus modal write to the same underlying comment record\n\nImportant semantics:\n\n- comments are region-scoped, not image-scoped\n- review notes survive viewport changes because the record is bound to the node id, not to the current zoom state\n\nThis is already a strong basis for structured review.\n\n## 8. Deletion and Override\n\nDeletion is not just removal. It is a review override.\n\nCurrent behavior:\n\n- any non-root node can be deleted\n- deleting a node removes its entire visible subtree\n- deleting a node also removes any comments attached to nodes in that subtree\n- if the deleted nodes came from extraction, their ids are recorded in `deletedNodeIdsByImage`\n- extracted nodes are therefore suppressed without editing the original manifest\n\nWhy this matters:\n\n- the reviewer can reject a machine-generated decomposition\n- the reviewer can then redraw a better manual decomposition on top of the same image\n- this gives the system a clean distinction between \"what the extractor found\" and \"what the reviewer wants to keep\"\n\nThis is the other major feature worth preserving in a rebuild.\n\n## Review Workflows The Tool Already Supports\n\n## 1. Inspect The Extracted Tree\n\nBasic workflow:\n\n1. Open an image.\n2. Review the extracted node tree.\n3. Click nodes in the tree or canvas to inspect them.\n4. Attach comments to nodes that need attention.\n\nThis is the simplest use case: evaluate the machine-generated decomposition without editing structure.\n\n## 2. Drill Into An Assembly\n\nFocused workflow:\n\n1. Double click a node in the tree, overlay, or crop preview.\n2. Treat that node as the current viewport.\n3. Inspect only its descendants.\n4. Repeat for deeper levels.\n\nThis is the core navigation model. It allows nested review without overwhelming the user with the full-page overlay at all times.\n\n## 3. Rework The Decomposition\n\nManual-override workflow:\n\n1. Use the extracted tree as a first draft.\n2. Delete bad extracted nodes or bad subtrees.\n3. Move into draw mode.\n4. Sketch better child regions within the current viewport.\n5. Save comments on the new manual nodes.\n\nThis is where the tool stops being a viewer and becomes a re-authoring surface.\n\n## 4. Treat Regions As Reusable Parts\n\nConceptual workflow:\n\n1. Start with the full page as the root assembly.\n2. Break it into large assemblies.\n3. Open one assembly as the local viewport.\n4. Break that assembly into smaller pieces.\n5. Continue until the screenshot is decomposed at the level the reviewer cares about.\n\nThe current UI never names these as parts, pieces, or assemblies, but that is already how the interaction behaves.\n\n## Data Model Surface Area\n\n## 1. Manifest\n\nThe manifest is the extractor-owned dataset.\n\nPrimary fields:\n\n- `version`\n- `createdAt`\n- `sourceDir`\n- `outputDir`\n- `imageCount`\n- `images[]`\n\nEach image includes:\n\n- `id`\n- `name`\n- `sourcePath`\n- `overlayPath`\n- `maskPath`\n- `size.width`\n- `size.height`\n- `rootNodeId`\n- `nodeCount`\n- `nodes[]`\n\nEach extracted node includes:\n\n- `id`\n- `kind`\n- `label`\n- `source`\n- `depth`\n- `parentId`\n- `childIds`\n- `areaRatio`\n- `bounds`\n\nImportant note:\n\n- the manifest already stores an explicit tree, not just a set of rectangles\n- overlay and mask assets are debug artifacts; the review client renders from the original screenshot and its own overlay layer\n\n## 2. Manual Node Records\n\nManual nodes live in the annotations ledger under `userNodesByImage`.\n\nPrimary fields:\n\n- `id`\n- `imageId`\n- `parentId`\n- `kind`\n- `label`\n- `bounds`\n- `source`\n- `createdAt`\n- `updatedAt`\n\nImportant semantics:\n\n- manual nodes are first-class tree members after merge\n- they preserve parent-child structure\n- they currently inherit generic numbering instead of meaningful naming\n\n## 3. Annotation Entries\n\nComments live in `entries`, keyed by `imageId:nodeId`.\n\nPrimary fields:\n\n- `imageId`\n- `nodeId`\n- `label`\n- `kind`\n- `parentId`\n- `bounds`\n- `sourcePath`\n- `source`\n- `comment`\n- `updatedAt`\n\nImportant semantics:\n\n- the comment record snapshots node metadata at save time\n- comments are only kept for visible nodes in the merged tree\n- comments on deleted nodes are removed when that subtree is deleted\n\n## 4. Deleted Node Ids\n\n`deletedNodeIdsByImage` records extracted nodes the reviewer has decided to hide.\n\nThis is effectively a rejection layer for auto-generated structure.\n\nThat pattern is worth formalizing in a real tool because it enables review without mutating the extractor output.\n\n## API Surface\n\nThe Bun server exposes a very small API.\n\n### `GET /api/dataset`\n\nReturns the full manifest, the normalized annotations file, and the path to the annotations ledger.\n\nPurpose:\n\n- bootstraps the entire client state\n- implicitly migrates and normalizes old annotation shapes on load\n\n### `POST /api/annotations`\n\nUpserts or clears a comment for one node.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n- `comment`\n\nBehavior:\n\n- empty comment removes the entry\n- non-empty comment writes or updates the entry\n\n### `POST /api/nodes`\n\nCreates a new manual node.\n\nRequest fields:\n\n- `imageId`\n- `parentId`\n- `bounds`\n\nBehavior:\n\n- bounds are normalized to image size\n- bounds must remain inside the current parent viewport\n- very small boxes are rejected\n- success appends a manual node to the annotations ledger\n\n### `DELETE /api/nodes`\n\nDeletes a node subtree.\n\nRequest fields:\n\n- `imageId`\n- `nodeId`\n\nBehavior:\n\n- deletes manual descendants outright\n- records extracted descendants as deleted overrides\n- deletes comments attached to the deleted subtree\n\n### `GET /files/...`\n\nServes repo-relative files needed by the UI, primarily screenshots.\n\n## Behavioral Rules Worth Preserving\n\nThese rules are easy to miss, but they define the current product semantics.\n\n## 1. The Root Node Is Sacred\n\nEvery image always has a full-page root. It cannot be deleted.\n\nThat gives the review model a stable anchor.\n\n## 2. The Viewport Chooses The Editing Context\n\nNew nodes are created under the current viewport, not merely under the current selection.\n\nThat means choosing a viewport is equivalent to choosing the assembly you are refining.\n\n## 3. Comments Belong To Nodes, Not To Zoom State\n\nThe same node can be viewed from the tree, the main viewer, the inspector crop, or the focus modal. The review note remains one shared record.\n\n## 4. Deletion Is A Structural Review Decision\n\nDeleting an extracted node is a decision to suppress that structural interpretation. It is not just a UI hide action.\n\n## 5. The Effective Tree Is Always Recomputed\n\nThe client reconstructs the visible tree from extracted nodes, manual nodes, and deleted ids every time it loads or receives fresh data.\n\nThat means the persisted truth is layered, not flattened.\n\n## What The Prototype Does Especially Well\n\n## 1. It Treats Screenshot Review As Hierarchical\n\nMost annotation tools are flat. This one is fundamentally tree-shaped.\n\nThat is the biggest differentiator.\n\n## 2. It Allows Human Correction Without Throwing Away Machine Work\n\nThe extracted tree serves as a draft. The reviewer can refine it rather than starting from zero.\n\n## 3. It Makes Local Refinement Natural\n\nUsing any region as the current viewport creates a simple, intuitive nested authoring loop.\n\n## 4. It Keeps The Persistence Model Small But Powerful\n\nOne manifest plus one annotations ledger is enough to support inspection, commenting, addition, and structural rejection.\n\n## 5. It Already Suggests A Better Vocabulary\n\nEven though the code says `node`, the interaction model clearly wants richer domain language such as:\n\n- page\n- assembly\n- piece\n- extracted region\n- manual region\n- review note\n- rejected extraction\n\n## Gaps And Limitations In The One-Off Implementation\n\nThese are not flaws in the concept. They are places where a product rebuild should go further.\n\n## 1. Node Meaning Is Too Generic\n\nEverything is labeled as a generic node with auto-numbered labels.\n\nMissing product features:\n\n- rename regions\n- assign semantic types\n- describe why a region matters\n- distinguish container assemblies from leaf pieces intentionally\n\n## 2. Review State Is Only Freeform Text\n\nThe only structured feedback is the existence of a comment.\n\nMissing product features:\n\n- status fields such as approved, needs work, split further, merge into parent, ignore\n- assignee or reviewer identity\n- timestamps beyond last save\n- review history\n\n## 3. Structural Editing Is Minimal\n\nThe tool supports create and delete, but not richer editing.\n\nMissing product features:\n\n- resize existing regions\n- drag to reposition\n- reparent a region\n- reorder siblings when order matters\n- split and merge operations\n\n## 4. Single-User Local Persistence Only\n\nThe tool writes to local JSON files.\n\nMissing product features:\n\n- multi-user collaboration\n- concurrency control\n- server-backed persistence\n- audit trail\n- project-level permissions\n\n## 5. No Review Queue Model\n\nImages are simply listed as thumbnails.\n\nMissing product features:\n\n- review status per image\n- filtering and search\n- progress by screenshot, subtree, or reviewer\n- assignment and batching\n\n## 6. Extraction And Review Are Loosely Coupled\n\nThe extraction result is loaded, but the UI does not expose extraction confidence, extraction version, or alternate interpretations.\n\nMissing product features:\n\n- extractor confidence or provenance per region\n- alternate candidate decompositions\n- side-by-side original vs extracted overlay comparison\n- re-run extraction on one image or one subtree\n\n## 7. Debug Artifacts Are Not Yet Product Concepts\n\nThe extractor produces overlays and optional masks, but those assets are mostly for debugging.\n\nA rebuilt tool could surface them intentionally for:\n\n- \"show me what the machine saw\"\n- \"compare reviewer structure vs extractor structure\"\n- \"debug bad detections\"\n\n## Productizable Primitives\n\nIf this were rebuilt as a proper tool, the current prototype suggests the following first-class primitives.\n\n## 1. Review Document\n\nA screenshot or image under review.\n\n## 2. Region\n\nA rectangular visual scope with stable identity and optional parent.\n\n## 3. Region Tree\n\nA hierarchical composition of regions, starting at the full page.\n\n## 4. Source Layer\n\nThe origin of a region, such as extracted, reviewer-created, imported, or inferred.\n\n## 5. Structural Override\n\nAn explicit reviewer decision to keep, suppress, replace, or refine a machine-generated region.\n\n## 6. Review Note\n\nA comment or structured judgment attached to a region.\n\n## 7. Focus Context\n\nA local viewport that turns one region into the active editing surface.\n\nThat set of primitives is already latent in the script. A real tool should make them explicit in naming, data model, and UI.\n\n## Recommended Direction For A Rebuild\n\nIf the goal is to turn this into a proper internal or product tool, the strongest path is not \"build a better screenshot annotator.\"\n\nThe stronger path is:\n\n> Build a hierarchical region review system where extracted assemblies can be inspected, rejected, refined, and commented on as reusable parts.\n\nConcretely, that means keeping these behaviors intact:\n\n1. Full-page root per image.\n2. Nested region tree as the primary model.\n3. Ability to turn any region into the active viewport.\n4. Coexistence of extracted and manual regions.\n5. Deletion as a suppression override, not a destructive edit to extractor output.\n6. Comments attached to regions, not just to images.\n\nThen add the missing product layers:\n\n1. semantic labels and types for regions\n2. structured review states\n3. richer structural editing\n4. collaboration and auditability\n5. queueing, filtering, and progress tracking\n6. explicit extraction provenance and confidence\n\n## Current Artifact Paths\n\nThe current implementation writes or serves these artifacts:\n\n- extractor input screenshots under `/tmp/mocks`\n- extracted review data under `/tmp/mock-review/manifest.json`\n- reviewer edits under `/tmp/mock-review/annotations.json`\n- overlay debug images under `/tmp/mock-review/overlay`\n- optional detection masks under `/tmp/mock-review/mask`\n\n## Current Commands\n\nCurrent setup is roughly:\n\n1. Run the extractor:\n   `uv run --with opencv-python-headless --with pillow python tmp/20260422-extract-mock-tree.py`\n2. Start the review UI:\n   `bun tmp/20260422-mock-review-ui.ts`\n\nThe exact hosting mechanics are not the important part. The important part is the two-phase contract:\n\n- extraction creates a structural draft\n- review refines that draft through layered overrides\n\n## Summary\n\nThe mock review UI is already more than a one-off screenshot viewer.\n\nIts real innovation is that it treats a screenshot as a nested assembly that can be explored and reworked piece by piece. The extracted boxes are only the starting point. The more important system is the layered review model that lets people:\n\n- inspect a region hierarchy\n- zoom into subassemblies\n- redraw better child regions\n- suppress bad automatic structure\n- attach feedback at any level of the tree\n\nIf rebuilt properly, this could become a strong general-purpose tool for visual decomposition, critique, and iterative refinement of interface screenshots or other structured imagery.",
                "rawContext": "",
                "images": [
                  {
                    "id": "426c8b48-bf93-465d-ba8d-2af2f6e41615",
                    "src": "/api/projects/tool-to-segment-ui-images/images/page-main-idea/426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "filename": "426c8b48-bf93-465d-ba8d-2af2f6e41615.png",
                    "originalName": "image.png",
                    "mimeType": "image/png",
                    "size": 209602,
                    "createdAt": "2026-04-30T14:27:07.069Z"
                  }
                ]
              }
            ],
            "links": []
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-schema-updated",
      "kind": "project.schema.updated",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled metadata fields and node types.",
      "nodeIds": [],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [],
            "links": []
          }
        }
      }
    },
    {
      "id": "legacy-backfill-tool-to-segment-ui-images-project-created",
      "kind": "project.created",
      "actor": {
        "kind": "human",
        "label": "Legacy project owner"
      },
      "createdAt": "2026-04-30T14:27:07.069Z",
      "summary": "Backfilled project creation.",
      "nodeIds": [],
      "linkKeys": [],
      "metadata": {
        "sourceOperationId": "legacy-backfill-tool-to-segment-ui-images"
      },
      "payload": {
        "state": {
          "nodeMetadataSchema": [
            {
              "key": "purpose",
              "label": "Purpose",
              "defaultValue": "Loose context, notes, and markdown content that needs a home in the map."
            },
            {
              "key": "implementation",
              "label": "Implementation",
              "defaultValue": "Freeform markdown note stored directly on the node for AI or human editing."
            }
          ],
          "nodeTypeSchema": [
            {
              "name": "page",
              "color": "#92bedd"
            },
            {
              "name": "area",
              "color": "#92ddb1"
            },
            {
              "name": "assembly",
              "color": "#ddc692"
            },
            {
              "name": "module",
              "color": "#dd9c92"
            },
            {
              "name": "component",
              "color": "#ae92dd"
            },
            {
              "name": "note",
              "color": "#92d8dd"
            }
          ],
          "spatialMap": {
            "nodes": [],
            "links": []
          }
        }
      }
    }
  ]
}
