doc: Update plans around hover actions and bidirectional hook communication.

docs/DESIGN.md

@@ -66,58 +66,164 @@ naturally to table/column display mode.
 
 ## Event Hooks
 
-Hooks are shell commands that run in response to lifecycle
-events. They receive the relevant item(s) as JSON on stdin.
+Lifecycle events fire as the user interacts with the menu.
+There are two ways to respond to them: **exec hooks** and
+**handler hooks**.
 
-### Available Hooks
+### Lifecycle Events
 
-| Hook | Fires When | Use Case |
+| Event | Fires When | Use Case |
 |---|---|---|
 | `on-open` | Menu opens | Save current state for revert |
 | `on-close` | Menu closes (any reason) | Cleanup |
-| `on-hover` | Selection cursor moves to a new item | Live wallpaper preview |
-| `on-select` | User confirms selection (Enter) | Apply the choice |
+| `on-hover` | Cursor moves to a new item | Live preview, prefetch |
+| `on-select` | User confirms (Enter) | Apply the choice |
 | `on-cancel` | User cancels (Escape) | Revert preview |
 | `on-filter` | Filter text changes | Dynamic item reloading |
-| `on-mark` | User marks/unmarks an item | Visual feedback, register management |
+| `on-mark` | User marks/unmarks an item | Visual feedback |
 
-### Configuration
+### Exec Hooks (fire-and-forget)
 
-Hooks are set via CLI flags:
+`--on-<event>-exec` spawns a subprocess for each event.
+The item is piped as JSON on stdin. Stdout is discarded.
+One subprocess per event, no state fed back.
 
 ```sh
-pikl --on-hover 'hyprctl hyprpaper wallpaper "DP-4, {path}"' \
-     --on-cancel 'hyprctl hyprpaper wallpaper "DP-4, {original}"' \
-     --on-hover-debounce 100
+pikl --on-hover-exec 'notify-send "$(jq -r .label)"' \
+     --on-select-exec 'apply-wallpaper.sh'
 ```
 
-Or via a manifest file for reusable configurations:
+Good for simple side effects: notifications, applying a
+setting, logging.
+
+### Handler Hooks (bidirectional)
+
+`--on-<event>` launches a **persistent process** that
+receives events as JSON lines on stdin over the menu's
+lifetime. The process can emit commands as JSON lines on
+stdout to modify menu state.
+
+```sh
+pikl --on-hover './wallpaper-handler.sh' \
+     --on-filter './search-provider.sh'
+```
+
+The handler process stays alive. Each time the event fires,
+a new JSON line is written to its stdin. The process reads
+them in a loop:
+
+```sh
+#!/bin/bash
+while IFS= read -r event; do
+  label=$(echo "$event" | jq -r '.label')
+  set-wallpaper "$label"
+  # optionally emit commands back to pikl on stdout
+done
+```
+
+Stdout from the handler is parsed line-by-line as JSON
+commands (see Handler Protocol below). Stderr passes
+through to the terminal for debug output.
+
+If the handler process exits unexpectedly, pikl logs a
+warning via tracing and stops sending events. When the
+menu closes, pikl closes the handler's stdin (breaking
+the read loop naturally) and gives it a moment to exit
+before killing it.
+
+### Handler Protocol
+
+Handler stdout commands, one JSON line per command:
+
+| Action | Payload | Effect |
+|---|---|---|
+| `add_items` | `{"items": [...]}` | Append items to the list |
+| `replace_items` | `{"items": [...]}` | Replace all items, preserve cursor position if possible |
+| `remove_items` | `{"indices": [0, 3]}` | Remove items by index |
+| `set_filter` | `{"text": "query"}` | Change the filter text |
+| `close` | (none) | Close the menu |
+
+Example:
+
+```jsonl
+{"action": "add_items", "items": [{"label": "new result"}]}
+{"action": "set_filter", "text": "updated query"}
+{"action": "close"}
+```
+
+Lines that don't parse as valid JSON or contain an unknown
+action are logged as warnings (via tracing) and skipped.
+Never fatal. A handler bug doesn't crash the menu.
+
+For atomic multi-step mutations, use `replace_items` instead
+of a `remove_items` + `add_items` pair. If a handler is
+cancelled mid-stream (due to debounce), commands already
+applied are not rolled back.
+
+### Configuration via Manifest
+
+Hooks can also be configured in a manifest file:
 
 ```toml
 # ~/.config/pikl/wallpaper.toml
 [hooks]
-on-hover = 'hyprctl hyprpaper wallpaper "DP-4, {label}"'
-on-cancel = 'restore-wallpaper.sh'
-on-hover-debounce = 100
+on-hover = './wallpaper-handler.sh'
+on-select-exec = 'apply-wallpaper.sh'
+on-cancel-exec = 'restore-wallpaper.sh'
 
 [display]
 columns = ["label", "meta.res"]
 ```
 
-### Bidirectional Hooks
-
-Hooks can return JSON on stdout to modify menu state:
-- Update an item's display
-- Add/remove items from the list
-- Set the filter text
-- Close the menu
-
 ### Debouncing
 
-All hooks support a debounce option
-(`--on-hover-debounce 100`). When the user is scrolling
-fast, only the last event in the debounce window fires.
-Built-in, no external tooling needed.
+All hooks (exec and handler) support debouncing. Three
+modes:
+
+| Mode | Behaviour | Default for |
+|---|---|---|
+| None | Fire immediately, every time | on-select, on-cancel, on-open, on-close |
+| Debounce(ms) | Wait for quiet period, fire last event | on-filter (200ms) |
+| Cancel-stale | New event cancels any in-flight invocation | (opt-in) |
+
+Debounce and cancel-stale can combine: wait for quiet,
+then fire, and if the previous invocation is still running,
+cancel it first. This is the default for on-hover (200ms
+debounce + cancel-stale).
+
+CLI flags:
+
+```sh
+# Set debounce duration
+pikl --on-hover './preview.sh' --on-hover-debounce 200
+
+# Disable debounce (fire every event)
+pikl --on-hover './preview.sh' --on-hover-debounce 0
+
+# Enable cancel-stale (for exec hooks, kills subprocess;
+# for handler hooks, a cancelled event is not sent)
+pikl --on-hover-exec 'slow-command' --on-hover-cancel-stale
+```
+
+### Hook Architecture (Core vs CLI)
+
+pikl-core defines the `HookHandler` trait and emits
+lifecycle events. It does not know what handlers do with
+them. The core manages debouncing and cancel-stale logic
+since that interacts with the event loop timing.
+
+```rust
+#[async_trait]
+pub trait HookHandler: Send + Sync {
+    async fn handle(&self, event: HookEvent)
+        -> Result<Vec<HookResponse>, PiklError>;
+}
+```
+
+The CLI binary provides `ShellHookHandler`, which maps
+CLI flags to shell commands. Library consumers implement
+their own handlers with whatever behaviour they want:
+in-process closures, network calls, anything.
 
 ## Filtering
 
@@ -651,18 +757,26 @@ complexity:
    branching, no custom DSL. For shell one-liners,
    integration tests, and simple automation. If you need
    conditionals, you've outgrown this.
-2. **IPC** (phase 6): bidirectional JSON over Unix socket.
+2. **Exec hooks:** fire-and-forget shell commands triggered
+   by lifecycle events. Subprocess per event, stdout
+   discarded. For simple side effects.
+3. **Handler hooks:** persistent bidirectional processes.
+   Receive events as JSON lines on stdin, emit commands on
+   stdout to modify menu state. The shell scripter's
+   extension point: anyone who can write a bash script can
+   extend pikl without touching Rust.
+4. **IPC** (phase 6): bidirectional JSON over Unix socket.
    External tools can read state and send actions while pikl
    runs interactively. Good for tool integration.
-3. **Lua** (post phase 6): embedded LuaJIT via mlua. Full
+5. **Lua** (post phase 6): embedded LuaJIT via mlua. Full
    stateful scripting: subscribe to events, branch on state,
    loops, the works. The Lua runtime is just another
    frontend pushing Actions and subscribing to MenuEvents.
    For anything complex enough to need a real language.
 
-No custom DSL. Action-fd stays simple forever. The jump from
-"I need conditionals" to "use Lua" is intentional: there's
-no value in a half-language.
+No custom DSL. Action-fd stays simple forever. The jump
+from "I need conditionals" to "use Lua" is intentional:
+there's no value in a half-language.
 
 ## Use Cases
 
@@ -688,5 +802,8 @@ with the full writeup.
 - Maximum practical item count before we need virtual
   scrolling? (Probably around 100k)
 - Should hooks run in a pool or strictly sequential?
+  Resolved: exec hooks are one subprocess per event.
+  Handler hooks are persistent processes. Debounce and
+  cancel-stale manage concurrency.
 - Plugin system via WASM for custom filter strategies?
   (Probably way later if ever)

docs/DEVPLAN.md

@@ -114,22 +114,61 @@ with vim muscle memory.
 
 ## Phase 3: Structured I/O & Hooks
 
-The structured data pipeline.
+The structured data pipeline and the full hook system.
+
+**Implementation order:**
+
+1. Item model expansion (sublabel, meta, icon, group as
+   explicit optional fields on Item, alongside the raw
+   Value)
+2. Output struct with action context (separate from the
+   original item, no mutation)
+3. HookHandler trait in pikl-core, HookEvent enum,
+   HookResponse enum
+4. Exec hooks in CLI: `--on-<event>-exec` flags, subprocess
+   per event, stdout discarded
+5. Debounce system: none / debounce(ms) / cancel-stale,
+   configurable per hook via CLI flags
+6. Handler hooks in CLI: `--on-<event>` flags, persistent
+   process, stdin/stdout JSON line protocol
+7. Handler protocol commands: add_items, replace_items,
+   remove_items, set_filter, close
+8. `--filter-fields` scoping (which fields the filter
+   searches against)
+9. `--format` template strings for display
+   (`{label} - {sublabel}`)
+10. Field filters in query syntax (`meta.res:3840`),
+    integrated into the filter pipeline
 
 **Deliverables:**
-- JSON line input parsing (label, sublabel, meta, icon,
-  group)
-- JSON output with action context
-- Full hook lifecycle: on-open, on-close, on-hover,
+- Item model: sublabel, meta, icon, group as first-class
+  optional fields
+- Output: separate struct with action context (action,
+  index) wrapping the original item
+- Exec hooks (`--on-<event>-exec`): fire-and-forget,
+  subprocess per event, item JSON on stdin
+- Handler hooks (`--on-<event>`): persistent bidirectional
+  process, JSON lines on stdin/stdout
+- Handler protocol: add_items, replace_items, remove_items,
+  set_filter, close
+- Full lifecycle events: on-open, on-close, on-hover,
   on-select, on-cancel, on-filter
-- Hook debouncing
-- Bidirectional hooks (hook stdout modifies menu state)
-- `--format` template strings for display
-- Field filters (`meta.res:3840`)
-- Filter scoping (`--filter-fields`)
+- Debounce: three modes (none, debounce, cancel-stale),
+  per-hook CLI flags
+- Default debounce: on-hover 200ms + cancel-stale,
+  on-filter 200ms, others none
+- HookHandler trait in pikl-core (core emits events, does
+  not know what handlers do)
+- `--filter-fields label,sublabel,meta.tags`
+- `--format '{label} - {sublabel}'` template rendering
+- Field filters: `meta.res:3840` in query text
+- tracing for hook warnings (bad JSON, unknown actions,
+  process exit)
 
 **Done when:** The wallpaper picker use case works entirely
-through hooks and structured I/O.
+through hooks and structured I/O. A handler hook can
+receive hover events and emit commands to modify menu
+state.
 
 ## Phase 4: Multi-Select & Registers