pikl/docs/DEVPLAN.md

# pikl-menu: Development Plan

Implementation order, phase definitions, and the running
list of ideas. The DESIGN.md is the source of truth for
*what* pikl-menu is. This doc is the plan for *how and
when* we build it.

## Principles

- Get a working vertical slice before going wide on features
- pikl-core stays rendering-agnostic. No GUI or TUI deps
  in the library.
- Every feature gets tests in pikl-core before frontend work
- Ship something usable early, iterate from real usage
- Don't optimize until there's a reason to

## Phase 1: Core Loop (TUI) ✓

The minimum thing that works end-to-end.

**Deliverables:**
- pikl-core: Item store (accepts JSON lines on stdin,
  plain text fallback)
- pikl-core: Basic fuzzy filter on label
- pikl-core: Single selection, Enter to confirm, Escape
  to cancel
- pikl-core: Event bus with on-select and on-cancel hooks
  (shell commands)
- pikl-tui: ratatui list renderer
- pikl (CLI): Reads stdin, opens TUI, emits selected item
  as JSON on stdout
- Exit codes: 0 = selected, 1 = cancelled, 2 = error
- CI: Strict clippy, fmt, tests on Linux + macOS

**Done when:** `ls | pikl` works and prints the selected
item.

## Phase 1.5: Action-fd (Headless Mode) ✓

Scriptable, non-interactive mode for integration tests and
automation. Small enough to slot in before phase 2. It's
about 100 lines of new code plus the binary orchestration
for `show-ui`.

**Deliverables:**
- `--action-fd <N>`: read action script from file
  descriptor N
- Action protocol: plain text, one action per line
  (filter, move-up/down, confirm, cancel, etc.)
- `show-ui` / `show-tui` / `show-gui` actions for
  headless-to-interactive handoff
- Upfront validation pass: reject unknown actions,
  malformed args, actions after show-ui
- `--stdin-timeout <seconds>`: default 30s in action-fd
  mode, 0 in interactive mode
- Default viewport height of 50 in headless mode
- Binary integration tests: spawn pikl with piped stdin +
  action-fd, assert stdout

**Done when:** `echo -e "hello\nworld" | pikl --action-fd 3`
with `confirm` on fd 3 prints `"hello"` to stdout.

## Phase 2: Navigation & Filtering ✓

Make it feel like home for a vim user. The filter system
is the real star here: strategy prefixes, pipeline
chaining, incremental caching.

**Implementation order:**

1. Mode system in core (Insert/Normal, Ctrl+N/Ctrl+E
   to switch)
2. Normal mode vim nav (j/k/gg/G/Ctrl+D/U/Ctrl+F/B)
3. `/` in normal mode enters insert mode with filter
   focused
4. Filter strategy engine (prefix parsing: `'`, `!`, `!'`,
   `/pattern/`, `!/pattern/`)
5. `fancy-regex` integration for the regex strategy
6. Filter pipeline (`|` chaining between stages,
   incremental caching)

**Deliverables:**
- Insert mode / normal mode (Ctrl+N to normal, Ctrl+E
  to insert)
- Escape always cancels and exits, in any mode
- Normal mode: j/k, gg, G, Ctrl+D/U, Ctrl+F/B
- `/` in normal mode enters filter (insert) mode
- `--start-mode normal` flag
- Filter strategies via inline prefixes: fuzzy (default),
  exact (`'term`), inverse (`!term`, `!'term`), regex
  (`/pattern/`, `!/pattern/`)
- fancy-regex integration (unlimited capture groups,
  lookaround)
- Filter pipeline with `|`: each stage narrows the
  previous stage's output
- Incremental filter caching: each stage caches item
  indices, only recomputes from the edited stage forward
- Arrow keys to navigate within the filter text and edit
  earlier pipeline segments
- `\|` escapes a literal pipe in the query
- Case rules: fuzzy = smart case, exact =
  case-insensitive, regex = case-sensitive (use `(?i)`
  for insensitive)

**Deferred to later:**
- H/M/L (viewport-relative jumps): nice-to-have, not
  essential
- Filter syntax highlighting in the input field: deferred
  to theming work

**Done when:** You can navigate and filter a large list
with vim muscle memory.
`'log | !temp | /[0-9]+/` works as a pipeline.

## Phase 3: Structured I/O & Hooks ✓

The structured data pipeline and the full hook system.

**Implementation order:**

1. Item model expansion (sublabel, meta, icon, group as
   explicit optional fields on Item, alongside the raw
   Value)
2. Output struct with action context (separate from the
   original item, no mutation)
3. HookHandler trait in pikl-core, HookEvent enum,
   HookResponse enum
4. Exec hooks in CLI: `--on-<event>-exec` flags, subprocess
   per event, stdout discarded
5. Debounce system: none / debounce(ms) / cancel-stale,
   configurable per hook via CLI flags
6. Handler hooks in CLI: `--on-<event>` flags, persistent
   process, stdin/stdout JSON line protocol
7. Handler protocol commands: add_items, replace_items,
   remove_items, set_filter, close
8. `--filter-fields` scoping (which fields the filter
   searches against)
9. `--format` template strings for display
   (`{label} - {sublabel}`)
10. Field filters in query syntax (`meta.res:3840`),
    integrated into the filter pipeline

**Deliverables:**
- Item model: sublabel, meta, icon, group as first-class
  optional fields
- Output: separate struct with action context (action,
  index) wrapping the original item
- Exec hooks (`--on-<event>-exec`): fire-and-forget,
  subprocess per event, item JSON on stdin
- Handler hooks (`--on-<event>`): persistent bidirectional
  process, JSON lines on stdin/stdout
- Handler protocol: add_items, replace_items, remove_items,
  set_filter, close
- Full lifecycle events: on-open, on-close, on-hover,
  on-select, on-cancel, on-filter
- Debounce: three modes (none, debounce, cancel-stale),
  per-hook CLI flags
- Default debounce: on-hover 200ms + cancel-stale,
  on-filter 200ms, others none
- HookHandler trait in pikl-core (core emits events, does
  not know what handlers do)
- `--filter-fields label,sublabel,meta.tags`
- `--format '{label} - {sublabel}'` template rendering
- Field filters: `meta.res:3840` in query text
- tracing for hook warnings (bad JSON, unknown actions,
  process exit)

**Done when:** The wallpaper picker use case works entirely
through hooks and structured I/O. A handler hook can
receive hover events and emit commands to modify menu
state.

## Phase 4: Multi-Select ✓

Selection with undo/redo. No marks or registers: filtering
already handles navigation, and selections surviving filter
changes covers the register use case.

**Deliverables:**
- `--multi` flag to gate all multi-select behaviour
- `Space`/`Tab` toggle-select, `Shift+Tab` toggle-up
- `V` visual line mode (force-select ranges)
- `u`/`Ctrl+R` undo/redo selection changes, `U` clear all
- Selections survive filter changes (tracked by original index)
- `--selection-order` flag for insertion-order output
- Multi-item output: one JSON line per selected item
- Action-fd commands: toggle-select, select-all, deselect-all,
  undo-selection, redo-selection
- Test DSL: `multi` fixture, `selected_items` assertion

**Done when:** `seq 1 20 | pikl --multi`, toggle a few items
with Space, confirm, and all selected items appear on stdout.

## Phase 5: Table Mode & CSV ✓

CSV/TSV input parsing and columnar table rendering.

**Deliverables:**
- `--input-format csv` and `--input-format tsv` for delimited input
- `--columns` flag for table layout with optional aliases
- Auto-generated column config from CSV headers
- Column widths computed in core (GUI gets them for free)
- Table header row with bold/underline styling
- CSV rows become normal Items (JSON objects), so filtering,
  field filters, hooks, multi-select, and output all work unchanged

**Deferred to future:**
- Cell/column selection in table mode
- Column sorting keybinds
- Horizontal scrolling for wide tables

**Done when:** `echo "name,age\nalice,30\nbob,25" | pikl --input-format csv`
renders a navigable table with headers.

## Phase 6: IPC & Session History

Two independent features bundled together. IPC adds live
external control over a Unix socket. Session history gives
lightweight filter recall for repeated workflows.

### IPC (External Control)

Opt-in Unix socket server for live control of a running
pikl instance by external processes.

**Deliverables:**
- `--ipc` flag to enable the socket listener (off by
  default, pikl is ephemeral and shouldn't have side
  effects unless asked)
- Socket path: `/run/user/$UID/pikl-{session}.sock`
  (named session) or `/run/user/$UID/pikl-{pid}.sock`
  (no session name)
- Socket path logged via tracing on startup
- Cleanup on exit (normal, cancel, SIGTERM)
- Protocol: newline-delimited JSON, one message per line
- Write commands: `add_items`, `replace_items`,
  `remove_items`, `set_filter`, `confirm`, `cancel`,
  `close`, plus navigation actions (`move_up`,
  `move_down`, `move_to_top`, `move_to_bottom`,
  `page_up`, `page_down`, `toggle_select`, `select_all`,
  `clear_selections`)
- Read commands: `get_state` (filter, cursor, counts,
  mode), `get_selection` (selected items). Both require
  an `id` field, response echoes it back.
- Event subscription: `subscribe` with event type list,
  `unsubscribe` to stop. Subscribed events pushed as
  `{"event": "...", ...}` lines.
- Multiple simultaneous client connections
- Auth: Unix socket permissions (user-only)
- IPC is just another frontend: deserializes JSON into
  Actions, optionally subscribes to MenuEvent broadcast

**Not in scope:**
- `get_items` (full item list read, potentially large)
- Auto-enable with `--session`
- Auth beyond Unix socket permissions

### Session Filter History

**Deliverables:**
- `--session name` flag
- On any confirm (single, multi, quicklist), append
  current filter text to
  `~/.local/state/pikl/sessions/{name}.history`
- Skip empty filters and consecutive duplicates
- Load history on startup if session file exists
- Ctrl+P / Ctrl+N in insert mode to cycle through
  filter history
- Selecting a history entry replaces the current filter
  text
- Per-session only, no global history

**Done when:** An external script can connect to a running
pikl instance over the socket, push items, read state, and
subscribe to events. A named session remembers filter
history across invocations.

## Phase 7: Streaming Stdin ✓

Auto-detected streaming when stdin is a pipe (not action-fd,
not CSV/TSV). The menu opens immediately with zero items and
streams them in as they arrive.

`--watch` was dropped from this phase. External tools like
`inotifywait -m ~/walls | pikl` compose naturally with
streaming stdin. No built-in file watcher needed.

**Deliverables:**
- Auto-detected streaming stdin (items arrive over time,
  list updates progressively)
- `StreamingDone` action signals end of input
- `streaming` flag on ViewState for frontend indicators
- TUI shows `...` suffix on the count while streaming
- Background reader with batched `AddItems` (up to 100
  items per batch, flushes immediately for slow sources)
- `streaming-done` action-fd command for test scripts
- No new CLI flags: streaming is auto-detected

**Done when:** `seq 1 10000 | pikl` opens immediately and
items stream in. Slow sources show items appearing one at
a time with a `...` indicator.

## Phase 8: GUI Frontend (Wayland + X11)

The graphical overlay.

**Deliverables:**
- pikl-gui crate with iced
- Wayland: layer-shell overlay via iced_layershell
- X11: override-redirect window or EWMH hints
- Auto-detection of Wayland vs X11 vs fallback to TUI
- `--mode gui` / `--mode tui` override
- Theming (TOML-based, a few built-in themes)
- Image/icon rendering in item list
- Preview pane (text + images)

**Done when:** pikl looks and feels like a native Wayland
overlay with keyboard-first interaction.

## Phase 9: Drill-Down & Groups

Hierarchical navigation.

**Deliverables:**
- `on-select` hook returning
  `{"action": "replace", "items": [...]}` for drill-down
- Backspace / `h` in normal mode to go back
- Navigation history stack
- Item groups with headers
- Tab to cycle groups
- `za` to collapse/expand groups

**Done when:** A file browser built on pikl-menu can
navigate directories without spawning new processes.

## Open Design Notes

- **Viewport cursor preservation on filter change.** When
  the filter narrows and the highlighted item is still in
  the result set, keep it highlighted (like fzf/rofi).
  When it's gone, fall back to the top. Needs the filter
  to report whether a specific original index survived the
  query, or the viewport to do a lookup after each filter
  pass.

- **Confirm-with-arguments (Shift+Enter).** Select an item
  and also pass free-text arguments alongside it. Primary
  use case: app launcher where you select `ls` and want to
  pass `-la` to it. The output would include both the
  selected item and the user-supplied arguments.

  **Partially resolved:** modifier key support has landed.
  Structured output and hook events now carry a `modifiers`
  field when shift/ctrl/alt are held during confirmation.
  This covers the signaling side: scripts can detect
  Shift+Enter and branch on it. The free-text argument
  input (where does the user type `-la`?) is still open.

  Remaining open questions:
  - UX flow: does the filter text become the args on
    Shift+Enter? Or does Shift+Enter open a second input
    field for args after selection? The filter-as-args
    approach is simpler but conflates filtering and
    argument input. A two-step flow (select, then type
    args) is cleaner but adds a mode.
  - Output format: separate field in the JSON output
    (`"args": "-la"`)? Second line on stdout? Appended to
    the label? Needs to be unambiguous for scripts.
  - Should regular Enter with a non-empty filter that
    matches exactly one item just confirm that item (current
    behaviour), or should it also treat any "extra" text
    as args? Probably not, too implicit.

  This is a core feature (new keybind, new output field),
  not just a launcher script concern. Fits naturally after
  phase 4 (multi-select) since it's another selection
  mode variant. The launcher script would assemble
  `{selected} {args}` for execution.

## Future Ideas (Unscheduled)

These are things we've talked about or thought of. No
commitment, no order.

- Lua scripting frontend (mlua + LuaJIT): stateful/
  conditional automation, natural follow-on to action-fd
  and IPC. Lua runtime is just another frontend pushing
  Actions and subscribing to MenuEvents. Deliberately
  deferred until after IPC (phase 6) so the event/action
  API is battle-tested before exposing it to a scripting
  language. See "Scripting Ladder" in DESIGN.md.
- WASM plugin system for custom filter strategies
- `pcre2` feature flag for JIT-compiled regex
- Frecency sorting (track selection frequency, boost
  common picks)
- Manifest file format for reusable pikl configurations
- Sixel / kitty graphics protocol support in TUI mode
- Custom action keybinds (Ctrl+1 through Ctrl+9) with
  distinct exit codes
- Accessibility / screen reader support
- Sections as a first-class concept separate from groups
- Network input sources (HTTP, WebSocket)
- Shell completion generation (bash, zsh, fish)
- Man page generation
- Homebrew formula + AUR PKGBUILD
- App launcher use case: global hotkey opens pikl as GUI
  overlay, fuzzy-filters PATH binaries, launches selection
  (optionally into a tmux session). Needs GUI frontend
  (phase 8) and frecency sorting.
  See `docs/use-cases/app-launcher.md`.
  Setup guides: `docs/guides/app-launcher.md`.
- App description indexing: a tool or subcommand that
  builds a local cache of binary descriptions from man
  pages (`whatis`), .desktop file Comment fields, and
  macOS Info.plist data. Solves the "whatis is too slow
  to run per-keystroke" problem for the app launcher.
  Could be a `pikl index` subcommand or a standalone
  helper script.
- Need vars in hooks like `--on-hover 'echo $out'`.