doc: Add project design, dev plan, coding standards, and use cases.

2026-03-13 21:47:37 -04:00
parent 24f6635927
commit b4129eaeaf
5 changed files with 1364 additions and 0 deletions
--- a/docs/DESIGN.md
+++ b/docs/DESIGN.md
@@ -0,0 +1,692 @@
+# pikl-menu: Design Document
+
+A keyboard-driven, hookable, streaming menu for Wayland and
+X11. Built as a Rust library with TUI and GUI frontends.
+
+Think rofi/wofi but with structured I/O, lifecycle hooks,
+vim keybindings, streaming input/output, and multi-select.
+The kind of tool you build a wallpaper picker on top of, or
+a clipboard manager, or a process killer, or anything else
+that's fundamentally "pick from a filterable list and do
+something with it."
+
+## Core Concept
+
+pikl-menu is a **filterable, hookable, streaming item list
+with structured I/O**. Everything else is a feature on top
+of that core.
+
+Items go in (stdin, file, directory watch, IPC). The user
+filters, navigates, selects. Events fire (hover, select,
+cancel, filter change). Selected items come out (stdout,
+hooks, IPC). That's it.
+
+## Structured I/O
+
+### Input
+
+Items are JSON lines on stdin. Each item has fields:
+
+```jsonl
+{"label": "beach_sunset.jpg", "sublabel": "/home/maple/walls/nature/", "meta": {"size": "2.4MB", "res": "3840x2160"}, "icon": "/path/to/thumb.png"}
+{"label": "city_night.png", "sublabel": "/home/maple/walls/urban/", "meta": {"size": "1.8MB", "res": "2560x1440"}}
+```
+
+Minimum viable item is just `{"label": "something"}`.
+Everything else is optional.
+
+Plain text mode also works: one line per item, each line
+becomes a label. Keeps it pipeable for simple use cases.
+
+### Output
+
+Selected items are emitted as JSON lines on stdout.
+Multi-select emits multiple lines. The output includes any
+metadata that came in, plus selection context:
+
+```jsonl
+{"label": "beach_sunset.jpg", "sublabel": "/home/maple/walls/nature/", "meta": {"size": "2.4MB"}, "action": "select", "index": 3}
+```
+
+### Streaming
+
+Input can arrive over time. The list populates progressively
+as items stream in. Useful for slow sources (network
+queries, recursive directory walks, long-running commands).
+
+Output can also stream. `on-hover` events can be configured
+to emit to stdout as they happen, not just on final
+selection. Useful for live preview pipelines.
+
+### CSV / TSV
+
+`--input-format csv` or `--input-format tsv` parses columnar
+input. First row is headers, which become field names. Maps
+naturally to table/column display mode.
+
+## Event Hooks
+
+Hooks are shell commands that run in response to lifecycle
+events. They receive the relevant item(s) as JSON on stdin.
+
+### Available Hooks
+
+| Hook | Fires When | Use Case |
+|---|---|---|
+| `on-open` | Menu opens | Save current state for revert |
+| `on-close` | Menu closes (any reason) | Cleanup |
+| `on-hover` | Selection cursor moves to a new item | Live wallpaper preview |
+| `on-select` | User confirms selection (Enter) | Apply the choice |
+| `on-cancel` | User cancels (Escape) | Revert preview |
+| `on-filter` | Filter text changes | Dynamic item reloading |
+| `on-mark` | User marks/unmarks an item | Visual feedback, register management |
+
+### Configuration
+
+Hooks are set via CLI flags:
+
+```sh
+pikl --on-hover 'hyprctl hyprpaper wallpaper "DP-4, {path}"' \
+     --on-cancel 'hyprctl hyprpaper wallpaper "DP-4, {original}"' \
+     --on-hover-debounce 100
+```
+
+Or via a manifest file for reusable configurations:
+
+```toml
+# ~/.config/pikl/wallpaper.toml
+[hooks]
+on-hover = 'hyprctl hyprpaper wallpaper "DP-4, {label}"'
+on-cancel = 'restore-wallpaper.sh'
+on-hover-debounce = 100
+
+[display]
+columns = ["label", "meta.res"]
+```
+
+### Bidirectional Hooks
+
+Hooks can return JSON on stdout to modify menu state:
+- Update an item's display
+- Add/remove items from the list
+- Set the filter text
+- Close the menu
+
+### Debouncing
+
+All hooks support a debounce option
+(`--on-hover-debounce 100`). When the user is scrolling
+fast, only the last event in the debounce window fires.
+Built-in, no external tooling needed.
+
+## Filtering
+
+### Strategies
+
+The filter strategy is determined by the prefix/syntax of
+the query text. No separate mode selector. The filter engine
+parses the query on every keystroke and picks the right
+strategy.
+
+| Prefix | Strategy | Keeps/Removes | Case | Example |
+|---|---|---|---|---|
+| (none) | Fuzzy (nucleo) | keeps matches | smart case | `bch snst` matches "beach_sunset" |
+| `'` | Exact substring | keeps matches | case-insensitive | `'sunset` matches "Sunset_beach" |
+| `!` | Fuzzy inverse | removes matches | smart case | `!urban` excludes items matching "urban" |
+| `!'` | Exact inverse | removes matches | case-insensitive | `!'temp` excludes items containing "temp" |
+| `/pattern/` | Regex | keeps matches | case-sensitive | `/\d{4}x\d{4}/` matches resolution strings |
+| `!/pattern/` | Regex inverse | removes matches | case-sensitive | `!/\.bak$/` excludes backup files |
+
+**Case sensitivity rules:**
+
+- **Fuzzy (nucleo):** Smart case. Case-insensitive until
+  you type an uppercase character.
+- **Exact substring:** Case-insensitive always.
+- **Regex:** Case-sensitive by default. Use `(?i)` flag
+  inside the pattern for case-insensitive matching. This
+  follows standard regex convention.
+
+The `!` prefix is a modifier that inverts any strategy. It
+stacks with `'` and `/pattern/`.
+
+### Chaining (Pipeline)
+
+Filters chain with `|` (pipe). Each stage takes the previous
+stage's output as its input and narrows further:
+
+```
+'log | !temp | /[0-9]+/
+```
+
+Means: exact match "log", then exclude items containing
+"temp", then keep only items with at least one digit. Each
+`|` is a pipeline step, not an OR.
+
+OR semantics live inside regex: `/foo|bar/` matches either.
+Outside regex, `|` always means "then filter again."
+
+**Escaping:** `\|` produces a literal pipe character in
+the query.
+
+**Whitespace:** Spaces around `|` are trimmed. `log|!temp`
+and `log | !temp` are identical.
+
+**Empty segments:** Skipped silently. `log | | foo` is the
+same as `log | foo`.
+
+### Incremental Evaluation
+
+The filter pipeline uses an incremental caching strategy to
+avoid redundant work:
+
+- Each pipeline stage caches its result as a set of item
+  indices (not copies of the items).
+- When the user edits a segment, only that stage and
+  everything downstream is recomputed. Upstream stages keep
+  their cached results.
+- Backspacing over a `|` pops the last stage. Instant, no
+  recomputation.
+- Typing a new `|` pushes a new empty stage.
+- Within a fuzzy stage, nucleo's own incremental narrowing
+  applies when appending characters.
+- Regex and exact stages recompute fully on each keystroke
+  within their segment, but they run against the previous
+  stage's (already filtered) output, so the input set is
+  small.
+
+The cursor position determines which segment is active.
+Arrow keys move through the query text normally. Editing an
+earlier segment invalidates from that point forward and
+cascades recomputation.
+
+### Regex Engine
+
+Built on `fancy-regex` (pure Rust, no C deps). Full
+PCRE2-style features:
+- Unlimited capture groups (no artificial cap)
+- Lookahead / lookbehind
+- Non-greedy quantifiers
+- Unicode support
+- Named groups
+
+Optional `pcre2` feature flag for JIT-compiled patterns
+when performance matters on huge datasets.
+
+### Filter Scoping
+
+By default, filters match against the label.
+`--filter-fields label,sublabel,meta.tags` expands the
+search scope.
+
+Field filters (`meta.res:3840`) are a Phase 3 feature that
+builds on top of this pipeline infrastructure.
+
+### Future: Syntax Highlighting
+
+The filter input is a single text field. A future
+enhancement is to syntax-highlight the query: color-code
+prefixes (`'`, `!`, `/`), pipe delimiters, and regex
+patterns so the user gets visual feedback on how their query
+is being parsed. Low-cost, high-value, but deferred until
+we get into proper theming work.
+
+## Navigation & Interaction
+
+### Modes
+
+pikl-menu has two input modes, inspired by vim:
+
+**Insert mode** (default on open):
+- Typing goes to the filter input
+- Arrow keys navigate the list
+- Ctrl+N switches to normal mode
+- Escape always cancels and exits (no exceptions, in any
+  mode)
+
+**Normal mode**:
+- Full vim keybindings for navigation
+- Ctrl+E switches to insert (edit) mode
+- `/` enters insert mode with filter focused (like vim
+  search)
+- Escape cancels and exits
+
+`--start-mode normal` to open in normal mode instead.
+
+**Escape is sacred.** It always means cancel and exit,
+regardless of mode. It never means "go back to previous
+mode." This is a deliberate departure from vim to keep the
+exit path predictable.
+
+**Command mode** is planned for the future but not yet
+designed.
+
+### Vim Keybindings (Normal Mode)
+
+| Key | Action |
+|---|---|
+| `j` / `k` | Down / up |
+| `g g` | Jump to top |
+| `G` | Jump to bottom |
+| `Ctrl+D` / `Ctrl+U` | Half-page down / up |
+| `Ctrl+F` / `Ctrl+B` | Full page down / up |
+| `/` | Enter filter (insert) mode |
+| `n` / `N` | Next / previous filter match |
+| `m{a-z}` | Set mark at current position |
+| `'{a-z}` | Jump to mark |
+| `"a` | Select into register `a` |
+| `v` | Toggle multi-select on current item |
+| `V` | Visual line select mode |
+| `Enter` | Confirm selection |
+| `Escape` | Cancel and exit |
+| `q` | Cancel and exit |
+| `Space` | Toggle select current item and move down |
+
+`H`/`M`/`L` (viewport-relative jumps) are deferred. They're
+nice-to-have but not essential for the first pass of vim
+navigation.
+
+### Multi-Select
+
+- `Space` or `v` toggles selection on current item
+- `V` enters visual line mode, moving up/down selects ranges
+- All selected items are emitted on confirm
+- `"a` through `"z`: select into named registers (like vim
+  yanking)
+- Registers can be recalled or included in output
+
+### Drill-Down
+
+An `on-select` hook can return
+`{"action": "replace", "items": [...]}` to replace the
+current list in-place. No new process spawn. Combined with
+a back keybind (`Backspace` or `h` in normal mode), this
+gives you hierarchical navigation.
+
+## Display
+
+### List Mode (default)
+
+Vertical list of items. Each entry shows available fields
+based on a format template:
+
+```sh
+pikl --format '{icon} {label} <dim>{sublabel}</dim>'
+```
+
+### Table / Column Mode
+
+```sh
+pikl --columns label,meta.res,meta.size --column-separator ' │ '
+```
+
+Auto-aligns columns. Sortable per-column with keybinds.
+CSV/TSV input maps naturally here.
+
+### Preview Pane
+
+Optional side panel that runs a command on the hovered item
+and displays its stdout:
+
+```sh
+pikl --preview 'cat {meta.path}' --preview-position right --preview-width 50%
+```
+
+For GUI mode, this can render images (thumbnails, etc). For
+TUI mode, text or sixel/kitty graphics protocol.
+
+### Sections / Groups
+
+Items can include a `group` field. pikl-menu renders group
+headers and allows Tab to cycle between groups. Groups are
+collapsible in normal mode with `za` (vim fold toggle).
+
+### Theming
+
+Minimal but configurable. TOML-based:
+
+```toml
+# ~/.config/pikl/theme.toml
+[colors]
+background = "#1a1a2e"
+foreground = "#e0e0e0"
+selection = "#e94560"
+filter = "#0f3460"
+dim = "#666666"
+border = "#333333"
+
+[layout]
+width = "40%"
+position = "right"  # left, right, center
+padding = 1
+```
+
+A few built-in themes ship with pikl. `--theme monokai` or
+`--theme nord` etc.
+
+## Sessions
+
+`--session name` persists the menu's state (filter text,
+scroll position, selected items, marks, registers) across
+invocations. State lives in
+`~/.local/state/pikl/sessions/`.
+
+No `--session` = ephemeral, nothing saved.
+
+Session names are just strings, use shell expansion for
+dynamic names:
+
+```sh
+pikl --session "walls-$(hostname)"
+pikl --session "logs-$USER"
+```
+
+Session history is a log file alongside the state. Other
+tools can tail it for observability.
+
+## Watched Sources
+
+```sh
+pikl --watch ~/Pictures/walls/ --watch-extensions jpg,png,webp
+```
+
+The list updates live as files are added, removed, or
+renamed. Combines naturally with hooks: a new file appears,
+the list updates, the user can immediately select it.
+
+Also supports watching a file (one item per line) or a
+named pipe.
+
+## IPC
+
+While running, pikl-menu listens on a Unix socket
+(`/run/user/$UID/pikl-$PID.sock`). External tools can:
+
+- Push new items
+- Remove items
+- Update item fields
+- Set the filter text
+- Read current selection
+- Close the menu
+
+Protocol is newline-delimited JSON. Simple enough to use
+with `socat` or any language's Unix socket support.
+
+## Exit Codes
+
+| Code | Meaning |
+|---|---|
+| 0 | User selected item(s) |
+| 1 | User cancelled (Escape / q) |
+| 2 | Error (bad input, hook failure, etc) |
+| 10-19 | Custom actions (configurable keybinds, future) |
+
+## Platform Support
+
+### Primary Targets
+
+- **Wayland** (layer-shell overlay via `iced` +
+  `iced_layershell`)
+- **X11** (standard window with override-redirect or EWMH
+  hints)
+- **TUI** (ratatui, runs in any terminal)
+
+### Rendering Selection
+
+pikl auto-detects the environment:
+1. If `$WAYLAND_DISPLAY` is set: GUI (Wayland)
+2. If `$DISPLAY` is set: GUI (X11)
+3. Otherwise: TUI
+
+Override with `--mode gui` or `--mode tui`.
+
+## Architecture
+
+```
+pikl-core (library crate)
+├── Item store       - structured items, streaming input, indexing
+├── Filter engine    - fuzzy, regex, fancy-regex, field filters, chaining
+├── Selection        - single, multi, registers, marks
+├── Hook system      - lifecycle events, debouncing, bidirectional
+├── Session          - persistence, history log
+├── IPC server       - Unix socket, JSON protocol
+└── Event bus        - internal pub/sub connecting all the above
+
+pikl-tui (binary crate)
+├── ratatui renderer
+├── Terminal input handling
+└── Sixel / kitty image protocol (optional)
+
+pikl-gui (binary crate)
+├── iced + layer-shell (Wayland)
+├── iced + X11 windowing
+└── Native image rendering
+
+pikl (binary crate, unified CLI)
+├── CLI argument parsing (clap)
+├── Auto-detection of GUI vs TUI
+├── Stdin/stdout JSON streaming
+└── Manifest file loading
+```
+
+`pikl-core` has zero rendering dependencies. It's the
+engine. The frontends are thin layers that subscribe to the
+event bus and render state.
+
+Third-party Rust tools can depend on `pikl-core` directly
+and skip the CLI entirely. Your wallpaper picker becomes a
+Rust binary that calls `pikl_core::Menu::new()` with items
+and hook closures.
+
+### Frontends
+
+Every way of driving pikl is a frontend. Frontends send
+`Action` variants into the core's `mpsc` channel and
+optionally subscribe to `MenuEvent` via the `broadcast`
+channel. The core is frontend-agnostic: it processes actions
+sequentially regardless of where they came from.
+
+| Frontend | Input Source | Interactive | Phase |
+|---|---|---|---|
+| TUI | Terminal keypresses (crossterm) | Yes | 1 |
+| GUI | GUI events (iced) | Yes | 8 |
+| Action-fd | Pre-validated script from a file descriptor | No (unless `show-ui`) | 1.5 |
+| IPC | Unix socket, JSON protocol | Yes (bidirectional) | 6 |
+| Lua | LuaJIT script via mlua | Yes (stateful, conditional) | Post-6 |
+
+This framing means new interaction modes don't require core
+changes: they're just new frontends that push actions.
+
+## Action-fd (Headless Mode)
+
+`--action-fd <N>` reads a pre-built action script from file
+descriptor N. Items come from stdin, selected output goes to
+stdout. No TUI or GUI is launched unless the script requests
+it.
+
+### Protocol
+
+One action per line, plain text:
+
+```
+filter foo
+move-down
+move-down
+confirm
+```
+
+Available actions:
+
+| Action | Argument | Maps to |
+|---|---|---|
+| `filter <text>` | Filter string | `Action::UpdateFilter` |
+| `move-up` | (none) | `Action::MoveUp` |
+| `move-down` | (none) | `Action::MoveDown` |
+| `move-to-top` | (none) | `Action::MoveToTop` |
+| `move-to-bottom` | (none) | `Action::MoveToBottom` |
+| `page-up` | (none) | `Action::PageUp` |
+| `page-down` | (none) | `Action::PageDown` |
+| `resize <n>` | Height (items) | `Action::Resize` |
+| `confirm` | (none) | `Action::Confirm` |
+| `cancel` | (none) | `Action::Cancel` |
+| `show-ui` | (none) | Hand off to interactive frontend |
+| `show-tui` | (none) | Hand off to TUI specifically |
+| `show-gui` | (none) | Hand off to GUI specifically |
+
+`show-ui` auto-detects the appropriate interactive frontend
+(Wayland: GUI, X11: GUI, otherwise: TUI). `show-tui` and
+`show-gui` are explicit overrides. All three must be the
+last action in the script: anything after them is a
+validation error.
+
+### Execution Model
+
+1. **Read + validate action-fd:** the entire script is
+   loaded and validated before anything runs. Unknown
+   actions, malformed arguments, and actions after
+   `show-ui`/`show-tui`/`show-gui` are rejected with
+   diagnostic errors.
+2. **Read all stdin until EOF:** items are fully loaded. If
+   stdin doesn't close within the timeout (default 30s),
+   pikl exits with an error.
+3. **Create Menu** with items, default viewport height
+   of 50.
+4. **Execute actions** sequentially.
+5. **Exit with result.** If the script ended with
+   `show-ui`/`show-tui`/`show-gui`, launch the interactive
+   frontend and let the user finish.
+
+### Stdin Timeout
+
+`--stdin-timeout <seconds>` controls how long pikl waits
+for stdin to close.
+
+- **Action-fd mode:** defaults to 30 seconds. Catches
+  mistakes like piping from a streaming source (`tail -f`).
+- **Interactive mode:** defaults to 0 (no timeout). The
+  user can Ctrl+C.
+- `--stdin-timeout 0` disables the timeout in any mode.
+
+### Constraints
+
+- **Action-fd is intentionally stateless.** The caller sends
+  actions but cannot read state back or branch on it. For
+  conditional/stateful automation, use Lua (when available)
+  or IPC.
+- **Action-fd and streaming stdin are mutually exclusive.**
+  Action-fd requires stdin to be finite and fully consumed
+  before actions execute. Streaming input (phase 7) is for
+  interactive frontends.
+- **Unknown actions are validation errors**, not silently
+  ignored. The script is pre-validated: there's no
+  forward-compatibility concern since the caller controls
+  both the script and the pikl version.
+
+### Examples
+
+```sh
+# Simple: filter and confirm
+exec 3<<'EOF'
+filter projects
+confirm
+EOF
+ls ~/dev | pikl --action-fd 3
+
+# Pre-navigate, then hand off to user
+exec 3<<'EOF'
+filter .rs
+move-down
+move-down
+show-ui
+EOF
+find . -type f | pikl --action-fd 3
+
+# Integration test (Rust)
+# Spawn pikl with items on stdin, actions on fd 3, assert stdout
+```
+
+## Key Crates
+
+| Crate | Purpose |
+|---|---|
+| `fancy-regex` | Regex engine with lookaround support |
+| `serde` + `serde_json` | Structured I/O |
+| `clap` | CLI argument parsing |
+| `ratatui` + `crossterm` | TUI rendering |
+| `iced` + `iced_layershell` | GUI rendering (Wayland) |
+| `nucleo` | Fuzzy matching with smart case and incremental narrowing |
+| `tokio` | Async runtime (streaming, IPC, hooks) |
+| `notify` | File/directory watching |
+| `csv` | CSV/TSV input parsing |
+| `toml` | Config and manifest parsing |
+| `dirs` | XDG directory paths |
+
+## Example Use Cases
+
+```sh
+# Simple picker: pipe in lines, get selection back
+ls ~/walls/*.jpg | pikl
+
+# Structured items with live preview hooks
+find-walls --json | pikl \
+  --on-hover 'set-wallpaper {label}' \
+  --on-cancel 'restore-wallpaper' \
+  --on-hover-debounce 100
+
+# Process killer with table view
+ps aux --no-headers | pikl --input-format tsv --columns 1,10,2 --multi
+
+# Drill-down file browser
+pikl --manifest ~/.config/pikl/filebrowser.toml --session filebrowser
+
+# Clipboard history with preview
+clipman list --json | pikl --preview 'echo {meta.content}' --on-select 'clipman paste {id}'
+```
+
+## Scripting Ladder
+
+pikl has a deliberate progression for automation
+complexity:
+
+1. **Action-fd:** fire-and-forget scripts. No state, no
+   branching, no custom DSL. For shell one-liners,
+   integration tests, and simple automation. If you need
+   conditionals, you've outgrown this.
+2. **IPC** (phase 6): bidirectional JSON over Unix socket.
+   External tools can read state and send actions while pikl
+   runs interactively. Good for tool integration.
+3. **Lua** (post phase 6): embedded LuaJIT via mlua. Full
+   stateful scripting: subscribe to events, branch on state,
+   loops, the works. The Lua runtime is just another
+   frontend pushing Actions and subscribing to MenuEvents.
+   For anything complex enough to need a real language.
+
+No custom DSL. Action-fd stays simple forever. The jump from
+"I need conditionals" to "use Lua" is intentional: there's
+no value in a half-language.
+
+## Use Cases
+
+Concrete workflows pikl-menu is designed to support.
+These are first-class targets that inform feature
+priorities, not hypotheticals. Each has its own doc
+with the full writeup.
+
+- [Wallpaper Picker](use-cases/wallpaper-picker.md):
+  keyboard-driven wallpaper browser with live preview
+  via hyprpaper hooks.
+- [App Launcher](use-cases/app-launcher.md): global
+  hotkey launcher that fuzzy-filters PATH binaries
+  or .desktop entries.
+
+## Open Questions
+
+- Should marks/registers persist across sessions or only
+  within a session?
+- Accessibility: screen reader support for TUI mode?
+- Should `--watch` support inotify on Linux and FSEvents on
+  macOS, or use `notify` crate to abstract?
+- Maximum practical item count before we need virtual
+  scrolling? (Probably around 100k)
+- Should hooks run in a pool or strictly sequential?
+- Plugin system via WASM for custom filter strategies?
+  (Probably way later if ever)