# pikl-menu: Design Document

A keyboard-driven, hookable, streaming menu for Wayland and
X11. Built as a Rust library with TUI and GUI frontends.

Think rofi/wofi but with structured I/O, lifecycle hooks,
vim keybindings, streaming input/output, and multi-select.
The kind of tool you build a wallpaper picker on top of, or
a clipboard manager, or a process killer, or anything else
that's fundamentally "pick from a filterable list and do
something with it."

## Core Concept

pikl-menu is a **filterable, hookable, streaming item list
with structured I/O**. Everything else is a feature on top
of that core.

Items go in (stdin, file, directory watch, IPC). The user
filters, navigates, selects. Events fire (hover, select,
cancel, filter change). Selected items come out (stdout,
hooks, IPC). That's it.

## Structured I/O

### Input

Items are JSON lines on stdin. Each item has fields:

```jsonl
{"label": "beach_sunset.jpg", "sublabel": "/home/maple/walls/nature/", "meta": {"size": "2.4MB", "res": "3840x2160"}, "icon": "/path/to/thumb.png"}
{"label": "city_night.png", "sublabel": "/home/maple/walls/urban/", "meta": {"size": "1.8MB", "res": "2560x1440"}}
```

Minimum viable item is just `{"label": "something"}`.
Everything else is optional.

Plain text mode also works: one line per item, each line
becomes a label. Keeps it pipeable for simple use cases.

### Output

Selected items are emitted as JSON lines on stdout.
Multi-select emits multiple lines. The output includes any
metadata that came in, plus selection context:

```jsonl
{"label": "beach_sunset.jpg", "sublabel": "/home/maple/walls/nature/", "meta": {"size": "2.4MB"}, "action": "select", "index": 3}
```

### Streaming

Input can arrive over time. The list populates progressively
as items stream in. Useful for slow sources (network
queries, recursive directory walks, long-running commands).

Output can also stream. `on-hover` events can be configured
to emit to stdout as they happen, not just on final
selection. Useful for live preview pipelines.

### CSV / TSV

`--input-format csv` or `--input-format tsv` parses columnar
input. First row is headers, which become field names. Maps
naturally to table/column display mode.

## Event Hooks

Lifecycle events fire as the user interacts with the menu.
There are two ways to respond to them: **exec hooks** and
**handler hooks**.

### Lifecycle Events

| Event | Fires When | Use Case |
|---|---|---|
| `on-open` | Menu opens | Save current state for revert |
| `on-close` | Menu closes (any reason) | Cleanup |
| `on-hover` | Cursor moves to a new item | Live preview, prefetch |
| `on-select` | User confirms (Enter) | Apply the choice |
| `on-cancel` | User cancels (Escape) | Revert preview |
| `on-filter` | Filter text changes | Dynamic item reloading |
| `on-mark` | User marks/unmarks an item | Visual feedback |

### Exec Hooks (fire-and-forget)

`--on-<event>-exec` spawns a subprocess for each event.
The item is piped as JSON on stdin. Stdout is discarded.
One subprocess per event, no state fed back.

```sh
pikl --on-hover-exec 'notify-send "$(jq -r .label)"' \
     --on-select-exec 'apply-wallpaper.sh'
```

Good for simple side effects: notifications, applying a
setting, logging.

### Handler Hooks (bidirectional)

`--on-<event>` launches a **persistent process** that
receives events as JSON lines on stdin over the menu's
lifetime. The process can emit commands as JSON lines on
stdout to modify menu state.

```sh
pikl --on-hover './wallpaper-handler.sh' \
     --on-filter './search-provider.sh'
```

The handler process stays alive. Each time the event fires,
a new JSON line is written to its stdin. The process reads
them in a loop:

```sh
#!/bin/bash
while IFS= read -r event; do
  label=$(echo "$event" | jq -r '.label')
  set-wallpaper "$label"
  # optionally emit commands back to pikl on stdout
done
```

Stdout from the handler is parsed line-by-line as JSON
commands (see Handler Protocol below). Stderr passes
through to the terminal for debug output.

If the handler process exits unexpectedly, pikl logs a
warning via tracing and stops sending events. When the
menu closes, pikl closes the handler's stdin (breaking
the read loop naturally) and gives it a moment to exit
before killing it.

### Handler Protocol

Handler stdout commands, one JSON line per command:

| Action | Payload | Effect |
|---|---|---|
| `add_items` | `{"items": [...]}` | Append items to the list |
| `replace_items` | `{"items": [...]}` | Replace all items, preserve cursor position if possible |
| `remove_items` | `{"indices": [0, 3]}` | Remove items by index |
| `set_filter` | `{"text": "query"}` | Change the filter text |
| `close` | (none) | Close the menu |

Example:

```jsonl
{"action": "add_items", "items": [{"label": "new result"}]}
{"action": "set_filter", "text": "updated query"}
{"action": "close"}
```

Lines that don't parse as valid JSON or contain an unknown
action are logged as warnings (via tracing) and skipped.
Never fatal. A handler bug doesn't crash the menu.

For atomic multi-step mutations, use `replace_items` instead
of a `remove_items` + `add_items` pair. If a handler is
cancelled mid-stream (due to debounce), commands already
applied are not rolled back.

### Configuration via Manifest

Hooks can also be configured in a manifest file:

```toml
# ~/.config/pikl/wallpaper.toml
[hooks]
on-hover = './wallpaper-handler.sh'
on-select-exec = 'apply-wallpaper.sh'
on-cancel-exec = 'restore-wallpaper.sh'

[display]
columns = ["label", "meta.res"]
```

### Debouncing

All hooks (exec and handler) support debouncing. Three
modes:

| Mode | Behaviour | Default for |
|---|---|---|
| None | Fire immediately, every time | on-select, on-cancel, on-open, on-close |
| Debounce(ms) | Wait for quiet period, fire last event | on-filter (200ms) |
| Cancel-stale | New event cancels any in-flight invocation | (opt-in) |

Debounce and cancel-stale can combine: wait for quiet,
then fire, and if the previous invocation is still running,
cancel it first. This is the default for on-hover (200ms
debounce + cancel-stale).

CLI flags:

```sh
# Set debounce duration
pikl --on-hover './preview.sh' --on-hover-debounce 200

# Disable debounce (fire every event)
pikl --on-hover './preview.sh' --on-hover-debounce 0

# Enable cancel-stale (for exec hooks, kills subprocess;
# for handler hooks, a cancelled event is not sent)
pikl --on-hover-exec 'slow-command' --on-hover-cancel-stale
```

### Hook Architecture (Core vs CLI)

pikl-core defines the `HookHandler` trait and emits
lifecycle events. It does not know what handlers do with
them. The core manages debouncing and cancel-stale logic
since that interacts with the event loop timing.

```rust
pub trait HookHandler: Send + Sync {
    fn handle(&self, event: HookEvent)
        -> Result<Vec<HookResponse>, PiklError>;
}
```

The trait is deliberately synchronous for dyn-compatibility.
Implementations that need async work (spawning processes,
writing to channels) use `tokio::spawn` internally. This
keeps the trait object-safe so the core can hold
`Arc<dyn HookHandler>`.

The CLI binary provides `ShellExecHandler` and
`ShellHandlerHook`, which map CLI flags to shell commands.
A `CompositeHookHandler` dispatches to both based on event
kind. Library consumers implement their own handlers with
whatever behaviour they want: in-process closures, network
calls, anything.

## Filtering

### Strategies

The filter strategy is determined by the prefix/syntax of
the query text. No separate mode selector. The filter engine
parses the query on every keystroke and picks the right
strategy.

| Prefix | Strategy | Keeps/Removes | Case | Example |
|---|---|---|---|---|
| (none) | Fuzzy (nucleo) | keeps matches | smart case | `bch snst` matches "beach_sunset" |
| `'` | Exact substring | keeps matches | case-insensitive | `'sunset` matches "Sunset_beach" |
| `!` | Fuzzy inverse | removes matches | smart case | `!urban` excludes items matching "urban" |
| `!'` | Exact inverse | removes matches | case-insensitive | `!'temp` excludes items containing "temp" |
| `/pattern/` | Regex | keeps matches | case-sensitive | `/\d{4}x\d{4}/` matches resolution strings |
| `!/pattern/` | Regex inverse | removes matches | case-sensitive | `!/\.bak$/` excludes backup files |

**Case sensitivity rules:**

- **Fuzzy (nucleo):** Smart case. Case-insensitive until
  you type an uppercase character.
- **Exact substring:** Case-insensitive always.
- **Regex:** Case-sensitive by default. Use `(?i)` flag
  inside the pattern for case-insensitive matching. This
  follows standard regex convention.

The `!` prefix is a modifier that inverts any strategy. It
stacks with `'` and `/pattern/`.

### Chaining (Pipeline)

Filters chain with `|` (pipe). Each stage takes the previous
stage's output as its input and narrows further:

```
'log | !temp | /[0-9]+/
```

Means: exact match "log", then exclude items containing
"temp", then keep only items with at least one digit. Each
`|` is a pipeline step, not an OR.

OR semantics live inside regex: `/foo|bar/` matches either.
Outside regex, `|` always means "then filter again."

**Escaping:** `\|` produces a literal pipe character in
the query.

**Whitespace:** Spaces around `|` are trimmed. `log|!temp`
and `log | !temp` are identical.

**Empty segments:** Skipped silently. `log | | foo` is the
same as `log | foo`.

### Incremental Evaluation

The filter pipeline uses an incremental caching strategy to
avoid redundant work:

- Each pipeline stage caches its result as a set of item
  indices (not copies of the items).
- When the user edits a segment, only that stage and
  everything downstream is recomputed. Upstream stages keep
  their cached results.
- Backspacing over a `|` pops the last stage. Instant, no
  recomputation.
- Typing a new `|` pushes a new empty stage.
- Within a fuzzy stage, nucleo's own incremental narrowing
  applies when appending characters.
- Regex and exact stages recompute fully on each keystroke
  within their segment, but they run against the previous
  stage's (already filtered) output, so the input set is
  small.

The cursor position determines which segment is active.
Arrow keys move through the query text normally. Editing an
earlier segment invalidates from that point forward and
cascades recomputation.

### Regex Engine

Built on `fancy-regex` (pure Rust, no C deps). Full
PCRE2-style features:
- Unlimited capture groups (no artificial cap)
- Lookahead / lookbehind
- Non-greedy quantifiers
- Unicode support
- Named groups

Optional `pcre2` feature flag for JIT-compiled patterns
when performance matters on huge datasets.

### Filter Scoping

By default, filters match against the label.
`--filter-fields label,sublabel,meta.tags` expands the
search scope.

Field filters (`meta.res:3840`) are a Phase 3 feature that
builds on top of this pipeline infrastructure.

### Future: Syntax Highlighting

The filter input is a single text field. A future
enhancement is to syntax-highlight the query: color-code
prefixes (`'`, `!`, `/`), pipe delimiters, and regex
patterns so the user gets visual feedback on how their query
is being parsed. Low-cost, high-value, but deferred until
we get into proper theming work.

## Navigation & Interaction

### Modes

pikl-menu has two input modes, inspired by vim:

**Insert mode** (default on open):
- Typing goes to the filter input
- Arrow keys navigate the list
- Ctrl+N switches to normal mode
- Escape always cancels and exits (no exceptions, in any
  mode)

**Normal mode**:
- Full vim keybindings for navigation
- Ctrl+E switches to insert (edit) mode
- `/` enters insert mode with filter focused (like vim
  search)
- Escape cancels and exits

`--start-mode normal` to open in normal mode instead.

**Escape is sacred.** It always means cancel and exit,
regardless of mode. It never means "go back to previous
mode." This is a deliberate departure from vim to keep the
exit path predictable.

**Command mode** is planned for the future but not yet
designed.

### Vim Keybindings (Normal Mode)

| Key | Action |
|---|---|
| `j` / `k` | Down / up |
| `g g` | Jump to top |
| `G` | Jump to bottom |
| `Ctrl+D` / `Ctrl+U` | Half-page down / up |
| `Ctrl+F` / `Ctrl+B` | Full page down / up |
| `/` | Enter filter (insert) mode |
| `n` / `N` | Next / previous filter match |
| `m{a-z}` | Set mark at current position |
| `'{a-z}` | Jump to mark |
| `"a` | Select into register `a` |
| `v` | Toggle multi-select on current item |
| `V` | Visual line select mode |
| `Enter` | Confirm selection |
| `Escape` | Cancel and exit |
| `q` | Cancel and exit |
| `Space` | Toggle select current item and move down |

`H`/`M`/`L` (viewport-relative jumps) are deferred. They're
nice-to-have but not essential for the first pass of vim
navigation.

### Multi-Select

- `Space` or `v` toggles selection on current item
- `V` enters visual line mode, moving up/down selects ranges
- All selected items are emitted on confirm
- `"a` through `"z`: select into named registers (like vim
  yanking)
- Registers can be recalled or included in output

### Drill-Down

An `on-select` hook can return
`{"action": "replace", "items": [...]}` to replace the
current list in-place. No new process spawn. Combined with
a back keybind (`Backspace` or `h` in normal mode), this
gives you hierarchical navigation.

## Display

### List Mode (default)

Vertical list of items. Each entry shows available fields
based on a format template:

```sh
pikl --format '{icon} {label} <dim>{sublabel}</dim>'
```

### Table / Column Mode

```sh
pikl --columns label,meta.res,meta.size --column-separator ' │ '
```

Auto-aligns columns. Sortable per-column with keybinds.
CSV/TSV input maps naturally here.

### Preview Pane

Optional side panel that runs a command on the hovered item
and displays its stdout:

```sh
pikl --preview 'cat {meta.path}' --preview-position right --preview-width 50%
```

For GUI mode, this can render images (thumbnails, etc). For
TUI mode, text or sixel/kitty graphics protocol.

### Sections / Groups

Items can include a `group` field. pikl-menu renders group
headers and allows Tab to cycle between groups. Groups are
collapsible in normal mode with `za` (vim fold toggle).

### Theming

Minimal but configurable. TOML-based:

```toml
# ~/.config/pikl/theme.toml
[colors]
background = "#1a1a2e"
foreground = "#e0e0e0"
selection = "#e94560"
filter = "#0f3460"
dim = "#666666"
border = "#333333"

[layout]
width = "40%"
position = "right"  # left, right, center
padding = 1
```

A few built-in themes ship with pikl. `--theme monokai` or
`--theme nord` etc.

## Sessions

`--session name` persists the menu's state (filter text,
scroll position, selected items, marks, registers) across
invocations. State lives in
`~/.local/state/pikl/sessions/`.

No `--session` = ephemeral, nothing saved.

Session names are just strings, use shell expansion for
dynamic names:

```sh
pikl --session "walls-$(hostname)"
pikl --session "logs-$USER"
```

Session history is a log file alongside the state. Other
tools can tail it for observability.

## Watched Sources

```sh
pikl --watch ~/Pictures/walls/ --watch-extensions jpg,png,webp
```

The list updates live as files are added, removed, or
renamed. Combines naturally with hooks: a new file appears,
the list updates, the user can immediately select it.

Also supports watching a file (one item per line) or a
named pipe.

## IPC

While running, pikl-menu listens on a Unix socket
(`/run/user/$UID/pikl-$PID.sock`). External tools can:

- Push new items
- Remove items
- Update item fields
- Set the filter text
- Read current selection
- Close the menu

Protocol is newline-delimited JSON. Simple enough to use
with `socat` or any language's Unix socket support.

## Exit Codes

| Code | Meaning |
|---|---|
| 0 | User selected item(s) |
| 1 | User cancelled (Escape / q) |
| 2 | Error (bad input, hook failure, etc) |
| 10-19 | Custom actions (configurable keybinds, future) |

## Platform Support

### Primary Targets

- **Wayland** (layer-shell overlay via `iced` +
  `iced_layershell`)
- **X11** (standard window with override-redirect or EWMH
  hints)
- **TUI** (ratatui, runs in any terminal)

### Rendering Selection

pikl auto-detects the environment:
1. If `$WAYLAND_DISPLAY` is set: GUI (Wayland)
2. If `$DISPLAY` is set: GUI (X11)
3. Otherwise: TUI

Override with `--mode gui` or `--mode tui`.

## Architecture

```
pikl-core (library crate)
├── Item store       - structured items, streaming input, indexing
├── Filter engine    - fuzzy, regex, fancy-regex, field filters, chaining
├── Selection        - single, multi, registers, marks
├── Hook system      - lifecycle events, debouncing, bidirectional
├── Session          - persistence, history log
├── IPC server       - Unix socket, JSON protocol
└── Event bus        - internal pub/sub connecting all the above

pikl-tui (binary crate)
├── ratatui renderer
├── Terminal input handling
└── Sixel / kitty image protocol (optional)

pikl-gui (binary crate)
├── iced + layer-shell (Wayland)
├── iced + X11 windowing
└── Native image rendering

pikl (binary crate, unified CLI)
├── CLI argument parsing (clap)
├── Auto-detection of GUI vs TUI
├── Stdin/stdout JSON streaming
└── Manifest file loading
```

`pikl-core` has zero rendering dependencies. It's the
engine. The frontends are thin layers that subscribe to the
event bus and render state.

Third-party Rust tools can depend on `pikl-core` directly
and skip the CLI entirely. Your wallpaper picker becomes a
Rust binary that calls `pikl_core::Menu::new()` with items
and hook closures.

### Frontends

Every way of driving pikl is a frontend. Frontends send
`Action` variants into the core's `mpsc` channel and
optionally subscribe to `MenuEvent` via the `broadcast`
channel. The core is frontend-agnostic: it processes actions
sequentially regardless of where they came from.

| Frontend | Input Source | Interactive | Phase |
|---|---|---|---|
| TUI | Terminal keypresses (crossterm) | Yes | 1 |
| GUI | GUI events (iced) | Yes | 8 |
| Action-fd | Pre-validated script from a file descriptor | No (unless `show-ui`) | 1.5 |
| IPC | Unix socket, JSON protocol | Yes (bidirectional) | 6 |
| Lua | LuaJIT script via mlua | Yes (stateful, conditional) | Post-6 |

This framing means new interaction modes don't require core
changes: they're just new frontends that push actions.

## Action-fd (Headless Mode)

`--action-fd <N>` reads a pre-built action script from file
descriptor N. Items come from stdin, selected output goes to
stdout. No TUI or GUI is launched unless the script requests
it.

### Protocol

One action per line, plain text:

```
filter foo
move-down
move-down
confirm
```

Available actions:

| Action | Argument | Maps to |
|---|---|---|
| `filter <text>` | Filter string | `Action::UpdateFilter` |
| `move-up` | (none) | `Action::MoveUp` |
| `move-down` | (none) | `Action::MoveDown` |
| `move-to-top` | (none) | `Action::MoveToTop` |
| `move-to-bottom` | (none) | `Action::MoveToBottom` |
| `page-up` | (none) | `Action::PageUp` |
| `page-down` | (none) | `Action::PageDown` |
| `resize <n>` | Height (items) | `Action::Resize` |
| `confirm` | (none) | `Action::Confirm` |
| `cancel` | (none) | `Action::Cancel` |
| `show-ui` | (none) | Hand off to interactive frontend |
| `show-tui` | (none) | Hand off to TUI specifically |
| `show-gui` | (none) | Hand off to GUI specifically |

`show-ui` auto-detects the appropriate interactive frontend
(Wayland: GUI, X11: GUI, otherwise: TUI). `show-tui` and
`show-gui` are explicit overrides. All three must be the
last action in the script: anything after them is a
validation error.

### Execution Model

1. **Read + validate action-fd:** the entire script is
   loaded and validated before anything runs. Unknown
   actions, malformed arguments, and actions after
   `show-ui`/`show-tui`/`show-gui` are rejected with
   diagnostic errors.
2. **Read all stdin until EOF:** items are fully loaded. If
   stdin doesn't close within the timeout (default 30s),
   pikl exits with an error.
3. **Create Menu** with items, default viewport height
   of 50.
4. **Execute actions** sequentially.
5. **Exit with result.** If the script ended with
   `show-ui`/`show-tui`/`show-gui`, launch the interactive
   frontend and let the user finish.

### Stdin Timeout

`--stdin-timeout <seconds>` controls how long pikl waits
for stdin to close.

- **Action-fd mode:** defaults to 30 seconds. Catches
  mistakes like piping from a streaming source (`tail -f`).
- **Interactive mode:** defaults to 0 (no timeout). The
  user can Ctrl+C.
- `--stdin-timeout 0` disables the timeout in any mode.

### Constraints

- **Action-fd is intentionally stateless.** The caller sends
  actions but cannot read state back or branch on it. For
  conditional/stateful automation, use Lua (when available)
  or IPC.
- **Action-fd and streaming stdin are mutually exclusive.**
  Action-fd requires stdin to be finite and fully consumed
  before actions execute. Streaming input (phase 7) is for
  interactive frontends.
- **Unknown actions are validation errors**, not silently
  ignored. The script is pre-validated: there's no
  forward-compatibility concern since the caller controls
  both the script and the pikl version.

### Examples

```sh
# Simple: filter and confirm
exec 3<<'EOF'
filter projects
confirm
EOF
ls ~/dev | pikl --action-fd 3

# Pre-navigate, then hand off to user
exec 3<<'EOF'
filter .rs
move-down
move-down
show-ui
EOF
find . -type f | pikl --action-fd 3

# Integration test (Rust)
# Spawn pikl with items on stdin, actions on fd 3, assert stdout
```

## Key Crates

| Crate | Purpose |
|---|---|
| `fancy-regex` | Regex engine with lookaround support |
| `serde` + `serde_json` | Structured I/O |
| `clap` | CLI argument parsing |
| `ratatui` + `crossterm` | TUI rendering |
| `iced` + `iced_layershell` | GUI rendering (Wayland) |
| `nucleo` | Fuzzy matching with smart case and incremental narrowing |
| `tokio` | Async runtime (streaming, IPC, hooks) |
| `notify` | File/directory watching |
| `csv` | CSV/TSV input parsing |
| `toml` | Config and manifest parsing |
| `dirs` | XDG directory paths |

## Example Use Cases

```sh
# Simple picker: pipe in lines, get selection back
ls ~/walls/*.jpg | pikl

# Structured items with live preview hooks
find-walls --json | pikl \
  --on-hover 'set-wallpaper {label}' \
  --on-cancel 'restore-wallpaper' \
  --on-hover-debounce 100

# Process killer with table view
ps aux --no-headers | pikl --input-format tsv --columns 1,10,2 --multi

# Drill-down file browser
pikl --manifest ~/.config/pikl/filebrowser.toml --session filebrowser

# Clipboard history with preview
clipman list --json | pikl --preview 'echo {meta.content}' --on-select 'clipman paste {id}'
```

## Scripting Ladder

pikl has a deliberate progression for automation
complexity:

1. **Action-fd:** fire-and-forget scripts. No state, no
   branching, no custom DSL. For shell one-liners,
   integration tests, and simple automation. If you need
   conditionals, you've outgrown this.
2. **Exec hooks:** fire-and-forget shell commands triggered
   by lifecycle events. Subprocess per event, stdout
   discarded. For simple side effects.
3. **Handler hooks:** persistent bidirectional processes.
   Receive events as JSON lines on stdin, emit commands on
   stdout to modify menu state. The shell scripter's
   extension point: anyone who can write a bash script can
   extend pikl without touching Rust.
4. **IPC** (phase 6): bidirectional JSON over Unix socket.
   External tools can read state and send actions while pikl
   runs interactively. Good for tool integration.
5. **Lua** (post phase 6): embedded LuaJIT via mlua. Full
   stateful scripting: subscribe to events, branch on state,
   loops, the works. The Lua runtime is just another
   frontend pushing Actions and subscribing to MenuEvents.
   For anything complex enough to need a real language.

No custom DSL. Action-fd stays simple forever. The jump
from "I need conditionals" to "use Lua" is intentional:
there's no value in a half-language.

## Use Cases

Concrete workflows pikl-menu is designed to support.
These are first-class targets that inform feature
priorities, not hypotheticals. Each has its own doc
with the full writeup.

- [Wallpaper Picker](use-cases/wallpaper-picker.md):
  keyboard-driven wallpaper browser with live preview
  via hyprpaper hooks.
- [App Launcher](use-cases/app-launcher.md): global
  hotkey launcher that fuzzy-filters PATH binaries
  or .desktop entries.

## Open Questions

- Should marks/registers persist across sessions or only
  within a session?
- Accessibility: screen reader support for TUI mode?
- Should `--watch` support inotify on Linux and FSEvents on
  macOS, or use `notify` crate to abstract?
- Maximum practical item count before we need virtual
  scrolling? (Probably around 100k)
- Should hooks run in a pool or strictly sequential?
  Resolved: exec hooks are one subprocess per event.
  Handler hooks are persistent processes. Debounce and
  cancel-stale manage concurrency.
- Plugin system via WASM for custom filter strategies?
  (Probably way later if ever)