pikl/docs/DESIGN.md

pikl-menu: Design Document

A keyboard-driven, hookable, streaming menu for Wayland and X11. Built as a Rust library with TUI and GUI frontends.

Think rofi/wofi but with structured I/O, lifecycle hooks, vim keybindings, streaming input/output, and multi-select. The kind of tool you build a wallpaper picker on top of, or a clipboard manager, or a process killer, or anything else that's fundamentally "pick from a filterable list and do something with it."

Core Concept

pikl-menu is a filterable, hookable, streaming item list with structured I/O. Everything else is a feature on top of that core.

Items go in (stdin, file, directory watch, IPC). The user filters, navigates, selects. Events fire (hover, select, cancel, filter change). Selected items come out (stdout, hooks, IPC). That's it.

Structured I/O

Input

Items are JSON lines on stdin. Each item has fields:

{"label": "beach_sunset.jpg", "sublabel": "/home/maple/walls/nature/", "meta": {"size": "2.4MB", "res": "3840x2160"}, "icon": "/path/to/thumb.png"}
{"label": "city_night.png", "sublabel": "/home/maple/walls/urban/", "meta": {"size": "1.8MB", "res": "2560x1440"}}

Minimum viable item is just {"label": "something"}. Everything else is optional.

Plain text mode also works: one line per item, each line becomes a label. Keeps it pipeable for simple use cases.

Output

Selected items are emitted as JSON lines on stdout. Multi-select emits multiple lines. The output includes any metadata that came in, plus selection context:

{"label": "beach_sunset.jpg", "sublabel": "/home/maple/walls/nature/", "meta": {"size": "2.4MB"}, "action": "select", "index": 3}

Streaming

Input can arrive over time. The list populates progressively as items stream in. Useful for slow sources (network queries, recursive directory walks, long-running commands).

Output can also stream. on-hover events can be configured to emit to stdout as they happen, not just on final selection. Useful for live preview pipelines.

CSV / TSV

--input-format csv or --input-format tsv parses columnar input. First row is headers, which become field names. Maps naturally to table/column display mode.

Event Hooks

Lifecycle events fire as the user interacts with the menu. There are two ways to respond to them: exec hooks and handler hooks.

Lifecycle Events

Event	Fires When	Use Case
on-open	Menu opens	Save current state for revert
on-close	Menu closes (any reason)	Cleanup
on-hover	Cursor moves to a new item	Live preview, prefetch
on-select	User confirms (Enter)	Apply the choice
on-cancel	User cancels (Escape)	Revert preview
on-filter	Filter text changes	Dynamic item reloading
on-mark	User marks/unmarks an item	Visual feedback

Exec Hooks (fire-and-forget)

--on-<event>-exec spawns a subprocess for each event. The item is piped as JSON on stdin. Stdout is discarded. One subprocess per event, no state fed back.

pikl --on-hover-exec 'notify-send "$(jq -r .label)"' \
     --on-select-exec 'apply-wallpaper.sh'

Good for simple side effects: notifications, applying a setting, logging.

Handler Hooks (bidirectional)

--on-<event> launches a persistent process that receives events as JSON lines on stdin over the menu's lifetime. The process can emit commands as JSON lines on stdout to modify menu state.

pikl --on-hover './wallpaper-handler.sh' \
     --on-filter './search-provider.sh'

The handler process stays alive. Each time the event fires, a new JSON line is written to its stdin. The process reads them in a loop:

#!/bin/bash
while IFS= read -r event; do
  label=$(echo "$event" | jq -r '.label')
  set-wallpaper "$label"
  # optionally emit commands back to pikl on stdout
done

Stdout from the handler is parsed line-by-line as JSON commands (see Handler Protocol below). Stderr passes through to the terminal for debug output.

If the handler process exits unexpectedly, pikl logs a warning via tracing and stops sending events. When the menu closes, pikl closes the handler's stdin (breaking the read loop naturally) and gives it a moment to exit before killing it.

Handler Protocol

Handler stdout commands, one JSON line per command:

Action	Payload	Effect
add_items	{"items": [...]}	Append items to the list
replace_items	{"items": [...]}	Replace all items, preserve cursor position if possible
remove_items	{"indices": [0, 3]}	Remove items by index
set_filter	{"text": "query"}	Change the filter text
close	(none)	Close the menu

Example:

{"action": "add_items", "items": [{"label": "new result"}]}
{"action": "set_filter", "text": "updated query"}
{"action": "close"}

Lines that don't parse as valid JSON or contain an unknown action are logged as warnings (via tracing) and skipped. Never fatal. A handler bug doesn't crash the menu.

For atomic multi-step mutations, use replace_items instead of a remove_items + add_items pair. If a handler is cancelled mid-stream (due to debounce), commands already applied are not rolled back.

Configuration via Manifest

Hooks can also be configured in a manifest file:

# ~/.config/pikl/wallpaper.toml
[hooks]
on-hover = './wallpaper-handler.sh'
on-select-exec = 'apply-wallpaper.sh'
on-cancel-exec = 'restore-wallpaper.sh'

[display]
columns = ["label", "meta.res"]

Debouncing

All hooks (exec and handler) support debouncing. Three modes:

Mode	Behaviour	Default for
None	Fire immediately, every time	on-select, on-cancel, on-open, on-close
Debounce(ms)	Wait for quiet period, fire last event	on-filter (200ms)
Cancel-stale	New event cancels any in-flight invocation	(opt-in)

Debounce and cancel-stale can combine: wait for quiet, then fire, and if the previous invocation is still running, cancel it first. This is the default for on-hover (200ms debounce + cancel-stale).

CLI flags:

# Set debounce duration
pikl --on-hover './preview.sh' --on-hover-debounce 200

# Disable debounce (fire every event)
pikl --on-hover './preview.sh' --on-hover-debounce 0

# Enable cancel-stale (for exec hooks, kills subprocess;
# for handler hooks, a cancelled event is not sent)
pikl --on-hover-exec 'slow-command' --on-hover-cancel-stale

Hook Architecture (Core vs CLI)

pikl-core defines the HookHandler trait and emits lifecycle events. It does not know what handlers do with them. The core manages debouncing and cancel-stale logic since that interacts with the event loop timing.

pub trait HookHandler: Send + Sync {
    fn handle(&self, event: HookEvent)
        -> Result<Vec<HookResponse>, PiklError>;
}

The trait is deliberately synchronous for dyn-compatibility. Implementations that need async work (spawning processes, writing to channels) use tokio::spawn internally. This keeps the trait object-safe so the core can hold Arc<dyn HookHandler>.

The CLI binary provides ShellExecHandler and ShellHandlerHook, which map CLI flags to shell commands. A CompositeHookHandler dispatches to both based on event kind. Library consumers implement their own handlers with whatever behaviour they want: in-process closures, network calls, anything.

Filtering

Strategies

The filter strategy is determined by the prefix/syntax of the query text. No separate mode selector. The filter engine parses the query on every keystroke and picks the right strategy.

Prefix	Strategy	Keeps/Removes	Case	Example
(none)	Fuzzy (nucleo)	keeps matches	smart case	bch snst matches "beach_sunset"
'	Exact substring	keeps matches	case-insensitive	'sunset matches "Sunset_beach"
!	Fuzzy inverse	removes matches	smart case	!urban excludes items matching "urban"
!'	Exact inverse	removes matches	case-insensitive	!'temp excludes items containing "temp"
/pattern/	Regex	keeps matches	case-sensitive	/\d{4}x\d{4}/ matches resolution strings
!/pattern/	Regex inverse	removes matches	case-sensitive	!/\.bak$/ excludes backup files

Case sensitivity rules:

• Fuzzy (nucleo): Smart case. Case-insensitive until you type an uppercase character.
• Exact substring: Case-insensitive always.
• Regex: Case-sensitive by default. Use (?i) flag inside the pattern for case-insensitive matching. This follows standard regex convention.

The ! prefix is a modifier that inverts any strategy. It stacks with ' and /pattern/.

Chaining (Pipeline)

Filters chain with | (pipe). Each stage takes the previous stage's output as its input and narrows further:

'log | !temp | /[0-9]+/

Means: exact match "log", then exclude items containing "temp", then keep only items with at least one digit. Each | is a pipeline step, not an OR.

OR semantics live inside regex: /foo|bar/ matches either. Outside regex, | always means "then filter again."

Escaping: \| produces a literal pipe character in the query.

Whitespace: Spaces around | are trimmed. log|!temp and log | !temp are identical.

Empty segments: Skipped silently. log | | foo is the same as log | foo.

Incremental Evaluation

The filter pipeline uses an incremental caching strategy to avoid redundant work:

• Each pipeline stage caches its result as a set of item indices (not copies of the items).
• When the user edits a segment, only that stage and everything downstream is recomputed. Upstream stages keep their cached results.
• Backspacing over a | pops the last stage. Instant, no recomputation.
• Typing a new | pushes a new empty stage.
• Within a fuzzy stage, nucleo's own incremental narrowing applies when appending characters.
• Regex and exact stages recompute fully on each keystroke within their segment, but they run against the previous stage's (already filtered) output, so the input set is small.

The cursor position determines which segment is active. Arrow keys move through the query text normally. Editing an earlier segment invalidates from that point forward and cascades recomputation.

Regex Engine

Built on fancy-regex (pure Rust, no C deps). Full PCRE2-style features:

• Unlimited capture groups (no artificial cap)
• Lookahead / lookbehind
• Non-greedy quantifiers
• Unicode support
• Named groups

Optional pcre2 feature flag for JIT-compiled patterns when performance matters on huge datasets.

Filter Scoping

By default, filters match against the label. --filter-fields label,sublabel,meta.tags expands the search scope.

Field filters (meta.res:3840) are a Phase 3 feature that builds on top of this pipeline infrastructure.

Future: Syntax Highlighting

The filter input is a single text field. A future enhancement is to syntax-highlight the query: color-code prefixes (', !, /), pipe delimiters, and regex patterns so the user gets visual feedback on how their query is being parsed. Low-cost, high-value, but deferred until we get into proper theming work.

Navigation & Interaction

Modes

pikl-menu has two input modes, inspired by vim:

Insert mode (default on open):

• Typing goes to the filter input
• Arrow keys navigate the list
• Ctrl+N switches to normal mode
• Escape always cancels and exits (no exceptions, in any mode)

Normal mode:

• Full vim keybindings for navigation
• Ctrl+E switches to insert (edit) mode
• / enters insert mode with filter focused (like vim search)
• Escape cancels and exits

--start-mode normal to open in normal mode instead.

Escape is sacred. It always means cancel and exit, regardless of mode. It never means "go back to previous mode." This is a deliberate departure from vim to keep the exit path predictable.

Command mode is planned for the future but not yet designed.

Vim Keybindings (Normal Mode)

Key	Action
j / k	Down / up
g g	Jump to top
G	Jump to bottom
Ctrl+D / Ctrl+U	Half-page down / up
Ctrl+F / Ctrl+B	Full page down / up
/	Enter filter (insert) mode
n / N	Next / previous filter match
m{a-z}	Set mark at current position
'{a-z}	Jump to mark
"a	Select into register a
v	Toggle multi-select on current item
V	Visual line select mode
Enter	Confirm selection
Escape	Cancel and exit
q	Cancel and exit
Space	Toggle select current item and move down

H/M/L (viewport-relative jumps) are deferred. They're nice-to-have but not essential for the first pass of vim navigation.

Multi-Select

• Space or v toggles selection on current item
• V enters visual line mode, moving up/down selects ranges
• All selected items are emitted on confirm
• "a through "z: select into named registers (like vim yanking)
• Registers can be recalled or included in output

Drill-Down

An on-select hook can return {"action": "replace", "items": [...]} to replace the current list in-place. No new process spawn. Combined with a back keybind (Backspace or h in normal mode), this gives you hierarchical navigation.

Display

List Mode (default)

Vertical list of items. Each entry shows available fields based on a format template:

pikl --format '{icon} {label} <dim>{sublabel}</dim>'

Table / Column Mode

pikl --columns label,meta.res,meta.size --column-separator ' │ '

Auto-aligns columns. Sortable per-column with keybinds. CSV/TSV input maps naturally here.

Preview Pane

Optional side panel that runs a command on the hovered item and displays its stdout:

pikl --preview 'cat {meta.path}' --preview-position right --preview-width 50%

For GUI mode, this can render images (thumbnails, etc). For TUI mode, text or sixel/kitty graphics protocol.

Sections / Groups

Items can include a group field. pikl-menu renders group headers and allows Tab to cycle between groups. Groups are collapsible in normal mode with za (vim fold toggle).

Theming

Minimal but configurable. TOML-based:

# ~/.config/pikl/theme.toml
[colors]
background = "#1a1a2e"
foreground = "#e0e0e0"
selection = "#e94560"
filter = "#0f3460"
dim = "#666666"
border = "#333333"

[layout]
width = "40%"
position = "right"  # left, right, center
padding = 1

A few built-in themes ship with pikl. --theme monokai or --theme nord etc.

Sessions

--session name persists the menu's state (filter text, scroll position, selected items, marks, registers) across invocations. State lives in ~/.local/state/pikl/sessions/.

No --session = ephemeral, nothing saved.

Session names are just strings, use shell expansion for dynamic names:

pikl --session "walls-$(hostname)"
pikl --session "logs-$USER"

Session history is a log file alongside the state. Other tools can tail it for observability.

Watched Sources

pikl --watch ~/Pictures/walls/ --watch-extensions jpg,png,webp

The list updates live as files are added, removed, or renamed. Combines naturally with hooks: a new file appears, the list updates, the user can immediately select it.

Also supports watching a file (one item per line) or a named pipe.

IPC

While running, pikl-menu listens on a Unix socket (/run/user/$UID/pikl-$PID.sock). External tools can:

• Push new items
• Remove items
• Update item fields
• Set the filter text
• Read current selection
• Close the menu

Protocol is newline-delimited JSON. Simple enough to use with socat or any language's Unix socket support.

Exit Codes

Code	Meaning
0	User selected item(s)
1	User cancelled (Escape / q)
2	Error (bad input, hook failure, etc)
10-19	Custom actions (configurable keybinds, future)

Platform Support

Primary Targets

• Wayland (layer-shell overlay via iced + iced_layershell)
• X11 (standard window with override-redirect or EWMH hints)
• TUI (ratatui, runs in any terminal)

Rendering Selection

pikl auto-detects the environment:

1. If $WAYLAND_DISPLAY is set: GUI (Wayland)
2. If $DISPLAY is set: GUI (X11)
3. Otherwise: TUI

Override with --mode gui or --mode tui.

Architecture

pikl-core (library crate)
├── Item store       - structured items, streaming input, indexing
├── Filter engine    - fuzzy, regex, fancy-regex, field filters, chaining
├── Selection        - single, multi, registers, marks
├── Hook system      - lifecycle events, debouncing, bidirectional
├── Session          - persistence, history log
├── IPC server       - Unix socket, JSON protocol
└── Event bus        - internal pub/sub connecting all the above

pikl-tui (binary crate)
├── ratatui renderer
├── Terminal input handling
└── Sixel / kitty image protocol (optional)

pikl-gui (binary crate)
├── iced + layer-shell (Wayland)
├── iced + X11 windowing
└── Native image rendering

pikl (binary crate, unified CLI)
├── CLI argument parsing (clap)
├── Auto-detection of GUI vs TUI
├── Stdin/stdout JSON streaming
└── Manifest file loading

pikl-core has zero rendering dependencies. It's the engine. The frontends are thin layers that subscribe to the event bus and render state.

Third-party Rust tools can depend on pikl-core directly and skip the CLI entirely. Your wallpaper picker becomes a Rust binary that calls pikl_core::Menu::new() with items and hook closures.

Frontends

Every way of driving pikl is a frontend. Frontends send Action variants into the core's mpsc channel and optionally subscribe to MenuEvent via the broadcast channel. The core is frontend-agnostic: it processes actions sequentially regardless of where they came from.

Frontend	Input Source	Interactive	Phase
TUI	Terminal keypresses (crossterm)	Yes	1
GUI	GUI events (iced)	Yes	8
Action-fd	Pre-validated script from a file descriptor	No (unless show-ui)	1.5
IPC	Unix socket, JSON protocol	Yes (bidirectional)	6
Lua	LuaJIT script via mlua	Yes (stateful, conditional)	Post-6

This framing means new interaction modes don't require core changes: they're just new frontends that push actions.

Action-fd (Headless Mode)

--action-fd <N> reads a pre-built action script from file descriptor N. Items come from stdin, selected output goes to stdout. No TUI or GUI is launched unless the script requests it.

Protocol

One action per line, plain text:

filter foo
move-down
move-down
confirm

Available actions:

Action	Argument	Maps to
filter <text>	Filter string	Action::UpdateFilter
move-up	(none)	Action::MoveUp
move-down	(none)	Action::MoveDown
move-to-top	(none)	Action::MoveToTop
move-to-bottom	(none)	Action::MoveToBottom
page-up	(none)	Action::PageUp
page-down	(none)	Action::PageDown
resize <n>	Height (items)	Action::Resize
confirm	(none)	Action::Confirm
cancel	(none)	Action::Cancel
show-ui	(none)	Hand off to interactive frontend
show-tui	(none)	Hand off to TUI specifically
show-gui	(none)	Hand off to GUI specifically

show-ui auto-detects the appropriate interactive frontend (Wayland: GUI, X11: GUI, otherwise: TUI). show-tui and show-gui are explicit overrides. All three must be the last action in the script: anything after them is a validation error.

Execution Model

1. Read + validate action-fd: the entire script is loaded and validated before anything runs. Unknown actions, malformed arguments, and actions after show-ui/show-tui/show-gui are rejected with diagnostic errors.
2. Read all stdin until EOF: items are fully loaded. If stdin doesn't close within the timeout (default 30s), pikl exits with an error.
3. Create Menu with items, default viewport height of 50.
4. Execute actions sequentially.
5. Exit with result. If the script ended with show-ui/show-tui/show-gui, launch the interactive frontend and let the user finish.

Stdin Timeout

--stdin-timeout <seconds> controls how long pikl waits for stdin to close.

• Action-fd mode: defaults to 30 seconds. Catches mistakes like piping from a streaming source (tail -f).
• Interactive mode: defaults to 0 (no timeout). The user can Ctrl+C.
• --stdin-timeout 0 disables the timeout in any mode.

Constraints

• Action-fd is intentionally stateless. The caller sends actions but cannot read state back or branch on it. For conditional/stateful automation, use Lua (when available) or IPC.
• Action-fd and streaming stdin are mutually exclusive. Action-fd requires stdin to be finite and fully consumed before actions execute. Streaming input (phase 7) is for interactive frontends.
• Unknown actions are validation errors, not silently ignored. The script is pre-validated: there's no forward-compatibility concern since the caller controls both the script and the pikl version.

Examples

# Simple: filter and confirm
exec 3<<'EOF'
filter projects
confirm
EOF
ls ~/dev | pikl --action-fd 3

# Pre-navigate, then hand off to user
exec 3<<'EOF'
filter .rs
move-down
move-down
show-ui
EOF
find . -type f | pikl --action-fd 3

# Integration test (Rust)
# Spawn pikl with items on stdin, actions on fd 3, assert stdout

Key Crates

Crate	Purpose
fancy-regex	Regex engine with lookaround support
serde + serde_json	Structured I/O
clap	CLI argument parsing
ratatui + crossterm	TUI rendering
iced + iced_layershell	GUI rendering (Wayland)
nucleo	Fuzzy matching with smart case and incremental narrowing
tokio	Async runtime (streaming, IPC, hooks)
notify	File/directory watching
csv	CSV/TSV input parsing
toml	Config and manifest parsing
dirs	XDG directory paths

Example Use Cases

# Simple picker: pipe in lines, get selection back
ls ~/walls/*.jpg | pikl

# Structured items with live preview hooks
find-walls --json | pikl \
  --on-hover 'set-wallpaper {label}' \
  --on-cancel 'restore-wallpaper' \
  --on-hover-debounce 100

# Process killer with table view
ps aux --no-headers | pikl --input-format tsv --columns 1,10,2 --multi

# Drill-down file browser
pikl --manifest ~/.config/pikl/filebrowser.toml --session filebrowser

# Clipboard history with preview
clipman list --json | pikl --preview 'echo {meta.content}' --on-select 'clipman paste {id}'

Scripting Ladder

pikl has a deliberate progression for automation complexity:

1. Action-fd: fire-and-forget scripts. No state, no branching, no custom DSL. For shell one-liners, integration tests, and simple automation. If you need conditionals, you've outgrown this.
2. Exec hooks: fire-and-forget shell commands triggered by lifecycle events. Subprocess per event, stdout discarded. For simple side effects.
3. Handler hooks: persistent bidirectional processes. Receive events as JSON lines on stdin, emit commands on stdout to modify menu state. The shell scripter's extension point: anyone who can write a bash script can extend pikl without touching Rust.
4. IPC (phase 6): bidirectional JSON over Unix socket. External tools can read state and send actions while pikl runs interactively. Good for tool integration.
5. Lua (post phase 6): embedded LuaJIT via mlua. Full stateful scripting: subscribe to events, branch on state, loops, the works. The Lua runtime is just another frontend pushing Actions and subscribing to MenuEvents. For anything complex enough to need a real language.

No custom DSL. Action-fd stays simple forever. The jump from "I need conditionals" to "use Lua" is intentional: there's no value in a half-language.

Use Cases

Concrete workflows pikl-menu is designed to support. These are first-class targets that inform feature priorities, not hypotheticals. Each has its own doc with the full writeup.

• Wallpaper Picker: keyboard-driven wallpaper browser with live preview via hyprpaper hooks.
• App Launcher: global hotkey launcher that fuzzy-filters PATH binaries or .desktop entries.

Open Questions

• Should marks/registers persist across sessions or only within a session?
• Accessibility: screen reader support for TUI mode?
• Should --watch support inotify on Linux and FSEvents on macOS, or use notify crate to abstract?
• Maximum practical item count before we need virtual scrolling? (Probably around 100k)
• Should hooks run in a pool or strictly sequential? Resolved: exec hooks are one subprocess per event. Handler hooks are persistent processes. Debounce and cancel-stale manage concurrency.
• Plugin system via WASM for custom filter strategies? (Probably way later if ever)