ollie-kde/prompts/tools-gui.md

GUI Automation Tools

KDE desktop automation via D-Bus and accessibility interfaces. All tools use call_tool.

Dependencies: gdbus (always), wmctrl/xdotool (window ops), ydotool (Wayland input), spectacle (screenshots), python3-gobject + AT-SPI2 typelib (accessibility).

---

gui_clipboard

Read/write the KDE clipboard via Klipper.

Args: [command] or [command, value]

Commands: get, set, history, clear

call_tool: calls=[{tool: "gui_clipboard", args: ["get"]}]
call_tool: calls=[{tool: "gui_clipboard", args: ["set", "hello world"]}]
call_tool: calls=[{tool: "gui_clipboard", args: ["history", "2"]}]

---

gui_notify

Show desktop notifications with optional interactive action buttons. Waits for user response when actions are specified.

Args: [summary] or [summary, body, --flags...]

Flags: --actions=label1,label2,..., --timeout=ms, --icon=name, --urgency=low|normal|critical

call_tool: calls=[{tool: "gui_notify", args: ["Build complete", "All tests passed"]}]
call_tool: calls=[{tool: "gui_notify", args: ["Deploy?", "Ready to push to prod", "--actions=Yes,No", "--urgency=critical"]}]

Returns: For action notifications, prints the chosen label (e.g., Yes) or dismissed/timeout.

---

gui_screenshot

Capture screenshots. Uses spectacle (KDE) with fallbacks to ImageMagick.

Args: [mode] with optional flags

Modes: active (default), screen, area, window

Flags: --output=path, --screen=N, --title=substr

call_tool: calls=[{tool: "gui_screenshot", args: ["active"]}]
call_tool: calls=[{tool: "gui_screenshot", args: ["screen", "--output=/tmp/shot.png"]}]
call_tool: calls=[{tool: "gui_screenshot", args: ["window", "--title=Firefox"]}]

Returns: output file path and size.

---

gui_windows

Manage windows: list, focus, close, move, resize, virtual desktops.

Args: [command, ...params]

Commands:

• list — all windows with geometry and state
• focus <title> — raise and focus
• close <title> — close window
• minimize <title> / maximize <title>
• move <title> <x> <y> — reposition
• resize <title> <w> <h> — resize
• desktops — list virtual desktops
• desktop <n> — switch to desktop N

call_tool: calls=[{tool: "gui_windows", args: ["list"]}]
call_tool: calls=[{tool: "gui_windows", args: ["focus", "Kate"]}]
call_tool: calls=[{tool: "gui_windows", args: ["move", "Terminal", "0", "0"]}]
call_tool: calls=[{tool: "gui_windows", args: ["resize", "Firefox", "1920", "1080"]}]
call_tool: calls=[{tool: "gui_windows", args: ["desktop", "2"]}]

Title matching is case-insensitive substring.

---

gui_apps

Launch, list, or activate applications.

Args: [command] or [command, app_name]

Commands: list (running), installed (all .desktop), launch <name>, activate <title_or_class>

call_tool: calls=[{tool: "gui_apps", args: ["launch", "firefox"]}]
call_tool: calls=[{tool: "gui_apps", args: ["activate", "dolphin"]}]
call_tool: calls=[{tool: "gui_apps", args: ["installed"]}]

---

gui_input

Simulate keyboard and mouse input. Uses ydotool (Wayland) or xdotool (X11).

Args: [command, ...params]

Commands:

• key <combo> — press key combo (e.g., ctrl+c, Return, alt+F4, super)
• type <text> — type a string character by character
• click [button] [x y] — mouse click (1=left, 2=middle, 3=right)
• move <x> <y> [--relative] — move cursor
• scroll <up|down|left|right> [amount]

call_tool: calls=[{tool: "gui_input", args: ["key", "ctrl+s"]}]
call_tool: calls=[{tool: "gui_input", args: ["type", "Hello, world!"]}]
call_tool: calls=[{tool: "gui_input", args: ["click", "1", "500", "300"]}]
call_tool: calls=[{tool: "gui_input", args: ["scroll", "down", "5"]}]

Caution: Input simulation is a write operation — confirm intent before using key with destructive combos (alt+F4, ctrl+w) or click on unknown targets.

---

gui_activities

Manage KDE Plasma Activities.

Args: [command] or [command, name_or_id]

Commands: list, current, switch <name_or_id>, create <name>, remove <name_or_id>

call_tool: calls=[{tool: "gui_activities", args: ["list"]}]
call_tool: calls=[{tool: "gui_activities", args: ["switch", "Development"]}]
call_tool: calls=[{tool: "gui_activities", args: ["create", "Research"]}]

---

gui_files

Search files via Baloo (KDE file indexer). Falls back to locate.

Args: [query, ...flags]

Flags: --type=document|image|video|audio|folder, --path=dir, --limit=N

call_tool: calls=[{tool: "gui_files", args: ["meeting notes"]}]
call_tool: calls=[{tool: "gui_files", args: ["invoice", "--type=document", "--path=/home/lkn/doc"]}]
call_tool: calls=[{tool: "gui_files", args: ["screenshot", "--type=image", "--limit=10"]}]

---

gui_shortcuts

Manage and trigger KDE global keyboard shortcuts.

Args: [command, ...params]

Commands:

• list — all shortcut components
• list <component> — shortcuts for that component
• get <component> <name> — current key binding
• trigger <component> <name> — invoke the shortcut action
• set <component> <name> <keys> — change key binding

call_tool: calls=[{tool: "gui_shortcuts", args: ["list"]}]
call_tool: calls=[{tool: "gui_shortcuts", args: ["list", "kwin"]}]
call_tool: calls=[{tool: "gui_shortcuts", args: ["trigger", "kwin", "Overview"]}]
call_tool: calls=[{tool: "gui_shortcuts", args: ["trigger", "plasmashell", "show dashboard"]}]

---

gui_accessibility

Full GUI automation via the AT-SPI2 accessibility tree. Inspect, navigate, and interact with any application's UI elements.

Args: [command, app_name, ...params]

Commands:

• apps — list all accessible applications (no app_name needed)
• tree <app> [depth] — dump widget tree (default depth: 4)
• inspect <app> [depth] — brief overview (default depth: 2)
• find <app> <role> [name] — find elements by role and optional name
• read <app> <path> — read text/value of element
• click <app> <path> — activate/click element
• type <app> <path> <text> — type into element
• action <app> <path> <action_name> — invoke named action

Paths use slash-separated indices (0/2/1) or role:name patterns (push button:OK).

call_tool: calls=[{tool: "gui_accessibility", args: ["apps"]}]
call_tool: calls=[{tool: "gui_accessibility", args: ["inspect", "Kate"]}]
call_tool: calls=[{tool: "gui_accessibility", args: ["find", "Firefox", "push button", "Reload"]}]
call_tool: calls=[{tool: "gui_accessibility", args: ["click", "Firefox", "push button:Reload"]}]
call_tool: calls=[{tool: "gui_accessibility", args: ["type", "Kate", "0/0/text:untitled", "Hello world"]}]
call_tool: calls=[{tool: "gui_accessibility", args: ["tree", "Dolphin", "3"]}]

Workflow: Use apps → inspect → find to locate elements, then click/type/action to interact.

Requires: python3-gobject and AT-SPI2 typelib (dev-libs/at-spi2-core on Gentoo).

---

gui_brightness

Get/set screen brightness via PowerDevil. Accepts percentages (0–100), not raw hardware values.

Args: [command] or [command, value]

Commands: get, set <percent>, up [step], down [step]

call_tool: calls=[{tool: "gui_brightness", args: ["get"]}]
call_tool: calls=[{tool: "gui_brightness", args: ["set", "50"]}]
call_tool: calls=[{tool: "gui_brightness", args: ["up", "10"]}]
call_tool: calls=[{tool: "gui_brightness", args: ["down", "25"]}]

Note: The underlying PowerDevil API uses absolute values (0 to ~10000). This tool handles the conversion — always pass percentages.