From 66018f19dea18d59143825c92bab4516e57bd1a9 Mon Sep 17 00:00:00 2001 From: Patrick Bertsch Date: Wed, 25 Mar 2026 23:35:43 -0600 Subject: [PATCH 1/7] feat: physical device support for iOS and Android (v0.5.0) --- CHANGELOG.md | 27 ++++ cmd/probe/main_test.go | 19 +++ docs/wiki/Architecture-Overview.md | 17 ++- docs/wiki/Home.md | 2 +- docs/wiki/Troubleshooting.md | 21 +++ docs/wiki/iOS-Integration-Guide.md | 59 ++++++++ internal/cli/cli_test.go | 47 +++++++ internal/cli/test.go | 52 +++++-- internal/device/manager.go | 215 +++++++++++++++++++++++++++-- internal/device/manager_test.go | 72 ++++++++++ internal/ios/devicectl.go | 119 ++++++++++++++++ internal/ios/devicectl_test.go | 25 ++++ internal/probelink/client.go | 70 +++++++++- internal/runner/device_context.go | 89 +++++++++--- internal/runner/executor.go | 81 +++++++++++ probe_agent/lib/src/agent.dart | 64 +++++++-- probe_agent/lib/src/executor.dart | 23 ++- probe_agent/lib/src/finder.dart | 58 ++++++-- probe_agent/lib/src/server.dart | 5 + vscode/package.json | 2 +- website/src/pages/index.astro | 2 +- 21 files changed, 996 insertions(+), 73 deletions(-) create mode 100644 cmd/probe/main_test.go create mode 100644 internal/cli/cli_test.go create mode 100644 internal/device/manager_test.go create mode 100644 internal/ios/devicectl.go create mode 100644 internal/ios/devicectl_test.go diff --git a/CHANGELOG.md b/CHANGELOG.md index 1462470..322d66f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,33 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). ## [Unreleased] +## [0.5.0] - 2026-03-26 + +### Added + +- Physical iOS device support: launch/terminate via `xcrun devicectl`, token reading via `idevicesyslog`, port forwarding via `iproxy` +- Physical Android device validation: `EnsureADB()` verifies binary, device reachability, and cleans stale port forwards +- Physical device detection: `IsPhysicalIOS` (simctl list check) and `IsPhysicalAndroid` (ro.hardware property check) +- Physical iOS devices listed in `probe device list` via `idevice_id` +- WebSocket ping/pong keepalive (5s interval) — prevents idle connection drops on physical devices via iproxy +- Auto-reconnect on WebSocket connection loss — up to 2 transparent retries per step with full re-dial +- `EnsureIProxy()` — automatic iproxy lifecycle management: checks installation, kills stale processes, starts fresh, defers cleanup +- Visibility filtering in widget finder — off-screen widgets (behind routes, Offstage, Visibility) no longer match `see`/`if appears` +- Unique pointer IDs for synthetic gestures — prevents collision with real touch events on physical devices +- ProbeAgent profile mode support — `ProbeAgent.start()` works in profile builds (required for physical iOS) +- ProbeAgent release mode safeguards — blocked by default, opt-in via `allowReleaseBuild: true` + `PROBE_AGENT_FORCE=true` +- Test files for all packages: `cmd/probe`, `internal/cli`, `internal/ios`, `internal/device` (manager tests) + +### Changed + +- Operations unsupported on physical devices now skip gracefully with warnings instead of crashing: + - `clear app data` on physical iOS → warning + skip + - `allow/deny permission` on physical iOS → warning + skip + - `set location` on any physical device → warning + skip +- `restart the app` on physical iOS uses `xcrun devicectl` instead of `simctl` +- iOS connection setup now branches: simulator path uses simctl permissions + loopback; physical path uses iproxy + idevicesyslog +- Android connection setup validates ADB availability and device state before port forwarding + ## [0.4.2] - 2026-03-25 ### Added diff --git a/cmd/probe/main_test.go b/cmd/probe/main_test.go new file mode 100644 index 0000000..97d26bd --- /dev/null +++ b/cmd/probe/main_test.go @@ -0,0 +1,19 @@ +package main + +import ( + "os" + "testing" +) + +func TestMainBinaryExists(t *testing.T) { + // Verify the main package compiles (this test existing is the assertion). + // The actual CLI is tested via internal/cli tests. +} + +func TestMainNotNilEntrypoint(t *testing.T) { + // Ensure we don't accidentally nil-deref on startup by checking + // that the os package is available (basic sanity check). + if os.Args == nil { + t.Fatal("os.Args is nil") + } +} diff --git a/docs/wiki/Architecture-Overview.md b/docs/wiki/Architecture-Overview.md index 869924e..f90b61d 100644 --- a/docs/wiki/Architecture-Overview.md +++ b/docs/wiki/Architecture-Overview.md @@ -7,7 +7,7 @@ FlutterProbe consists of two main components that communicate via WebSocket + JS ``` +------------------+ WebSocket (JSON-RPC 2.0) +------------------+ | Go CLI | <----------------------------------> | Dart Agent | -| (probe) | ws://127.0.0.1:48686 | (ProbeAgent) | +| (probe) | ws://127.0.0.1:48686 + ping/pong | (ProbeAgent) | +------------------+ +------------------+ | | | | | - Parser | | - Server | @@ -35,8 +35,8 @@ The CLI is the orchestrator. It parses `.probe` test files, manages device conne | `parser/` | Indent-aware lexer + recursive-descent parser producing AST | | `runner/` | Test orchestration: loads files, walks AST, dispatches to ProbeLink | | `probelink/` | JSON-RPC 2.0 WebSocket client | -| `device/` | ADB integration for Android (port forwarding, permissions, token extraction) | -| `ios/` | iOS simulator management via `xcrun simctl` | +| `device/` | ADB integration for Android, device detection (physical vs emulator), iproxy management | +| `ios/` | iOS simulator management via `xcrun simctl`, physical device management via `xcrun devicectl` | | `config/` | `probe.yaml` parsing with layered resolution (CLI flag > config > default) | | `report/` | HTML report generation with portable relative paths | | `cloud/` | Cloud provider integration (BrowserStack, SauceLabs, AWS, Firebase, LambdaTest) | @@ -75,6 +75,17 @@ The agent runs **inside the production Flutter app** using `WidgetsFlutterBindin 3. CLI connects: `ws://127.0.0.1:/probe?token=` 4. Fallback: parse token from `simctl spawn ... log show` +### iOS Physical Device + +1. CLI detects physical device (UDID not in `simctl list`) +2. CLI kills stale `iproxy` processes for this UDID +3. CLI starts `iproxy --udid ` +4. CLI reads token from `idevicesyslog -u --match PROBE_TOKEN` +5. CLI connects: `ws://127.0.0.1:/probe?token=` +6. WebSocket ping/pong keepalive (5s) prevents idle drops +7. Auto-reconnect on connection loss (up to 2 retries per step) +8. App lifecycle managed via `xcrun devicectl` (launch/terminate) + ### Cloud (Relay Mode) 1. CLI creates relay session (gets URL + token) diff --git a/docs/wiki/Home.md b/docs/wiki/Home.md index 5ddd042..97bc97b 100644 --- a/docs/wiki/Home.md +++ b/docs/wiki/Home.md @@ -17,7 +17,7 @@ Welcome to the FlutterProbe wiki. This documentation covers architecture details ## Project Status -FlutterProbe is in active development. Current version: **0.4.2**. +FlutterProbe is in active development. Current version: **0.5.0**. ### Repository Structure diff --git a/docs/wiki/Troubleshooting.md b/docs/wiki/Troubleshooting.md index be8f49a..c026fe7 100644 --- a/docs/wiki/Troubleshooting.md +++ b/docs/wiki/Troubleshooting.md @@ -77,6 +77,27 @@ if (!probeEnabled) { 3. Read the token: `cat /tmp/probe/token` 4. Ensure `PROBE_AGENT=true` was passed during build +### Physical iOS Device — WebSocket Drops + +**Cause**: iproxy connections on physical devices can be closed by iOS during idle periods. + +**Fix**: FlutterProbe v0.5.0+ includes automatic ping/pong keepalive (every 5s) and auto-reconnect (up to 2 retries). If you still see drops: +1. Check iproxy is running: `pgrep -fl iproxy` +2. Restart iproxy: `pkill -f "iproxy.*" && iproxy 48686 48686 --udid &` +3. Use `--reconnect-delay 5s` for slower devices + +### Physical iOS Device — "devicectl launch: not installed" + +**Cause**: Bundle ID mismatch between `probe.yaml` and the installed app. Debug builds may use a `.dev` suffix. + +**Fix**: Check the installed bundle ID: `xcrun devicectl device info apps --device | grep ` and update `project.app` in `probe.yaml` to match. + +### Physical iOS Device — `tap #key` Not Working + +**Cause**: Widgets matched by `Semantics(identifier:)` may not forward gestures properly if pointer IDs collide with real touches. + +**Fix**: Update to FlutterProbe v0.5.0+ which uses unique pointer IDs starting at 900 to avoid collisions. + ## Android-Specific Issues ### Permission Dialogs Block Tests diff --git a/docs/wiki/iOS-Integration-Guide.md b/docs/wiki/iOS-Integration-Guide.md index d0e756b..cfc55de 100644 --- a/docs/wiki/iOS-Integration-Guide.md +++ b/docs/wiki/iOS-Integration-Guide.md @@ -117,6 +117,65 @@ DART_DEFINES=$(echo 'PROBE_AGENT=true' | base64) xcodebuild ... GCC_PREPROCESSOR_DEFINITIONS='$(inherited)' DART_DEFINES="$DART_DEFINES" ``` +## Physical iOS Devices + +Physical device testing requires additional tools and has some limitations compared to simulator testing. + +### Prerequisites + +```bash +# Install libimobiledevice for device communication +brew install libimobiledevice + +# Verify device is detected +idevice_id -l +``` + +### Build and Run + +Physical iOS devices require **debug** or **profile** mode (release mode blocks ProbeAgent by default): + +```bash +# Debug mode (USB-connected) +flutter run --debug --dart-define=PROBE_AGENT=true \ + --device-id + +# Profile mode (better performance, no debug overhead) +flutter run --profile --dart-define=PROBE_AGENT=true \ + --device-id +``` + +### How It Works + +1. **Port forwarding**: FlutterProbe automatically starts `iproxy` to forward port 48686 from the host to the device +2. **Token reading**: The CLI reads the agent token from `idevicesyslog` (instead of simctl file paths) +3. **App lifecycle**: `xcrun devicectl` handles launch/terminate (instead of `simctl`) +4. **Keepalive**: WebSocket ping/pong frames every 5s prevent idle connection drops +5. **Auto-reconnect**: If the connection drops mid-test, the CLI reconnects transparently (up to 2 retries) + +### Limitations on Physical Devices + +| Operation | Physical iOS | Notes | +|---|---|---| +| `tap`, `type`, `see`, etc. | Works | Via agent RPC | +| `restart the app` | Works | Via `xcrun devicectl` | +| `take screenshot` | Works | Via agent RPC | +| `clear app data` | Skipped | No filesystem access; uninstall/reinstall as workaround | +| `allow/deny permission` | Skipped | Requires MDM or manual action | +| `set location` | Skipped | No CLI tool for GPS mocking on real hardware | + +Skipped operations print a warning and continue — they do not fail the test. + +### Troubleshooting Physical Devices + +**"iproxy not found"**: Install libimobiledevice: `brew install libimobiledevice` + +**"token not found within 30s"**: Ensure the app is running with `--dart-define=PROBE_AGENT=true`. Check `idevicesyslog -u | grep PROBE_TOKEN` to verify the agent is printing tokens. + +**Stale iproxy processes**: FlutterProbe automatically kills stale iproxy processes matching your device UDID. If you still have port conflicts, run: `pkill -f "iproxy.*"` + +**White/grey screen on launch**: Profile builds may fail if Firebase or other native SDKs aren't configured for the device. Try debug mode instead. + ## Common probe.yaml for iOS ```yaml diff --git a/internal/cli/cli_test.go b/internal/cli/cli_test.go new file mode 100644 index 0000000..94cd246 --- /dev/null +++ b/internal/cli/cli_test.go @@ -0,0 +1,47 @@ +package cli + +import ( + "bytes" + "testing" +) + +func TestStatusOK(t *testing.T) { + var buf bytes.Buffer + statusOK(&buf, "Connected to %s", "device-1") + out := buf.String() + if out == "" { + t.Error("statusOK produced no output") + } + if !bytes.Contains([]byte(out), []byte("Connected to device-1")) { + t.Errorf("statusOK output missing message: %q", out) + } +} + +func TestStatusWarn(t *testing.T) { + var buf bytes.Buffer + statusWarn(&buf, "timeout exceeded") + out := buf.String() + if out == "" { + t.Error("statusWarn produced no output") + } + if !bytes.Contains([]byte(out), []byte("timeout exceeded")) { + t.Errorf("statusWarn output missing message: %q", out) + } +} + +func TestMessageConstants(t *testing.T) { + // Verify key message constants are non-empty + messages := []struct { + name string + val string + }{ + {"msgNoProbeFiles", msgNoProbeFiles}, + {"msgWaitingForTokenIOS", msgWaitingForTokenIOS}, + {"msgWaitingForToken", msgWaitingForToken}, + } + for _, m := range messages { + if m.val == "" { + t.Errorf("%s is empty", m.name) + } + } +} diff --git a/internal/cli/test.go b/internal/cli/test.go index 6588781..b4acf03 100644 --- a/internal/cli/test.go +++ b/internal/cli/test.go @@ -667,19 +667,31 @@ func runTests(cmd *cobra.Command, args []string) error { } if platform == device.PlatformIOS { - // iOS: grant privacy permissions and relaunch the app. - // simctl privacy grant terminates the running app, so we must - // grant first, then relaunch. This prevents native OS dialogs - // (camera, location, etc.) from blocking the Flutter UI. - if autoYes && cfg.Project.App != "" { - for _, svc := range []string{"camera", "microphone", "location", "photos", "contacts-limited", "calendar"} { - _ = dm.SimCtl().GrantPrivacy(ctx, deviceSerial, cfg.Project.App, svc) + // Detect physical vs simulator early for connection setup + isPhysicalIOS := dm.IsPhysicalIOS(ctx, deviceSerial) + + if isPhysicalIOS { + // Physical iOS: set up iproxy for port forwarding + cleanupIProxy, iproxyErr := dm.EnsureIProxy(ctx, deviceSerial, cfg.Agent.Port, cfg.Agent.AgentDevicePort()) + if iproxyErr != nil { + return fmt.Errorf("iproxy setup: %w", iproxyErr) + } + defer cleanupIProxy() + } else { + // iOS simulator: grant privacy permissions and relaunch the app. + // simctl privacy grant terminates the running app, so we must + // grant first, then relaunch. This prevents native OS dialogs + // (camera, location, etc.) from blocking the Flutter UI. + if autoYes && cfg.Project.App != "" { + for _, svc := range []string{"camera", "microphone", "location", "photos", "contacts-limited", "calendar"} { + _ = dm.SimCtl().GrantPrivacy(ctx, deviceSerial, cfg.Project.App, svc) + } + // Relaunch the app — simctl privacy grant terminates it + _ = dm.SimCtl().Launch(ctx, deviceSerial, cfg.Project.App) + time.Sleep(3 * time.Second) // give agent time to start } - // Relaunch the app — simctl privacy grant terminates it - _ = dm.SimCtl().Launch(ctx, deviceSerial, cfg.Project.App) - time.Sleep(3 * time.Second) // give agent time to start } - // iOS: simulators share host loopback — no port forwarding needed + // Simulators share host loopback; physical uses iproxy — both on 127.0.0.1 fmt.Fprintln(statusW, msgWaitingForTokenIOS) token, err := dm.ReadTokenIOS(ctx, deviceSerial, cfg.Agent.TokenReadTimeout, cfg.Project.App) if err != nil { @@ -692,6 +704,12 @@ func runTests(cmd *cobra.Command, args []string) error { } defer client.Close() } else { + // Android: verify ADB is available and device is reachable, + // clean up stale port forwards + if err := dm.EnsureADB(ctx, deviceSerial, cfg.Agent.Port); err != nil { + return fmt.Errorf("android setup: %w", err) + } + // Android: grant permissions BEFORE reading token when -y is used. // This prevents OS permission dialogs from blocking the app on // first launch (especially POST_NOTIFICATIONS on Android 13+). @@ -813,6 +831,17 @@ func runTests(cmd *cobra.Command, args []string) error { // Only available for local devices — cloud mode has no ADB/simctl access. var devCtx *runner.DeviceContext if !dryRun && client != nil && dm != nil { + // Detect if this is a physical device (vs emulator/simulator) + isPhysical := false + if platform == device.PlatformIOS { + isPhysical = dm.IsPhysicalIOS(ctx, deviceSerial) + } else if platform == device.PlatformAndroid { + isPhysical = dm.IsPhysicalAndroid(ctx, deviceSerial) + } + if isPhysical { + fmt.Fprintf(statusW, " \033[36mℹ\033[0m Physical device detected: %s\n", deviceSerial) + } + devCtx = &runner.DeviceContext{ Manager: dm, Serial: deviceSerial, @@ -820,6 +849,7 @@ func runTests(cmd *cobra.Command, args []string) error { AppID: cfg.Project.App, Port: cfg.Agent.Port, DevicePort: cfg.Agent.AgentDevicePort(), + IsPhysical: isPhysical, AllowClearData: autoYes, Confirm: promptUserConfirm, GrantPermissionsOnClear: autoYes || cfg.Defaults.GrantPermissionsOnClear, diff --git a/internal/device/manager.go b/internal/device/manager.go index a9769e2..814d412 100644 --- a/internal/device/manager.go +++ b/internal/device/manager.go @@ -1,6 +1,7 @@ package device import ( + "bufio" "context" "fmt" "os/exec" @@ -36,31 +37,33 @@ type ToolPaths struct { // Manager handles device discovery and lifecycle. type Manager struct { - adb *ADB - simctl *ios.SimCtl - tools ToolPaths + adb *ADB + simctl *ios.SimCtl + devicectl *ios.DeviceCtl + tools ToolPaths } // NewManager creates a Manager using tools found in PATH. func NewManager() *Manager { - return &Manager{adb: NewADB(), simctl: ios.New()} + return &Manager{adb: NewADB(), simctl: ios.New(), devicectl: ios.NewDeviceCtl()} } // NewManagerWithPaths creates a Manager with configurable tool paths. // This is useful for CI/CD environments or when tools are not in PATH. func NewManagerWithPaths(paths ToolPaths) *Manager { return &Manager{ - adb: NewADBWithPath(paths.ADB), - simctl: ios.New(), - tools: paths, + adb: NewADBWithPath(paths.ADB), + simctl: ios.New(), + devicectl: ios.NewDeviceCtl(), + tools: paths, } } -// List returns all connected Android emulators/devices and iOS simulators. +// List returns all connected Android devices/emulators, iOS simulators, and physical iOS devices. func (m *Manager) List(ctx context.Context) ([]Device, error) { var all []Device - // Android devices + // Android devices (both emulators and physical) androids, err := m.adb.Devices(ctx) if err == nil { all = append(all, androids...) @@ -80,9 +83,162 @@ func (m *Manager) List(ctx context.Context) ([]Device, error) { } } + // Physical iOS devices (via libimobiledevice) + physicalUDIDs, err := ios.ListPhysicalDevices(ctx) + if err == nil { + simUDIDs := make(map[string]bool) + if sims != nil { + for _, s := range sims { + simUDIDs[s.UDID] = true + } + } + for _, udid := range physicalUDIDs { + if !simUDIDs[udid] { // avoid duplicates if somehow listed in both + all = append(all, Device{ + ID: udid, + Name: "Physical iOS Device", + Platform: PlatformIOS, + State: "online", + }) + } + } + } + return all, nil } +// DeviceCtl returns the DeviceCtl instance for physical iOS device operations. +func (m *Manager) DeviceCtl() *ios.DeviceCtl { + return m.devicectl +} + +// IsPhysicalIOS returns true if the given UDID is a physical iOS device +// (not found in the simulator list). +func (m *Manager) IsPhysicalIOS(ctx context.Context, udid string) bool { + sims, err := m.simctl.List(ctx) + if err != nil { + return true // assume physical if simctl fails + } + for _, s := range sims { + if s.UDID == udid { + return false + } + } + return true +} + +// IsPhysicalAndroid returns true if the given serial is a physical Android device +// (not an emulator). +func (m *Manager) IsPhysicalAndroid(ctx context.Context, serial string) bool { + if strings.HasPrefix(serial, "emulator-") { + return false + } + out, err := m.adb.Shell(ctx, serial, "getprop", "ro.hardware") + if err != nil { + return !strings.HasPrefix(serial, "emulator-") + } + hw := strings.TrimSpace(string(out)) + return hw != "ranchu" && hw != "goldfish" +} + +// EnsureADB checks that the ADB binary is available and can communicate with +// the specified device. It also cleans up any stale port forwards for the given port. +func (m *Manager) EnsureADB(ctx context.Context, serial string, hostPort int) error { + // Check ADB is installed + if _, err := exec.LookPath(m.adb.Bin()); err != nil { + return fmt.Errorf("adb not found at %q — install Android SDK platform-tools or set tools.adb in probe.yaml", m.adb.Bin()) + } + + // Verify the device is reachable + devices, err := m.adb.Devices(ctx) + if err != nil { + return fmt.Errorf("adb devices: %w", err) + } + found := false + for _, d := range devices { + if d.ID == serial && d.State == "device" { + found = true + break + } + } + if !found { + return fmt.Errorf("adb: device %s not found or not online — check USB connection", serial) + } + + // Clean up stale port forwards for this port to avoid conflicts + out, err := m.adb.Run(ctx, serial, "forward", "--list") + if err == nil { + rule := fmt.Sprintf("tcp:%d", hostPort) + for _, line := range strings.Split(string(out), "\n") { + if strings.Contains(line, serial) && strings.Contains(line, rule) { + _ = m.adb.RemoveForward(ctx, serial, hostPort) + break + } + } + } + + return nil +} + +// EnsureIProxy checks that iproxy is installed, kills any stale iproxy processes +// for the given UDID, and starts a fresh iproxy forwarding hostPort to devicePort. +// Returns a cleanup function that kills the iproxy process. +func (m *Manager) EnsureIProxy(ctx context.Context, udid string, hostPort, devicePort int) (cleanup func(), err error) { + // Check iproxy is installed + if _, lookErr := exec.LookPath("iproxy"); lookErr != nil { + return nil, fmt.Errorf("iproxy not found — install via: brew install libimobiledevice") + } + + // Kill stale iproxy processes for this UDID + KillStaleIProxy(udid) + + // Start a fresh iproxy + cmd := exec.CommandContext(ctx, "iproxy", + fmt.Sprintf("%d", hostPort), + fmt.Sprintf("%d", devicePort), + "--udid", udid, + ) + if err := cmd.Start(); err != nil { + return nil, fmt.Errorf("iproxy start: %w", err) + } + + // Give iproxy time to bind the port + time.Sleep(1 * time.Second) + + cleanup = func() { + if cmd.Process != nil { + cmd.Process.Kill() + } + } + return cleanup, nil +} + +// KillStaleIProxy kills all iproxy processes that match the given UDID. +// This prevents port conflicts from leftover processes after flutter run crashes. +func KillStaleIProxy(udid string) { + // Find all iproxy PIDs + out, err := exec.Command("pgrep", "-f", "iproxy").Output() + if err != nil { + return // no iproxy processes + } + for _, line := range strings.Split(strings.TrimSpace(string(out)), "\n") { + pid := strings.TrimSpace(line) + if pid == "" { + continue + } + // Check if this PID's command line contains our UDID + cmdline, err := exec.Command("ps", "-p", pid, "-o", "args=").Output() + if err != nil { + continue + } + if strings.Contains(string(cmdline), udid) { + exec.Command("kill", pid).Run() + } + } + // Brief pause to let ports be released + time.Sleep(500 * time.Millisecond) +} + // Start boots an Android emulator identified by avdName. // bootTimeout and pollInterval control startup behavior (0 = use defaults). func (m *Manager) Start(ctx context.Context, avdName string, bootTimeout, pollInterval time.Duration) (*Device, error) { @@ -215,10 +371,47 @@ func (m *Manager) RemoveForward(ctx context.Context, serial string, hostPort int return m.adb.RemoveForward(ctx, serial, hostPort) } -// ReadTokenIOS reads the ProbeAgent token from the iOS simulator. +// ReadTokenIOS reads the ProbeAgent token from an iOS device. +// For simulators, reads the token file via simctl. +// For physical devices, reads from idevicesyslog. // bundleID is optional — if provided, it checks the app container's token file first. func (m *Manager) ReadTokenIOS(ctx context.Context, udid string, timeout time.Duration, bundleID ...string) (string, error) { - return m.simctl.ReadToken(ctx, udid, timeout, bundleID...) + // Try simulator path first (fast, file-based) + token, err := m.simctl.ReadToken(ctx, udid, timeout, bundleID...) + if err == nil { + return token, nil + } + + // Fallback: physical device via idevicesyslog + return m.readTokenPhysicalIOS(ctx, udid, timeout) +} + +// readTokenPhysicalIOS reads the token from a physical iOS device using idevicesyslog. +func (m *Manager) readTokenPhysicalIOS(ctx context.Context, udid string, timeout time.Duration) (string, error) { + tCtx, cancel := context.WithTimeout(ctx, timeout) + defer cancel() + + cmd := exec.CommandContext(tCtx, "idevicesyslog", "-u", udid, "--match", "PROBE_TOKEN") + stdout, err := cmd.StdoutPipe() + if err != nil { + return "", fmt.Errorf("ios physical: syslog pipe: %w", err) + } + if err := cmd.Start(); err != nil { + return "", fmt.Errorf("ios physical: idevicesyslog not found — install via: brew install libimobiledevice") + } + defer cmd.Process.Kill() + + scanner := bufio.NewScanner(stdout) + for scanner.Scan() { + line := scanner.Text() + if idx := strings.Index(line, "PROBE_TOKEN="); idx >= 0 { + token := strings.TrimSpace(line[idx+len("PROBE_TOKEN="):]) + if len(token) >= 16 { + return token, nil + } + } + } + return "", fmt.Errorf("ios physical: token not found within %s — is the app running with probe_agent?", timeout) } // ReadToken reads the ProbeAgent token from the Android device. diff --git a/internal/device/manager_test.go b/internal/device/manager_test.go new file mode 100644 index 0000000..6233f3e --- /dev/null +++ b/internal/device/manager_test.go @@ -0,0 +1,72 @@ +package device_test + +import ( + "context" + "testing" + + "github.com/alphawavesystems/flutter-probe/internal/device" +) + +func TestNewManager(t *testing.T) { + m := device.NewManager() + if m == nil { + t.Fatal("NewManager() returned nil") + } +} + +func TestNewManagerWithPaths(t *testing.T) { + m := device.NewManagerWithPaths(device.ToolPaths{ADB: "adb"}) + if m == nil { + t.Fatal("NewManagerWithPaths() returned nil") + } +} + +func TestNewManagerWithPaths_EmptyADB(t *testing.T) { + // Empty ADB path should default to "adb" from PATH + m := device.NewManagerWithPaths(device.ToolPaths{}) + if m == nil { + t.Fatal("NewManagerWithPaths() returned nil") + } +} + +func TestKillStaleIProxy_NoProcesses(t *testing.T) { + // Should not panic when there are no matching processes + device.KillStaleIProxy("nonexistent-udid-12345") +} + +func TestEnsureADB_InvalidBinary(t *testing.T) { + m := device.NewManagerWithPaths(device.ToolPaths{ADB: "/nonexistent/adb"}) + err := m.EnsureADB(context.Background(), "fake-serial", 48686) + if err == nil { + t.Error("EnsureADB with invalid binary should return error") + } +} + +func TestIsPhysicalAndroid_EmulatorSerial(t *testing.T) { + m := device.NewManager() + // Emulator serials always start with "emulator-" + if m.IsPhysicalAndroid(context.Background(), "emulator-5554") { + t.Error("emulator-5554 should not be detected as physical") + } +} + +func TestADB_Bin(t *testing.T) { + adb := device.NewADB() + if adb.Bin() != "adb" { + t.Errorf("NewADB().Bin() = %q, want %q", adb.Bin(), "adb") + } +} + +func TestADB_BinWithPath(t *testing.T) { + adb := device.NewADBWithPath("/custom/path/adb") + if adb.Bin() != "/custom/path/adb" { + t.Errorf("NewADBWithPath().Bin() = %q, want %q", adb.Bin(), "/custom/path/adb") + } +} + +func TestADB_BinWithPathEmpty(t *testing.T) { + adb := device.NewADBWithPath("") + if adb.Bin() != "adb" { + t.Errorf("NewADBWithPath(\"\").Bin() = %q, want %q", adb.Bin(), "adb") + } +} diff --git a/internal/ios/devicectl.go b/internal/ios/devicectl.go new file mode 100644 index 0000000..b51aa1c --- /dev/null +++ b/internal/ios/devicectl.go @@ -0,0 +1,119 @@ +// Package ios provides wrappers for Apple device management tools. +// DeviceCtl wraps `xcrun devicectl` for physical iOS device operations. +// SimCtl (in simctl.go) wraps `xcrun simctl` for simulator operations. + +package ios + +import ( + "context" + "encoding/json" + "fmt" + "os" + "os/exec" + "path/filepath" + "strconv" + "strings" +) + +// DeviceCtl wraps `xcrun devicectl` for managing physical iOS devices. +// Requires Xcode 15+ and a USB-connected device. +type DeviceCtl struct{} + +// NewDeviceCtl creates a DeviceCtl instance. +func NewDeviceCtl() *DeviceCtl { + return &DeviceCtl{} +} + +// Launch starts an app on a physical device by bundle ID. +func (d *DeviceCtl) Launch(ctx context.Context, udid, bundleID string) error { + cmd := exec.CommandContext(ctx, "xcrun", "devicectl", "device", "process", "launch", + "--device", udid, bundleID) + out, err := cmd.CombinedOutput() + if err != nil { + return fmt.Errorf("devicectl launch: %s: %w", strings.TrimSpace(string(out)), err) + } + return nil +} + +// Terminate stops an app on a physical device by resolving its PID first. +// Returns nil if the app is not running. +func (d *DeviceCtl) Terminate(ctx context.Context, udid, bundleID string) error { + pid, err := d.FindPID(ctx, udid, bundleID) + if err != nil || pid == 0 { + return nil // not running + } + cmd := exec.CommandContext(ctx, "xcrun", "devicectl", "device", "process", "terminate", + "--device", udid, "--pid", strconv.Itoa(pid)) + out, err := cmd.CombinedOutput() + if err != nil { + return fmt.Errorf("devicectl terminate (pid %d): %s: %w", pid, strings.TrimSpace(string(out)), err) + } + return nil +} + +// Install installs an .app bundle on a physical device. +func (d *DeviceCtl) Install(ctx context.Context, udid, appPath string) error { + cmd := exec.CommandContext(ctx, "xcrun", "devicectl", "device", "install", "app", + "--device", udid, appPath) + out, err := cmd.CombinedOutput() + if err != nil { + return fmt.Errorf("devicectl install: %s: %w", strings.TrimSpace(string(out)), err) + } + return nil +} + +// FindPID looks up the process ID of a running app by its bundle ID. +// Returns 0 if the app is not running. +func (d *DeviceCtl) FindPID(ctx context.Context, udid, bundleID string) (int, error) { + tmpFile := filepath.Join(os.TempDir(), fmt.Sprintf("devicectl-procs-%s.json", udid[:8])) + defer os.Remove(tmpFile) + + cmd := exec.CommandContext(ctx, "xcrun", "devicectl", "device", "info", "processes", + "--device", udid, "--json-output", tmpFile) + if err := cmd.Run(); err != nil { + return 0, fmt.Errorf("devicectl info processes: %w", err) + } + + data, err := os.ReadFile(tmpFile) + if err != nil { + return 0, fmt.Errorf("reading process list: %w", err) + } + + // Parse the JSON output to find matching bundle ID + var result struct { + Result struct { + RunningProcesses []struct { + BundleIdentifier string `json:"bundleIdentifier"` + ProcessIdentifier int `json:"processIdentifier"` + Executable string `json:"executable"` + } `json:"runningProcesses"` + } `json:"result"` + } + if err := json.Unmarshal(data, &result); err != nil { + return 0, fmt.Errorf("parsing process list: %w", err) + } + + for _, p := range result.Result.RunningProcesses { + if p.BundleIdentifier == bundleID { + return p.ProcessIdentifier, nil + } + } + + return 0, nil // not found = not running +} + +// ListDevices returns UDIDs of connected physical iOS devices via idevice_id. +func ListPhysicalDevices(ctx context.Context) ([]string, error) { + out, err := exec.CommandContext(ctx, "idevice_id", "-l").Output() + if err != nil { + return nil, err + } + var udids []string + for _, line := range strings.Split(string(out), "\n") { + udid := strings.TrimSpace(line) + if udid != "" { + udids = append(udids, udid) + } + } + return udids, nil +} diff --git a/internal/ios/devicectl_test.go b/internal/ios/devicectl_test.go new file mode 100644 index 0000000..a7723f9 --- /dev/null +++ b/internal/ios/devicectl_test.go @@ -0,0 +1,25 @@ +package ios + +import ( + "testing" +) + +func TestNewDeviceCtl(t *testing.T) { + d := NewDeviceCtl() + if d == nil { + t.Fatal("NewDeviceCtl() returned nil") + } +} + +func TestListPhysicalDevices_NoDevices(t *testing.T) { + // ListPhysicalDevices should not panic even if no devices are connected. + // It may return an error if idevice_id is not installed, which is acceptable. + _, _ = ListPhysicalDevices(t.Context()) +} + +func TestNewSimCtl(t *testing.T) { + s := New() + if s == nil { + t.Fatal("New() returned nil") + } +} diff --git a/internal/probelink/client.go b/internal/probelink/client.go index 19db92e..d85f130 100644 --- a/internal/probelink/client.go +++ b/internal/probelink/client.go @@ -43,6 +43,8 @@ type Client struct { pending map[uint64]chan Response token string addr string + done chan struct{} // signals ping loop to stop + closed bool OnNotify func(method string, params json.RawMessage) } @@ -93,9 +95,19 @@ func DialWithOptions(ctx context.Context, opts DialOptions) (*Client, error) { pending: make(map[uint64]chan Response), token: opts.Token, addr: safeAddr, + done: make(chan struct{}), } + // Set pong handler to extend read deadline on every pong received + conn.SetPongHandler(func(string) error { + conn.SetReadDeadline(time.Now().Add(3 * defaultPingInterval)) + return nil + }) + // Set initial read deadline (will be refreshed by pong responses) + conn.SetReadDeadline(time.Now().Add(3 * defaultPingInterval)) + go c.readLoop() + go c.pingLoop() return c, nil } @@ -144,17 +156,63 @@ func DialRelay(ctx context.Context, relayURL, cliToken string, timeout time.Dura conn: conn, pending: make(map[uint64]chan Response), addr: safeAddr, + done: make(chan struct{}), } + conn.SetPongHandler(func(string) error { + conn.SetReadDeadline(time.Now().Add(3 * defaultPingInterval)) + return nil + }) + conn.SetReadDeadline(time.Now().Add(3 * defaultPingInterval)) + go c.readLoop() + go c.pingLoop() return c, nil } -// Close terminates the connection. +// Close terminates the connection and stops the keepalive loop. func (c *Client) Close() error { + c.mu.Lock() + if !c.closed { + c.closed = true + close(c.done) + } + c.mu.Unlock() return c.conn.Close() } +// Connected returns true if the client has not been closed. +func (c *Client) Connected() bool { + c.mu.Lock() + defer c.mu.Unlock() + return !c.closed +} + +// pingLoop sends WebSocket ping frames at regular intervals to keep the +// connection alive. This is critical for physical device connections via +// iproxy where idle TCP connections are aggressively closed by iOS. +func (c *Client) pingLoop() { + ticker := time.NewTicker(defaultPingInterval) + defer ticker.Stop() + for { + select { + case <-c.done: + return + case <-ticker.C: + c.mu.Lock() + err := c.conn.WriteControl( + websocket.PingMessage, + []byte{}, + time.Now().Add(2*time.Second), + ) + c.mu.Unlock() + if err != nil { + return + } + } + } +} + // Call sends a JSON-RPC request and waits for the response. func (c *Client) Call(ctx context.Context, method string, params any) (json.RawMessage, error) { req, err := NewRequest(method, params) @@ -209,10 +267,20 @@ func (c *Client) WaitSettled(ctx context.Context, timeout time.Duration) error { // readLoop dispatches incoming JSON-RPC messages to pending callers. func (c *Client) readLoop() { for { + // Extend read deadline before each read — pong handler also resets it, + // but this ensures we don't timeout during long-running RPC calls + // (e.g., wait 10 seconds). We set a generous deadline here; + // the ping/pong mechanism handles actual liveness detection. + c.conn.SetReadDeadline(time.Now().Add(5 * time.Minute)) + _, msg, err := c.conn.ReadMessage() if err != nil { // Connection closed — drain all pending with error c.mu.Lock() + if !c.closed { + c.closed = true + close(c.done) + } for id, ch := range c.pending { ch <- Response{ID: id, Error: &RPCError{Code: -32000, Message: err.Error()}} delete(c.pending, id) diff --git a/internal/runner/device_context.go b/internal/runner/device_context.go index c2841f5..cda153f 100644 --- a/internal/runner/device_context.go +++ b/internal/runner/device_context.go @@ -26,6 +26,7 @@ type DeviceContext struct { AppID string // bundle ID / package name Port int // host-side agent port (default 48686) DevicePort int // on-device agent port (default: same as Port) + IsPhysical bool // true for physical devices (vs emulator/simulator) AllowClearData bool // if true, skip confirmation for clear app data (CI/CD mode) Confirm ConfirmFunc // interactive confirmation callback (nil = deny destructive ops unless AllowClearData) GrantPermissionsOnClear bool // if true, auto-grant all permissions after clearing data @@ -90,12 +91,22 @@ func (dc *DeviceContext) RestartApp(ctx context.Context) error { } case device.PlatformIOS: - simctl := dc.Manager.SimCtl() - _ = simctl.Terminate(ctx, dc.Serial, dc.AppID) // ignore if not running - time.Sleep(dc.restartDelay()) - fmt.Printf(" \033[33m↻\033[0m Relaunching %s...\n", dc.AppID) - if err := simctl.Launch(ctx, dc.Serial, dc.AppID); err != nil { - return fmt.Errorf("restart: launch: %w", err) + if dc.IsPhysical { + dctl := dc.Manager.DeviceCtl() + _ = dctl.Terminate(ctx, dc.Serial, dc.AppID) + time.Sleep(dc.restartDelay()) + fmt.Printf(" \033[33m↻\033[0m Relaunching %s...\n", dc.AppID) + if err := dctl.Launch(ctx, dc.Serial, dc.AppID); err != nil { + return fmt.Errorf("restart: launch: %w", err) + } + } else { + simctl := dc.Manager.SimCtl() + _ = simctl.Terminate(ctx, dc.Serial, dc.AppID) + time.Sleep(dc.restartDelay()) + fmt.Printf(" \033[33m↻\033[0m Relaunching %s...\n", dc.AppID) + if err := simctl.Launch(ctx, dc.Serial, dc.AppID); err != nil { + return fmt.Errorf("restart: launch: %w", err) + } } } return nil @@ -141,17 +152,20 @@ func (dc *DeviceContext) ClearAppData(ctx context.Context) error { } case device.PlatformIOS: + if dc.IsPhysical { + fmt.Println(" \033[33m⚠\033[0m clear app data is not supported on physical iOS devices") + fmt.Println(" Workaround: uninstall and reinstall the app manually") + return nil + } + simctl := dc.Manager.SimCtl() _ = simctl.Terminate(ctx, dc.Serial, dc.AppID) - // Get the app data container path via simctl (official Apple tooling) dataPath := simctl.AppDataPath(ctx, dc.Serial, dc.AppID) if err := dc.validateIOSDataPath(dataPath); err != nil { return fmt.Errorf("clear data: %w", err) } - // Clear data container contents (not the container dir itself) - // This is safer than rm -rf on the container — we only remove the contents if dataPath != "" { for _, subdir := range []string{"Documents", "Library", "tmp"} { target := dataPath + "/" + subdir @@ -160,13 +174,10 @@ func (dc *DeviceContext) ClearAppData(ctx context.Context) error { fmt.Printf(" \033[32m✓\033[0m Cleared data container: %s\n", dataPath) } - // Reset the simulator keychain to clear stored credentials (passwords, - // tokens, etc.) that persist outside the app's data container. if err := simctl.KeychainReset(ctx, dc.Serial); err != nil { fmt.Printf(" \033[33m⚠\033[0m keychain reset: %v\n", err) } - // Auto-grant permissions before relaunch to prevent OS permission dialogs if dc.GrantPermissionsOnClear { if err := dc.GrantAllPermissions(ctx); err != nil { fmt.Printf(" \033[33m⚠\033[0m auto-grant permissions: %v\n", err) @@ -228,6 +239,10 @@ func (dc *DeviceContext) AllowPermission(ctx context.Context, name string) error } } case device.PlatformIOS: + if dc.IsPhysical { + fmt.Printf(" \033[33m⚠\033[0m permission management is not supported on physical iOS devices — skipping\n") + return nil + } svc, err := device.ResolveIOSService(name) if err != nil { return err @@ -253,6 +268,10 @@ func (dc *DeviceContext) DenyPermission(ctx context.Context, name string) error } } case device.PlatformIOS: + if dc.IsPhysical { + fmt.Printf(" \033[33m⚠\033[0m permission management is not supported on physical iOS devices — skipping\n") + return nil + } svc, err := device.ResolveIOSService(name) if err != nil { return err @@ -271,11 +290,14 @@ func (dc *DeviceContext) GrantAllPermissions(ctx context.Context) error { case device.PlatformAndroid: for _, perms := range device.AndroidPermissions { for _, perm := range perms { - // Best-effort: some permissions may not apply to this API level _ = dc.Manager.ADB().GrantPermission(ctx, dc.Serial, dc.AppID, perm) } } case device.PlatformIOS: + if dc.IsPhysical { + fmt.Printf(" \033[33m⚠\033[0m permission management is not supported on physical iOS devices — skipping\n") + return nil + } for _, svc := range device.IOSPrivacyServices { _ = dc.Manager.SimCtl().GrantPrivacy(ctx, dc.Serial, dc.AppID, svc) } @@ -293,6 +315,10 @@ func (dc *DeviceContext) RevokeAllPermissions(ctx context.Context) error { } } case device.PlatformIOS: + if dc.IsPhysical { + fmt.Printf(" \033[33m⚠\033[0m permission management is not supported on physical iOS devices — skipping\n") + return nil + } _ = dc.Manager.SimCtl().ResetPrivacy(ctx, dc.Serial, dc.AppID) } return nil @@ -319,11 +345,16 @@ func (dc *DeviceContext) Reconnect(ctx context.Context) (*probelink.Client, erro } _ = dc.Manager.ForwardPort(ctx, dc.Serial, dc.Port, devPort) case device.PlatformIOS: - // Delete stale token file so ReadTokenIOS picks up the fresh one - simctl := dc.Manager.SimCtl() - tokenPath := dc.iosTokenPath() - if tokenPath != "" { - _, _ = simctl.Spawn(ctx, dc.Serial, "rm", "-f", tokenPath) + if dc.IsPhysical { + // Physical iOS: no token file to delete — idevicesyslog reads live stream + // Just wait for the app to restart + } else { + // Simulator: delete stale token file so ReadTokenIOS picks up the fresh one + simctl := dc.Manager.SimCtl() + tokenPath := dc.iosTokenPath() + if tokenPath != "" { + _, _ = simctl.Spawn(ctx, dc.Serial, "rm", "-f", tokenPath) + } } } @@ -392,7 +423,11 @@ func (dc *DeviceContext) KillApp(ctx context.Context) error { return fmt.Errorf("kill app: %w", err) } case device.PlatformIOS: - _ = dc.Manager.SimCtl().Terminate(ctx, dc.Serial, dc.AppID) + if dc.IsPhysical { + _ = dc.Manager.DeviceCtl().Terminate(ctx, dc.Serial, dc.AppID) + } else { + _ = dc.Manager.SimCtl().Terminate(ctx, dc.Serial, dc.AppID) + } } return nil } @@ -407,19 +442,29 @@ func (dc *DeviceContext) LaunchApp(ctx context.Context) error { return fmt.Errorf("launch app: %w", err) } case device.PlatformIOS: - if err := dc.Manager.SimCtl().Launch(ctx, dc.Serial, dc.AppID); err != nil { - return fmt.Errorf("launch app: %w", err) + if dc.IsPhysical { + if err := dc.Manager.DeviceCtl().Launch(ctx, dc.Serial, dc.AppID); err != nil { + return fmt.Errorf("launch app: %w", err) + } + } else { + if err := dc.Manager.SimCtl().Launch(ctx, dc.Serial, dc.AppID); err != nil { + return fmt.Errorf("launch app: %w", err) + } } } return nil } // SetLocation sets the device's GPS location. +// Not supported on physical devices — skips with a warning. func (dc *DeviceContext) SetLocation(ctx context.Context, lat, lng string) error { + if dc.IsPhysical { + fmt.Printf(" \033[33m⚠\033[0m set location is not supported on physical devices — skipping\n") + return nil + } fmt.Printf(" \033[36m📍\033[0m Setting location to %s, %s\n", lat, lng) switch dc.Platform { case device.PlatformAndroid: - // adb emu geo fix takes longitude first, then latitude if _, err := dc.Manager.ADB().Shell(ctx, dc.Serial, "emu", "geo", "fix", lng, lat); err != nil { return fmt.Errorf("set location: %w", err) } diff --git a/internal/runner/executor.go b/internal/runner/executor.go index 7758a60..3165639 100644 --- a/internal/runner/executor.go +++ b/internal/runner/executor.go @@ -26,6 +26,7 @@ type Executor struct { depth int // indentation depth for verbose logging artifacts []string // collected screenshot paths (on-device) visual *visual.Comparator // nil if visual regression is not configured + maxReconnectAttempts int // max auto-reconnect attempts per call (default 2) } // NewExecutor creates an Executor. @@ -79,10 +80,15 @@ func (e *Executor) RunBody(ctx context.Context, steps []parser.Step) error { func (e *Executor) runStep(ctx context.Context, step parser.Step) error { // Use a longer timeout for restart/clear — they kill the app and reconnect stepTimeout := e.timeout + isLifecycleAction := false if a, ok := step.(parser.ActionStep); ok { if a.Verb == parser.VerbRestart || a.Verb == parser.VerbClearAppData { stepTimeout = 90 * time.Second } + // Don't auto-reconnect for actions that intentionally close the connection + if a.Verb == parser.VerbRestart || a.Verb == parser.VerbClearAppData || a.Verb == parser.VerbKill { + isLifecycleAction = true + } } stepCtx, cancel := context.WithTimeout(ctx, stepTimeout) defer cancel() @@ -112,6 +118,39 @@ func (e *Executor) runStep(ctx context.Context, step parser.Step) error { err = e.runHTTPCall(stepCtx, s) } + // Auto-reconnect: if the step failed due to a connection error and this + // isn't a lifecycle action (restart/kill/clear), try to reconnect and retry. + if err != nil && !isLifecycleAction && isConnectionError(err) && e.deviceCtx != nil { + maxAttempts := e.maxReconnectAttempts + if maxAttempts == 0 { + maxAttempts = 2 + } + for attempt := 1; attempt <= maxAttempts; attempt++ { + if reconnErr := e.tryReconnect(ctx); reconnErr != nil { + err = fmt.Errorf("%w (auto-reconnect attempt %d failed: %v)", err, attempt, reconnErr) + break + } + // Retry the step with a fresh timeout + retryCtx, retryCancel := context.WithTimeout(ctx, stepTimeout) + switch s := step.(type) { + case parser.ActionStep: + err = e.runAction(retryCtx, s) + case parser.AssertStep: + err = e.runAssert(retryCtx, s) + case parser.WaitStep: + err = e.runWait(retryCtx, s) + case parser.DartBlock: + err = e.runDart(retryCtx, s) + case parser.MockBlock: + err = e.runMock(retryCtx, s) + } + retryCancel() + if err == nil || !isConnectionError(err) { + break // either succeeded or a non-connection error + } + } + } + if e.verbose && desc != "" { elapsed := time.Since(start) indent := strings.Repeat(" ", e.depth) @@ -592,6 +631,48 @@ func (e *Executor) runRecipeCall(ctx context.Context, rc parser.RecipeCall) erro return err } +// ---- Auto-Reconnect ---- + +// isConnectionError returns true if the error indicates a broken WebSocket connection. +func isConnectionError(err error) bool { + if err == nil { + return false + } + msg := err.Error() + return strings.Contains(msg, "probelink: write:") || + strings.Contains(msg, "use of closed network connection") || + strings.Contains(msg, "broken pipe") || + strings.Contains(msg, "connection reset") || + strings.Contains(msg, "EOF") || + strings.Contains(msg, "rpc error -32000") +} + +// tryReconnect attempts to re-establish the WebSocket connection to the agent. +// This is used when the connection drops mid-test (e.g., physical device via iproxy). +// It does NOT restart the app — the app is still running, we just lost the TCP connection. +func (e *Executor) tryReconnect(ctx context.Context) error { + if e.deviceCtx == nil { + return fmt.Errorf("reconnect: no device context (cloud mode)") + } + fmt.Printf(" \033[33m⟳\033[0m WebSocket connection lost — attempting to reconnect...\n") + e.client.Close() + + // The app is still running — we just need to re-dial. + // Give the agent a moment, then reconnect. + time.Sleep(1 * time.Second) + + newClient, err := e.deviceCtx.Reconnect(ctx) + if err != nil { + return fmt.Errorf("auto-reconnect failed: %w", err) + } + e.client = newClient + if e.onReconnect != nil { + e.onReconnect(newClient) + } + fmt.Printf(" \033[32m⟳\033[0m Reconnected successfully\n") + return nil +} + // ---- Helpers ---- // resolve substitutes placeholders with values from the vars map diff --git a/probe_agent/lib/src/agent.dart b/probe_agent/lib/src/agent.dart index 981191c..ad88eab 100644 --- a/probe_agent/lib/src/agent.dart +++ b/probe_agent/lib/src/agent.dart @@ -37,14 +37,62 @@ class ProbeAgent { /// via `--dart-define` (connects out to a relay server). /// - **Local mode** otherwise (listens on localhost for CLI connections). /// - /// No-op on non-debug builds (checked via assert). - static Future start({int port = 48686}) async { - // Safety guard — only meaningful in debug / profile builds. - // In release mode this is a no-op because asserts are disabled. - assert(() { - _startInternal(port); - return true; - }()); + /// Build modes: + /// - **Debug**: works out of the box with `--dart-define=PROBE_AGENT=true` + /// - **Profile**: works out of the box (needed for physical iOS devices) + /// - **Release**: blocked by default. Pass `allowReleaseBuild: true` to + /// override, and `--dart-define=PROBE_AGENT_FORCE=true` to skip the + /// console warning. + /// + /// Requires `--dart-define=PROBE_AGENT=true` at build time. Without it, + /// this is always a no-op. + static Future start({ + int port = 48686, + bool allowReleaseBuild = false, + }) async { + const enabled = bool.fromEnvironment('PROBE_AGENT', defaultValue: false); + if (!enabled) return; + + // Detect build mode + const isRelease = bool.fromEnvironment('dart.vm.product', defaultValue: false); + const isProfile = bool.fromEnvironment('dart.vm.profile', defaultValue: false); + + if (isRelease && !allowReleaseBuild) { + // ignore: avoid_print + print('⚠️ ProbeAgent: BLOCKED — running in release mode.'); + // ignore: avoid_print + print(' The ProbeAgent opens a WebSocket server on the device.'); + // ignore: avoid_print + print(' Do NOT ship this to production users.'); + // ignore: avoid_print + print(''); + // ignore: avoid_print + print(' To allow: ProbeAgent.start(allowReleaseBuild: true)'); + // ignore: avoid_print + print(' To silence: add --dart-define=PROBE_AGENT_FORCE=true'); + return; + } + + if (isRelease && allowReleaseBuild) { + const force = bool.fromEnvironment('PROBE_AGENT_FORCE', defaultValue: false); + if (!force) { + // ignore: avoid_print + print('⚠️ ProbeAgent: WARNING — starting in RELEASE mode.'); + // ignore: avoid_print + print(' This build has a WebSocket debug server enabled.'); + // ignore: avoid_print + print(' Do NOT distribute to end users.'); + // ignore: avoid_print + print(' Add --dart-define=PROBE_AGENT_FORCE=true to suppress this warning.'); + } + } + + if (isProfile) { + // ignore: avoid_print + print('ProbeAgent: starting in profile mode (physical device testing)'); + } + + await _startInternal(port); } static Future _startInternal(int port) async { diff --git a/probe_agent/lib/src/executor.dart b/probe_agent/lib/src/executor.dart index 7fa0f76..8f84bb2 100644 --- a/probe_agent/lib/src/executor.dart +++ b/probe_agent/lib/src/executor.dart @@ -636,11 +636,17 @@ class ProbeExecutor { return nav; } + int _nextPointer = 900; // Start high to avoid collisions with real touches + Future<_ProbeGesture> _createGesture(Offset position) async { final binding = GestureBinding.instance; - final pointer = PointerDownEvent(position: position); + final pointerId = _nextPointer++; + final pointer = PointerDownEvent( + pointer: pointerId, + position: position, + ); binding.handlePointerEvent(pointer); - return _ProbeGesture(position, binding); + return _ProbeGesture(position, binding, pointerId); } Future _restartApp() async { @@ -655,15 +661,22 @@ class ProbeExecutor { class _ProbeGesture { Offset _position; final GestureBinding _binding; + final int _pointer; - _ProbeGesture(this._position, this._binding); + _ProbeGesture(this._position, this._binding, this._pointer); Future up() async { - _binding.handlePointerEvent(PointerUpEvent(position: _position)); + _binding.handlePointerEvent(PointerUpEvent( + pointer: _pointer, + position: _position, + )); } Future moveTo(Offset location, {Duration? timeStamp}) async { - _binding.handlePointerEvent(PointerMoveEvent(position: location)); + _binding.handlePointerEvent(PointerMoveEvent( + pointer: _pointer, + position: location, + )); _position = location; } diff --git a/probe_agent/lib/src/finder.dart b/probe_agent/lib/src/finder.dart index 053d1f5..2e45373 100644 --- a/probe_agent/lib/src/finder.dart +++ b/probe_agent/lib/src/finder.dart @@ -9,25 +9,28 @@ class ProbeFinder { static final ProbeFinder instance = ProbeFinder._(); /// Returns all [Element]s matching the given selector map. + /// Only returns elements that are currently visible on screen + /// (not behind Offstage, Visibility(false), or off-screen routes). List findElements(Map sel) { final kind = sel['kind'] as String? ?? 'text'; final text = sel['text'] as String? ?? ''; final ordinal = (sel['ordinal'] as num?)?.toInt() ?? 1; final container = sel['container'] as String? ?? ''; + List raw; switch (kind) { case 'text': - return _findByText(text); + raw = _findByText(text); case 'id': final key = text.startsWith('#') ? text.substring(1) : text; - return _findByKey(key); + raw = _findByKey(key); case 'type': - return _findByType(text); + raw = _findByType(text); case 'ordinal': - final matches = _findByText(text); + final matches = _findByText(text).where(_isVisible).toList(); if (ordinal > 0 && ordinal <= matches.length) { return [matches[ordinal - 1]]; } @@ -35,24 +38,25 @@ class ProbeFinder { case 'positional': if (container.isNotEmpty) { - final containers = _findByText(container); + final containers = _findByText(container).where(_isVisible).toList(); if (containers.isEmpty) return []; - // Find text within the container element's subtree final results = []; for (final c in containers) { _visitElement(c, (e) { - if (_matchesText(e.widget, text)) { + if (_matchesText(e.widget, text) && _isVisible(e)) { results.add(e); } }); } return results; } - return _findByText(text); + raw = _findByText(text); default: - return _findByText(text); + raw = _findByText(text); } + // Filter to only visible elements + return raw.where(_isVisible).toList(); } List _findByText(String text) { @@ -107,6 +111,38 @@ class ProbeFinder { return false; } + /// Returns true if the element is currently visible on screen. + /// Checks that the render object is painted and not hidden behind + /// Offstage or Visibility widgets. + bool _isVisible(Element element) { + final ro = element.renderObject; + if (ro == null || !ro.attached) return false; + if (ro is RenderBox) { + // Zero-size widgets are not visible + if (ro.size == Size.zero) return false; + // Check if the widget is actually painted (not behind Offstage etc.) + if (!ro.hasSize) return false; + } + // Walk up the tree to check for Offstage / Visibility ancestors + Element? current = element; + while (current != null) { + final widget = current.widget; + if (widget is Offstage && widget.offstage) return false; + if (widget is Visibility && !widget.visible) return false; + current = _parentElement(current); + } + return true; + } + + Element? _parentElement(Element element) { + Element? parent; + element.visitAncestorElements((e) { + parent = e; + return false; // stop after first ancestor + }); + return parent; + } + void walkTree(void Function(Element) visitor) { final rootElement = WidgetsBinding.instance.rootElement; if (rootElement == null) return; @@ -114,6 +150,10 @@ class ProbeFinder { } void _visitElement(Element element, void Function(Element) visitor) { + // Skip subtrees rooted at Offstage or Visibility(visible: false) + final widget = element.widget; + if (widget is Offstage && widget.offstage) return; + if (widget is Visibility && !widget.visible) return; visitor(element); element.visitChildren((child) => _visitElement(child, visitor)); } diff --git a/probe_agent/lib/src/server.dart b/probe_agent/lib/src/server.dart index d3c9335..7b87fab 100644 --- a/probe_agent/lib/src/server.dart +++ b/probe_agent/lib/src/server.dart @@ -67,6 +67,11 @@ class ProbeServer { void _handleConnection(WebSocket ws) { // ignore: avoid_print print('ProbeAgent: CLI connected'); + + // Enable WebSocket-level ping/pong keepalive to prevent idle connections + // from being dropped by iproxy or iOS network stack on physical devices. + ws.pingInterval = const Duration(seconds: 5); + final executor = ProbeExecutor((msg) => ws.add(msg)); _executor = executor; diff --git a/vscode/package.json b/vscode/package.json index b56e7af..158575a 100644 --- a/vscode/package.json +++ b/vscode/package.json @@ -2,7 +2,7 @@ "name": "flutterprobe", "displayName": "FlutterProbe", "description": "High-performance E2E testing for Flutter apps — ProbeScript language support, local & cloud device testing (BrowserStack, Sauce Labs, AWS Device Farm, Firebase Test Lab, LambdaTest), visual regression, test recording, and Studio integration", - "version": "0.4.2", + "version": "0.5.0", "publisher": "flutterprobe", "icon": "resources/probe-icon.png", "engines": { "vscode": "^1.85.0" }, diff --git a/website/src/pages/index.astro b/website/src/pages/index.astro index 965333f..6854bb0 100644 --- a/website/src/pages/index.astro +++ b/website/src/pages/index.astro @@ -429,7 +429,7 @@
-
v0.4.2 — BSL 1.1 open source
+
v0.5.0 — BSL 1.1 open source

Flutter E2E tests in
plain English

Write tests your whole team can read. Execute with direct widget-tree access — no accessibility layer, no WebDriver overhead.

From dbde2a109f216498382cba33e3e77278cd995d8a Mon Sep 17 00:00:00 2001 From: Patrick Bertsch Date: Wed, 25 Mar 2026 23:57:24 -0600 Subject: [PATCH 2/7] fix: skip clear app data immediately on physical iOS --- internal/runner/device_context.go | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/internal/runner/device_context.go b/internal/runner/device_context.go index cda153f..8cc5efd 100644 --- a/internal/runner/device_context.go +++ b/internal/runner/device_context.go @@ -116,6 +116,14 @@ func (dc *DeviceContext) RestartApp(ctx context.Context) error { // operation that wipes SharedPreferences, databases, and all local files. // It requires explicit opt-in via --allow-clear-data flag or interactive confirmation. func (dc *DeviceContext) ClearAppData(ctx context.Context) error { + // Physical iOS: skip immediately — no filesystem access, and attempting + // clear would kill the app/agent before we can recover. + if dc.Platform == device.PlatformIOS && dc.IsPhysical { + fmt.Println(" \033[33m⚠\033[0m clear app data is not supported on physical iOS devices — skipping") + fmt.Println(" Workaround: uninstall and reinstall the app manually") + return nil + } + // Gate: require explicit permission for destructive data wipe if !dc.AllowClearData { if dc.Confirm == nil { @@ -152,12 +160,7 @@ func (dc *DeviceContext) ClearAppData(ctx context.Context) error { } case device.PlatformIOS: - if dc.IsPhysical { - fmt.Println(" \033[33m⚠\033[0m clear app data is not supported on physical iOS devices") - fmt.Println(" Workaround: uninstall and reinstall the app manually") - return nil - } - + // Physical iOS already handled above (early return). simctl := dc.Manager.SimCtl() _ = simctl.Terminate(ctx, dc.Serial, dc.AppID) From d70e187b0f5bc4e1a536c12acf7ca452ec84801d Mon Sep 17 00:00:00 2001 From: Patrick Bertsch Date: Thu, 26 Mar 2026 02:08:16 -0600 Subject: [PATCH 3/7] feat: HTTP fallback transport, WiFi testing, and `if visible` syntax --- CHANGELOG.md | 6 + docs/wiki/Architecture-Overview.md | 23 +- docs/wiki/Troubleshooting.md | 18 +- docs/wiki/iOS-Integration-Guide.md | 26 ++- internal/cli/test.go | 43 +++- internal/parser/ast.go | 1 + internal/parser/parser.go | 35 ++- internal/probelink/http_client.go | 299 +++++++++++++++++++++++++ internal/probelink/http_client_test.go | 110 +++++++++ internal/probelink/iface.go | 38 ++++ internal/runner/device_context.go | 17 +- internal/runner/executor.go | 24 +- internal/runner/runner.go | 10 +- probe_agent/lib/src/agent.dart | 6 +- probe_agent/lib/src/executor.dart | 46 +++- probe_agent/lib/src/server.dart | 85 ++++++- website/src/pages/index.astro | 2 +- 17 files changed, 731 insertions(+), 58 deletions(-) create mode 100644 internal/probelink/http_client.go create mode 100644 internal/probelink/http_client_test.go create mode 100644 internal/probelink/iface.go diff --git a/CHANGELOG.md b/CHANGELOG.md index 322d66f..1ff7c45 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -22,6 +22,12 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). - ProbeAgent profile mode support — `ProbeAgent.start()` works in profile builds (required for physical iOS) - ProbeAgent release mode safeguards — blocked by default, opt-in via `allowReleaseBuild: true` + `PROBE_AGENT_FORCE=true` - Test files for all packages: `cmd/probe`, `internal/cli`, `internal/ios`, `internal/device` (manager tests) +- HTTP POST fallback transport (`POST /probe/rpc`) — stateless alternative to WebSocket for physical devices, eliminates persistent connection drops +- `ProbeClient` interface — both WebSocket `Client` and `HTTPClient` satisfy it, enabling transport-agnostic test execution +- WiFi testing mode (`--host ` + `--token ` + `--dart-define=PROBE_WIFI=true`) — test physical devices without USB, no iproxy needed +- `tap "X" if visible` ProbeScript syntax — silently skips tap when widget is not found, replaces verbose dialog-dismissal recipes +- Direct `onTap` invocation fallback for `Semantics`-wrapped widgets — fixes tap failures on physical devices where synthetic gestures don't reach `GestureDetector` +- `take screenshot "name"` now accepts name directly (previously required `called` keyword) ### Changed diff --git a/docs/wiki/Architecture-Overview.md b/docs/wiki/Architecture-Overview.md index f90b61d..caec878 100644 --- a/docs/wiki/Architecture-Overview.md +++ b/docs/wiki/Architecture-Overview.md @@ -75,16 +75,23 @@ The agent runs **inside the production Flutter app** using `WidgetsFlutterBindin 3. CLI connects: `ws://127.0.0.1:/probe?token=` 4. Fallback: parse token from `simctl spawn ... log show` -### iOS Physical Device +### iOS Physical Device (USB) 1. CLI detects physical device (UDID not in `simctl list`) -2. CLI kills stale `iproxy` processes for this UDID -3. CLI starts `iproxy --udid ` -4. CLI reads token from `idevicesyslog -u --match PROBE_TOKEN` -5. CLI connects: `ws://127.0.0.1:/probe?token=` -6. WebSocket ping/pong keepalive (5s) prevents idle drops -7. Auto-reconnect on connection loss (up to 2 retries per step) -8. App lifecycle managed via `xcrun devicectl` (launch/terminate) +2. CLI kills stale `iproxy` processes, starts fresh `iproxy` +3. CLI reads token from `idevicesyslog` +4. CLI connects via HTTP POST: `POST http://127.0.0.1:/probe/rpc?token=` +5. Each command is an independent HTTP request (no persistent connection to drop) +6. Auto-reconnect on connection loss (up to 2 retries per step) +7. App lifecycle managed via `xcrun devicectl` (launch/terminate) + +### iOS Physical Device (WiFi — recommended) + +1. App built with `--dart-define=PROBE_WIFI=true` (binds to `0.0.0.0`) +2. CLI connects directly to device IP: `--host --token ` +3. No `iproxy` needed — direct HTTP POST over WiFi +4. Zero USB-C charge/data switching drops +5. App lifecycle managed via `xcrun devicectl` ### Cloud (Relay Mode) diff --git a/docs/wiki/Troubleshooting.md b/docs/wiki/Troubleshooting.md index c026fe7..75ef725 100644 --- a/docs/wiki/Troubleshooting.md +++ b/docs/wiki/Troubleshooting.md @@ -77,14 +77,20 @@ if (!probeEnabled) { 3. Read the token: `cat /tmp/probe/token` 4. Ensure `PROBE_AGENT=true` was passed during build -### Physical iOS Device — WebSocket Drops +### Physical iOS Device — Connection Drops via USB -**Cause**: iproxy connections on physical devices can be closed by iOS during idle periods. +**Cause**: USB-C cables cause intermittent drops when the device switches between charging and data transfer modes. This kills the iproxy tunnel. -**Fix**: FlutterProbe v0.5.0+ includes automatic ping/pong keepalive (every 5s) and auto-reconnect (up to 2 retries). If you still see drops: -1. Check iproxy is running: `pgrep -fl iproxy` -2. Restart iproxy: `pkill -f "iproxy.*" && iproxy 48686 48686 --udid &` -3. Use `--reconnect-delay 5s` for slower devices +**Fix**: Use **WiFi mode** instead of USB to eliminate drops entirely: +```bash +# Build with WiFi enabled +flutter build ios --profile --dart-define=PROBE_AGENT=true --dart-define=PROBE_WIFI=true + +# Run over WiFi (find token in app console logs) +probe test tests/ --host --token +``` + +If you must use USB: FlutterProbe v0.5.0+ uses HTTP POST transport (not WebSocket) for physical devices, with auto-reconnect (up to 2 retries per step). Use a USB-A cable if available — no charge/data switching issue. ### Physical iOS Device — "devicectl launch: not installed" diff --git a/docs/wiki/iOS-Integration-Guide.md b/docs/wiki/iOS-Integration-Guide.md index cfc55de..f98ca06 100644 --- a/docs/wiki/iOS-Integration-Guide.md +++ b/docs/wiki/iOS-Integration-Guide.md @@ -145,12 +145,32 @@ flutter run --profile --dart-define=PROBE_AGENT=true \ --device-id ``` +### Connection Modes + +**USB mode** (default): +1. FlutterProbe starts `iproxy` to forward port 48686 from host to device +2. CLI reads the agent token from `idevicesyslog` +3. Uses HTTP POST transport (stateless, no persistent connection to drop) + +**WiFi mode** (recommended — no USB-C charging/data switching drops): +```bash +# Build with WiFi enabled +flutter build ios --profile --flavor \ + --dart-define=PROBE_AGENT=true \ + --dart-define=PROBE_WIFI=true + +# Run tests over WiFi (no USB cable needed) +probe test tests/ --host --token --device +``` + +To find the token, check the app's console output for `PROBE_TOKEN=...` (printed every 3 seconds). + ### How It Works -1. **Port forwarding**: FlutterProbe automatically starts `iproxy` to forward port 48686 from the host to the device -2. **Token reading**: The CLI reads the agent token from `idevicesyslog` (instead of simctl file paths) +1. **USB**: `iproxy` forwards port 48686; CLI reads token via `idevicesyslog`; HTTP POST transport +2. **WiFi**: Agent binds to `0.0.0.0` (via `PROBE_WIFI=true`); CLI connects directly to device IP; no iproxy needed 3. **App lifecycle**: `xcrun devicectl` handles launch/terminate (instead of `simctl`) -4. **Keepalive**: WebSocket ping/pong frames every 5s prevent idle connection drops +4. **Keepalive**: WebSocket ping/pong frames every 5s prevent idle connection drops (USB mode) 5. **Auto-reconnect**: If the connection drops mid-test, the CLI reconnects transparently (up to 2 retries) ### Limitations on Physical Devices diff --git a/internal/cli/test.go b/internal/cli/test.go index b4acf03..57f026c 100644 --- a/internal/cli/test.go +++ b/internal/cli/test.go @@ -63,8 +63,10 @@ func init() { f.Duration("timeout", 0, "per-step timeout; 0 uses probe.yaml or default 30s") // Agent connection + f.String("host", "", "ProbeAgent host IP (default: 127.0.0.1; use device IP for WiFi testing)") f.Int("port", 0, "ProbeAgent WebSocket port (default: 48686)") f.Duration("dial-timeout", 0, "max time to establish WebSocket connection (default: 30s)") + f.String("token", "", "ProbeAgent auth token (skip auto-detection; use with --host for WiFi testing)") f.Duration("token-timeout", 0, "max time to wait for agent auth token on startup (default: 30s)") f.Duration("reconnect-delay", 0, "delay after app restart before reconnecting WebSocket (default: 2s)") @@ -150,6 +152,8 @@ func runTests(cmd *cobra.Command, args []string) error { noVideoFlag, _ := cmd.Flags().GetBool("no-video") // Agent connection overrides: CLI flag > probe.yaml (already loaded) + agentHost, _ := cmd.Flags().GetString("host") + agentToken, _ := cmd.Flags().GetString("token") agentPort, _ := cmd.Flags().GetInt("port") dialTimeout, _ := cmd.Flags().GetDuration("dial-timeout") tokenTimeout, _ := cmd.Flags().GetDuration("token-timeout") @@ -353,7 +357,7 @@ func runTests(cmd *cobra.Command, args []string) error { // ── Single-device mode (default) ── // Connect to ProbeAgent (skip if dry-run) - var client *probelink.Client + var client probelink.ProbeClient var dm *device.Manager var platform device.Platform var cloudSession *cloud.Session // non-nil when using a cloud provider @@ -660,8 +664,12 @@ func runTests(cmd *cobra.Command, args []string) error { statusOK(statusW, msgAppInstalledAndLaunched) } + host := "127.0.0.1" + if agentHost != "" { + host = agentHost + } dialOpts := probelink.DialOptions{ - Host: "127.0.0.1", + Host: host, Port: cfg.Agent.Port, DialTimeout: cfg.Agent.DialTimeout, } @@ -670,13 +678,16 @@ func runTests(cmd *cobra.Command, args []string) error { // Detect physical vs simulator early for connection setup isPhysicalIOS := dm.IsPhysicalIOS(ctx, deviceSerial) - if isPhysicalIOS { - // Physical iOS: set up iproxy for port forwarding + if isPhysicalIOS && agentHost == "" { + // Physical iOS via USB: set up iproxy for port forwarding cleanupIProxy, iproxyErr := dm.EnsureIProxy(ctx, deviceSerial, cfg.Agent.Port, cfg.Agent.AgentDevicePort()) if iproxyErr != nil { return fmt.Errorf("iproxy setup: %w", iproxyErr) } defer cleanupIProxy() + } else if isPhysicalIOS && agentHost != "" { + // Physical iOS via WiFi: connect directly, no iproxy needed + fmt.Fprintf(statusW, " \033[36mℹ\033[0m WiFi mode: connecting to %s:%d\n", agentHost, cfg.Agent.Port) } else { // iOS simulator: grant privacy permissions and relaunch the app. // simctl privacy grant terminates the running app, so we must @@ -691,14 +702,25 @@ func runTests(cmd *cobra.Command, args []string) error { time.Sleep(3 * time.Second) // give agent time to start } } - // Simulators share host loopback; physical uses iproxy — both on 127.0.0.1 - fmt.Fprintln(statusW, msgWaitingForTokenIOS) - token, err := dm.ReadTokenIOS(ctx, deviceSerial, cfg.Agent.TokenReadTimeout, cfg.Project.App) - if err != nil { - return fmt.Errorf("agent token: %w — is the app running with probe_agent?", err) + // Read token: use --token flag if provided, otherwise auto-detect + var token string + if agentToken != "" { + token = agentToken + } else { + fmt.Fprintln(statusW, msgWaitingForTokenIOS) + var tokenErr error + token, tokenErr = dm.ReadTokenIOS(ctx, deviceSerial, cfg.Agent.TokenReadTimeout, cfg.Project.App) + if tokenErr != nil { + return fmt.Errorf("agent token: %w — is the app running with probe_agent?", tokenErr) + } } dialOpts.Token = token - client, err = probelink.DialWithOptions(ctx, dialOpts) + if isPhysicalIOS { + // Physical iOS: use HTTP POST fallback (no persistent connection to drop) + client, err = probelink.DialHTTP(ctx, dialOpts) + } else { + client, err = probelink.DialWithOptions(ctx, dialOpts) + } if err != nil { return fmt.Errorf("connecting to ProbeAgent: %w", err) } @@ -850,6 +872,7 @@ func runTests(cmd *cobra.Command, args []string) error { Port: cfg.Agent.Port, DevicePort: cfg.Agent.AgentDevicePort(), IsPhysical: isPhysical, + UseHTTP: isPhysical, // physical devices use HTTP fallback AllowClearData: autoYes, Confirm: promptUserConfirm, GrantPermissionsOnClear: autoYes || cfg.Defaults.GrantPermissionsOnClear, diff --git a/internal/parser/ast.go b/internal/parser/ast.go index 21534ef..6a2dc96 100644 --- a/internal/parser/ast.go +++ b/internal/parser/ast.go @@ -177,6 +177,7 @@ type ActionStep struct { Direction SwipeDirection // for swipe / scroll Name string // for screenshot name, rotate direction, locale, etc. To *Selector // for drag: destination + IfVisible bool // if true, skip silently when selector is not found Line int } diff --git a/internal/parser/parser.go b/internal/parser/parser.go index 680ae4b..36a4696 100644 --- a/internal/parser/parser.go +++ b/internal/parser/parser.go @@ -396,6 +396,23 @@ func (p *Parser) parseStep() (Step, error) { } } +// checkIfVisible checks for a trailing "if visible" suffix before the newline. +// Returns true if "if visible" was consumed. +func (p *Parser) checkIfVisible() bool { + if p.peek().Type != TOKEN_IF { + return false + } + // Save position to restore if next word isn't "visible" + saved := p.pos + p.advance() // if + if p.peek().Literal == "visible" { + p.advance() // visible + return true + } + p.pos = saved // restore — this was a regular "if", not "if visible" + return false +} + // ---- Action parsers ---- func (p *Parser) parseActionOpen() (Step, error) { @@ -416,10 +433,10 @@ func (p *Parser) parseActionTap() (Step, error) { line := p.peek().Line p.advance() // tap p.skipFillers() - // Check for ordinal sel := p.parseSelector() + ifVis := p.checkIfVisible() p.consumeNewline() - return ActionStep{Verb: VerbTap, Sel: &sel, Line: line}, nil + return ActionStep{Verb: VerbTap, Sel: &sel, IfVisible: ifVis, Line: line}, nil } func (p *Parser) parseActionType() (Step, error) { @@ -437,8 +454,9 @@ func (p *Parser) parseActionType() (Step, error) { for p.peek().Type == TOKEN_FIELD || p.peek().Type == TOKEN_BUTTON { p.advance() } + ifVis := p.checkIfVisible() p.consumeNewline() - return ActionStep{Verb: VerbType, Text: text, Sel: sel, Line: line}, nil + return ActionStep{Verb: VerbType, Text: text, Sel: sel, IfVisible: ifVis, Line: line}, nil } func (p *Parser) parseActionSwipe() (Step, error) { @@ -476,8 +494,9 @@ func (p *Parser) parseActionLongPress() (Step, error) { p.advance() // long press p.skipFillers() sel := p.parseSelector() + ifVis := p.checkIfVisible() p.consumeNewline() - return ActionStep{Verb: VerbLongPress, Sel: &sel, Line: line}, nil + return ActionStep{Verb: VerbLongPress, Sel: &sel, IfVisible: ifVis, Line: line}, nil } func (p *Parser) parseActionDoubleTap() (Step, error) { @@ -485,8 +504,9 @@ func (p *Parser) parseActionDoubleTap() (Step, error) { p.advance() // double tap p.skipFillers() sel := p.parseSelector() + ifVis := p.checkIfVisible() p.consumeNewline() - return ActionStep{Verb: VerbDoubleTap, Sel: &sel, Line: line}, nil + return ActionStep{Verb: VerbDoubleTap, Sel: &sel, IfVisible: ifVis, Line: line}, nil } func (p *Parser) parseActionClear() (Step, error) { @@ -494,8 +514,9 @@ func (p *Parser) parseActionClear() (Step, error) { p.advance() // clear p.skipFillers() sel := p.parseSelector() + ifVis := p.checkIfVisible() p.consumeNewline() - return ActionStep{Verb: VerbClear, Sel: &sel, Line: line}, nil + return ActionStep{Verb: VerbClear, Sel: &sel, IfVisible: ifVis, Line: line}, nil } func (p *Parser) parseActionClose() (Step, error) { @@ -536,6 +557,8 @@ func (p *Parser) parseActionTakeShot() (Step, error) { if p.peek().Type == TOKEN_CALLED { p.advance() name = p.expectString("screenshot name") + } else if p.peek().Type == TOKEN_STRING { + name = p.expectString("screenshot name") } p.consumeNewline() return ActionStep{Verb: VerbTakeShot, Name: name, Line: line}, nil diff --git a/internal/probelink/http_client.go b/internal/probelink/http_client.go new file mode 100644 index 0000000..72dfd62 --- /dev/null +++ b/internal/probelink/http_client.go @@ -0,0 +1,299 @@ +package probelink + +import ( + "bytes" + "context" + "encoding/json" + "fmt" + "io" + "net/http" + "os" + "path/filepath" + "sync" + "time" +) + +// HTTPClient is a stateless HTTP POST transport for the ProbeAgent. +// Each Call() sends an independent HTTP request to POST /probe/rpc. +// This avoids the persistent WebSocket connection that is fragile on +// physical devices via iproxy. +type HTTPClient struct { + baseURL string // http://127.0.0.1:48686/probe/rpc?token= + token string + httpClient *http.Client + mu sync.Mutex + closed bool + addr string // safe address for logging (no token) + OnNotify func(method string, params json.RawMessage) // no-op in HTTP mode +} + +// DialHTTP creates an HTTPClient and verifies connectivity with a Ping. +func DialHTTP(ctx context.Context, opts DialOptions) (*HTTPClient, error) { + if opts.Port == 0 { + opts.Port = defaultAgentPort + } + if opts.Host == "" { + opts.Host = "127.0.0.1" + } + if opts.DialTimeout == 0 { + opts.DialTimeout = defaultDialTimeout + } + + c := &HTTPClient{ + baseURL: fmt.Sprintf("http://%s:%d/probe/rpc", opts.Host, opts.Port), + token: opts.Token, + addr: fmt.Sprintf("http://%s:%d/probe/rpc", opts.Host, opts.Port), + httpClient: &http.Client{ + Timeout: 2 * time.Minute, // generous for long-running commands like wait + }, + } + + // Verify connectivity + pingCtx, cancel := context.WithTimeout(ctx, opts.DialTimeout) + defer cancel() + if err := c.Ping(pingCtx); err != nil { + return nil, fmt.Errorf("probelink: http dial %s: ping failed: %w", c.addr, err) + } + + return c, nil +} + +// Call sends a JSON-RPC request as an HTTP POST and returns the response. +func (c *HTTPClient) Call(ctx context.Context, method string, params any) (json.RawMessage, error) { + c.mu.Lock() + if c.closed { + c.mu.Unlock() + return nil, fmt.Errorf("probelink: http client closed") + } + c.mu.Unlock() + + req, err := NewRequest(method, params) + if err != nil { + return nil, fmt.Errorf("probelink: marshal params: %w", err) + } + + data, err := json.Marshal(req) + if err != nil { + return nil, err + } + + url := fmt.Sprintf("%s?token=%s", c.baseURL, c.token) + httpReq, err := http.NewRequestWithContext(ctx, "POST", url, bytes.NewReader(data)) + if err != nil { + return nil, fmt.Errorf("probelink: http request: %w", err) + } + httpReq.Header.Set("Content-Type", "application/json") + + httpResp, err := c.httpClient.Do(httpReq) + if err != nil { + return nil, fmt.Errorf("probelink: http call %s: %w", method, err) + } + defer httpResp.Body.Close() + + body, err := io.ReadAll(httpResp.Body) + if err != nil { + return nil, fmt.Errorf("probelink: http read response: %w", err) + } + + if httpResp.StatusCode != http.StatusOK { + return nil, fmt.Errorf("probelink: http %s: status %d: %s", method, httpResp.StatusCode, string(body)) + } + + var resp Response + if err := json.Unmarshal(body, &resp); err != nil { + return nil, fmt.Errorf("probelink: http unmarshal: %w", err) + } + + if resp.Error != nil { + return nil, resp.Error + } + return resp.Result, nil +} + +// Ping verifies the agent is alive. +func (c *HTTPClient) Ping(ctx context.Context) error { + _, err := c.Call(ctx, MethodPing, nil) + return err +} + +// Close marks the client as closed. +func (c *HTTPClient) Close() error { + c.mu.Lock() + c.closed = true + c.mu.Unlock() + return nil +} + +// Connected returns true if the client has not been closed. +func (c *HTTPClient) Connected() bool { + c.mu.Lock() + defer c.mu.Unlock() + return !c.closed +} + +// WaitSettled blocks until the agent reports the UI is fully settled. +func (c *HTTPClient) WaitSettled(ctx context.Context, timeout time.Duration) error { + params := WaitParams{Kind: "settled", Timeout: timeout.Seconds()} + _, err := c.Call(ctx, MethodSettled, params) + return err +} + +// ---- High-level helper methods (same API as WebSocket Client) ---- + +func (c *HTTPClient) Open(ctx context.Context, screen string) error { + _, err := c.Call(ctx, MethodOpen, OpenParams{Screen: screen}) + return err +} + +func (c *HTTPClient) Tap(ctx context.Context, sel SelectorParam) error { + _, err := c.Call(ctx, MethodTap, TapParams{Selector: sel}) + return err +} + +func (c *HTTPClient) TypeText(ctx context.Context, sel SelectorParam, text string) error { + _, err := c.Call(ctx, MethodType, TypeParams{Selector: sel, Text: text}) + return err +} + +func (c *HTTPClient) See(ctx context.Context, params SeeParams) error { + _, err := c.Call(ctx, MethodSee, params) + return err +} + +func (c *HTTPClient) Wait(ctx context.Context, params WaitParams) error { + _, err := c.Call(ctx, MethodWait, params) + return err +} + +func (c *HTTPClient) Swipe(ctx context.Context, direction string, sel *SelectorParam) error { + _, err := c.Call(ctx, MethodSwipe, SwipeParams{Direction: direction, Selector: sel}) + return err +} + +func (c *HTTPClient) Scroll(ctx context.Context, direction string, sel *SelectorParam) error { + _, err := c.Call(ctx, MethodScroll, ScrollParams{Direction: direction, Selector: sel}) + return err +} + +func (c *HTTPClient) LongPress(ctx context.Context, sel SelectorParam) error { + _, err := c.Call(ctx, MethodLongPress, TapParams{Selector: sel}) + return err +} + +func (c *HTTPClient) DoubleTap(ctx context.Context, sel SelectorParam) error { + _, err := c.Call(ctx, MethodDoubleTap, TapParams{Selector: sel}) + return err +} + +func (c *HTTPClient) Clear(ctx context.Context, sel SelectorParam) error { + _, err := c.Call(ctx, MethodClear, TapParams{Selector: sel}) + return err +} + +func (c *HTTPClient) Screenshot(ctx context.Context, name string) (string, error) { + type params struct { + Name string `json:"name"` + } + raw, err := c.Call(ctx, MethodScreenshot, params{Name: name}) + if err != nil { + return "", err + } + var result ScreenshotResult + if err := json.Unmarshal(raw, &result); err != nil { + return "", err + } + if result.Data != "" { + decoded, decErr := base64Decode(result.Data) + if decErr == nil && len(decoded) > 0 { + localDir := "reports/screenshots" + _ = mkdirAll(localDir) + localPath := joinPath(localDir, basePath(result.Path)) + if writeErr := writeFile(localPath, decoded); writeErr == nil { + absPath, _ := absFilePath(localPath) + if absPath != "" { + return absPath, nil + } + return localPath, nil + } + } + } + return result.Path, nil +} + +func (c *HTTPClient) DumpWidgetTree(ctx context.Context) (string, error) { + raw, err := c.Call(ctx, MethodDumpTree, nil) + if err != nil { + return "", err + } + var result WidgetTreeResult + if err := json.Unmarshal(raw, &result); err != nil { + return "", err + } + return result.Tree, nil +} + +func (c *HTTPClient) RunDart(ctx context.Context, code string) error { + _, err := c.Call(ctx, MethodRunDart, DartParam{Code: code}) + return err +} + +func (c *HTTPClient) RegisterMock(ctx context.Context, m MockParam) error { + _, err := c.Call(ctx, MethodMock, m) + return err +} + +func (c *HTTPClient) DeviceAction(ctx context.Context, action, value string) error { + _, err := c.Call(ctx, MethodDeviceAction, DeviceActionParams{Action: action, Value: value}) + return err +} + +func (c *HTTPClient) SaveLogs(ctx context.Context) error { + _, err := c.Call(ctx, MethodSaveLogs, nil) + return err +} + +func (c *HTTPClient) CopyToClipboard(ctx context.Context, text string) error { + _, err := c.Call(ctx, MethodCopyClipboard, map[string]string{"text": text}) + return err +} + +func (c *HTTPClient) PasteFromClipboard(ctx context.Context) (string, error) { + raw, err := c.Call(ctx, MethodPasteClipboard, nil) + if err != nil { + return "", err + } + var result struct { + Text string `json:"text"` + } + if err := json.Unmarshal(raw, &result); err != nil { + return "", err + } + return result.Text, nil +} + +func (c *HTTPClient) VerifyBrowser(ctx context.Context) error { + _, err := c.Call(ctx, MethodVerifyBrowser, nil) + return err +} + +// File helpers to avoid importing os/filepath in this file +// (they delegate to the stdlib, matching client.go's Screenshot usage) +func mkdirAll(path string) error { + return os.MkdirAll(path, 0755) +} + +func joinPath(elem ...string) string { + return filepath.Join(elem...) +} + +func basePath(path string) string { + return filepath.Base(path) +} + +func writeFile(path string, data []byte) error { + return os.WriteFile(path, data, 0644) +} + +func absFilePath(path string) (string, error) { + return filepath.Abs(path) +} diff --git a/internal/probelink/http_client_test.go b/internal/probelink/http_client_test.go new file mode 100644 index 0000000..56f8208 --- /dev/null +++ b/internal/probelink/http_client_test.go @@ -0,0 +1,110 @@ +package probelink + +import ( + "context" + "encoding/json" + "io" + "net/http" + "net/http/httptest" + "testing" +) + +func TestHTTPClient_Call(t *testing.T) { + // Mock server that returns a JSON-RPC response + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if r.Method != "POST" { + t.Errorf("expected POST, got %s", r.Method) + } + if r.URL.Query().Get("token") != "test-token" { + w.WriteHeader(http.StatusUnauthorized) + return + } + + body, _ := io.ReadAll(r.Body) + var req Request + json.Unmarshal(body, &req) + + resp := Response{ + JSONRPC: "2.0", + ID: req.ID, + Result: json.RawMessage(`{"ok":true}`), + } + w.Header().Set("Content-Type", "application/json") + json.NewEncoder(w).Encode(resp) + })) + defer srv.Close() + + c := &HTTPClient{ + baseURL: srv.URL + "/probe/rpc", + token: "test-token", + httpClient: srv.Client(), + addr: srv.URL, + } + + result, err := c.Call(context.Background(), MethodPing, nil) + if err != nil { + t.Fatalf("Call failed: %v", err) + } + + var res struct { + OK bool `json:"ok"` + } + json.Unmarshal(result, &res) + if !res.OK { + t.Error("expected ok=true") + } +} + +func TestHTTPClient_CallRPCError(t *testing.T) { + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + body, _ := io.ReadAll(r.Body) + var req Request + json.Unmarshal(body, &req) + + resp := Response{ + JSONRPC: "2.0", + ID: req.ID, + Error: &RPCError{Code: -32001, Message: "Widget not found"}, + } + w.Header().Set("Content-Type", "application/json") + json.NewEncoder(w).Encode(resp) + })) + defer srv.Close() + + c := &HTTPClient{ + baseURL: srv.URL + "/probe/rpc", + token: "t", + httpClient: srv.Client(), + } + + _, err := c.Call(context.Background(), MethodTap, nil) + if err == nil { + t.Fatal("expected RPC error") + } + if err.Error() != "rpc error -32001: Widget not found" { + t.Errorf("unexpected error: %v", err) + } +} + +func TestHTTPClient_Connected(t *testing.T) { + c := &HTTPClient{} + if !c.Connected() { + t.Error("new client should be connected") + } + c.Close() + if c.Connected() { + t.Error("closed client should not be connected") + } +} + +func TestHTTPClient_CallWhenClosed(t *testing.T) { + c := &HTTPClient{closed: true} + _, err := c.Call(context.Background(), MethodPing, nil) + if err == nil { + t.Error("expected error when calling closed client") + } +} + +// Verify HTTPClient satisfies ProbeClient interface +var _ ProbeClient = (*HTTPClient)(nil) +var _ ProbeClient = (*Client)(nil) diff --git a/internal/probelink/iface.go b/internal/probelink/iface.go new file mode 100644 index 0000000..1c57a92 --- /dev/null +++ b/internal/probelink/iface.go @@ -0,0 +1,38 @@ +package probelink + +import ( + "context" + "encoding/json" + "time" +) + +// ProbeClient is the interface for communicating with the ProbeAgent. +// Both the WebSocket Client and the HTTP HTTPClient implement this interface. +type ProbeClient interface { + Call(ctx context.Context, method string, params any) (json.RawMessage, error) + Ping(ctx context.Context) error + Close() error + Connected() bool + WaitSettled(ctx context.Context, timeout time.Duration) error + + // High-level commands + Open(ctx context.Context, screen string) error + Tap(ctx context.Context, sel SelectorParam) error + TypeText(ctx context.Context, sel SelectorParam, text string) error + See(ctx context.Context, params SeeParams) error + Wait(ctx context.Context, params WaitParams) error + Swipe(ctx context.Context, direction string, sel *SelectorParam) error + Scroll(ctx context.Context, direction string, sel *SelectorParam) error + LongPress(ctx context.Context, sel SelectorParam) error + DoubleTap(ctx context.Context, sel SelectorParam) error + Clear(ctx context.Context, sel SelectorParam) error + Screenshot(ctx context.Context, name string) (string, error) + DumpWidgetTree(ctx context.Context) (string, error) + RunDart(ctx context.Context, code string) error + RegisterMock(ctx context.Context, m MockParam) error + DeviceAction(ctx context.Context, action, value string) error + SaveLogs(ctx context.Context) error + CopyToClipboard(ctx context.Context, text string) error + PasteFromClipboard(ctx context.Context) (string, error) + VerifyBrowser(ctx context.Context) error +} diff --git a/internal/runner/device_context.go b/internal/runner/device_context.go index 8cc5efd..d759cc5 100644 --- a/internal/runner/device_context.go +++ b/internal/runner/device_context.go @@ -27,6 +27,7 @@ type DeviceContext struct { Port int // host-side agent port (default 48686) DevicePort int // on-device agent port (default: same as Port) IsPhysical bool // true for physical devices (vs emulator/simulator) + UseHTTP bool // if true, use HTTP POST instead of WebSocket for reconnection AllowClearData bool // if true, skip confirmation for clear app data (CI/CD mode) Confirm ConfirmFunc // interactive confirmation callback (nil = deny destructive ops unless AllowClearData) GrantPermissionsOnClear bool // if true, auto-grant all permissions after clearing data @@ -329,7 +330,7 @@ func (dc *DeviceContext) RevokeAllPermissions(ctx context.Context) error { // Reconnect waits for the app to boot its agent, reads the new token, // and establishes a fresh WebSocket connection. -func (dc *DeviceContext) Reconnect(ctx context.Context) (*probelink.Client, error) { +func (dc *DeviceContext) Reconnect(ctx context.Context) (probelink.ProbeClient, error) { tokenTimeout := dc.tokenTimeout() switch dc.Platform { @@ -377,12 +378,22 @@ func (dc *DeviceContext) Reconnect(ctx context.Context) (*probelink.Client, erro return nil, fmt.Errorf("reconnect: read token: %w", err) } - client, err := probelink.DialWithOptions(ctx, probelink.DialOptions{ + dialOpts := probelink.DialOptions{ Host: "127.0.0.1", Port: dc.Port, Token: token, DialTimeout: dc.dialTimeoutVal(), - }) + } + + if dc.UseHTTP { + client, err := probelink.DialHTTP(ctx, dialOpts) + if err != nil { + return nil, fmt.Errorf("reconnect: http dial: %w", err) + } + return client, nil + } + + client, err := probelink.DialWithOptions(ctx, dialOpts) if err != nil { return nil, fmt.Errorf("reconnect: dial: %w", err) } diff --git a/internal/runner/executor.go b/internal/runner/executor.go index 3165639..7615939 100644 --- a/internal/runner/executor.go +++ b/internal/runner/executor.go @@ -16,9 +16,9 @@ import ( // Executor walks an AST body and dispatches commands to a ProbeLink client. type Executor struct { - client *probelink.Client + client probelink.ProbeClient deviceCtx *DeviceContext // nil in dry-run mode - onReconnect func(*probelink.Client) // callback to update Runner's client ref + onReconnect func(probelink.ProbeClient) // callback to update Runner's client ref timeout time.Duration recipes map[string]parser.RecipeDef // loaded recipes by name vars map[string]string // variable scope for data-driven tests @@ -30,7 +30,7 @@ type Executor struct { } // NewExecutor creates an Executor. -func NewExecutor(client *probelink.Client, deviceCtx *DeviceContext, onReconnect func(*probelink.Client), timeout time.Duration, verbose bool) *Executor { +func NewExecutor(client probelink.ProbeClient, deviceCtx *DeviceContext, onReconnect func(probelink.ProbeClient), timeout time.Duration, verbose bool) *Executor { return &Executor{ client: client, deviceCtx: deviceCtx, @@ -272,6 +272,22 @@ func (e *Executor) stepDescription(step parser.Step) string { // ---- Action execution ---- func (e *Executor) runAction(ctx context.Context, a parser.ActionStep) error { + // "if visible" suffix: check if the selector is visible before executing. + // If not visible, skip silently (no error). Connection errors propagate. + if a.IfVisible && a.Sel != nil { + checkCtx, cancel := context.WithTimeout(ctx, 1*time.Second) + err := e.client.See(checkCtx, probelink.SeeParams{ + Selector: toSelectorParam(*a.Sel), + }) + cancel() + if err != nil { + if isConnectionError(err) { + return err // propagate connection errors for auto-reconnect + } + return nil // not visible — skip silently + } + } + switch a.Verb { case parser.VerbOpen: screen := "" @@ -654,7 +670,7 @@ func (e *Executor) tryReconnect(ctx context.Context) error { if e.deviceCtx == nil { return fmt.Errorf("reconnect: no device context (cloud mode)") } - fmt.Printf(" \033[33m⟳\033[0m WebSocket connection lost — attempting to reconnect...\n") + fmt.Printf(" \033[33m⟳\033[0m Connection lost — attempting to reconnect...\n") e.client.Close() // The app is still running — we just need to re-dial. diff --git a/internal/runner/runner.go b/internal/runner/runner.go index 59a1e91..3947369 100644 --- a/internal/runner/runner.go +++ b/internal/runner/runner.go @@ -34,7 +34,7 @@ type TestResult struct { // Runner coordinates parsing, connecting, and executing .probe files. type Runner struct { cfg *config.Config - client *probelink.Client + client probelink.ProbeClient deviceCtx *DeviceContext // nil in dry-run mode opts RunOptions recipes map[string]parser.RecipeDef @@ -56,7 +56,7 @@ type RunOptions struct { } // New creates a Runner. -func New(cfg *config.Config, client *probelink.Client, deviceCtx *DeviceContext, opts RunOptions) *Runner { +func New(cfg *config.Config, client probelink.ProbeClient, deviceCtx *DeviceContext, opts RunOptions) *Runner { if opts.Timeout == 0 { opts.Timeout = cfg.Defaults.Timeout } @@ -132,7 +132,7 @@ func (r *Runner) runFile(ctx context.Context, path string) ([]TestResult, error) // Run beforeAll hooks (fail-fast: if beforeAll fails, skip all tests in file) for _, hook := range prog.Hooks { if hook.Kind == parser.HookBeforeAll { - exec := NewExecutor(r.client, r.deviceCtx, func(newClient *probelink.Client) { + exec := NewExecutor(r.client, r.deviceCtx, func(newClient probelink.ProbeClient) { r.client = newClient }, r.opts.Timeout, r.opts.Verbose) if err := exec.RunBody(ctx, hook.Body); err != nil { @@ -163,7 +163,7 @@ func (r *Runner) runFile(ctx context.Context, path string) ([]TestResult, error) // Run afterAll hooks (always, best-effort) for _, hook := range prog.Hooks { if hook.Kind == parser.HookAfterAll { - exec := NewExecutor(r.client, r.deviceCtx, func(newClient *probelink.Client) { + exec := NewExecutor(r.client, r.deviceCtx, func(newClient probelink.ProbeClient) { r.client = newClient }, r.opts.Timeout, r.opts.Verbose) _ = exec.RunBody(ctx, hook.Body) @@ -209,7 +209,7 @@ func (r *Runner) runDataDriven(ctx context.Context, prog *parser.Program, t pars func (r *Runner) runSingleTest(ctx context.Context, prog *parser.Program, t parser.TestDef, file string, vars map[string]string, row int) TestResult { start := time.Now() - exec := NewExecutor(r.client, r.deviceCtx, func(newClient *probelink.Client) { + exec := NewExecutor(r.client, r.deviceCtx, func(newClient probelink.ProbeClient) { r.client = newClient }, r.opts.Timeout, r.opts.Verbose) for name, rec := range r.recipes { diff --git a/probe_agent/lib/src/agent.dart b/probe_agent/lib/src/agent.dart index ad88eab..ee8aad0 100644 --- a/probe_agent/lib/src/agent.dart +++ b/probe_agent/lib/src/agent.dart @@ -109,8 +109,10 @@ class ProbeAgent { ); await _relayClient!.connect(); } else { - // Local mode: listen on port (existing behavior) - _server = ProbeServer(port: port); + // Local mode: listen on port + // PROBE_WIFI=true enables binding to 0.0.0.0 for WiFi testing + const allowWifi = bool.fromEnvironment('PROBE_WIFI', defaultValue: false); + _server = ProbeServer(port: port, allowRemoteConnections: allowWifi); await _server!.start(); } } diff --git a/probe_agent/lib/src/executor.dart b/probe_agent/lib/src/executor.dart index 8f84bb2..0b79e2f 100644 --- a/probe_agent/lib/src/executor.dart +++ b/probe_agent/lib/src/executor.dart @@ -4,7 +4,7 @@ import 'dart:io'; import 'dart:ui' as ui; import 'package:flutter/gestures.dart'; -import 'package:flutter/material.dart' show ElevatedButton, TextButton, OutlinedButton, TextField; +import 'package:flutter/material.dart' show ElevatedButton, GestureDetector, InkWell, TextButton, OutlinedButton, TextField; import 'package:flutter/rendering.dart'; import 'package:flutter/services.dart'; import 'package:flutter/widgets.dart'; @@ -18,7 +18,7 @@ typedef SendFn = void Function(String message); /// ProbeExecutor handles all JSON-RPC method calls from the CLI. class ProbeExecutor { - final SendFn _send; + SendFn _send; final ProbeFinder _finder = ProbeFinder.instance; final ProbeSync _sync = ProbeSync.instance; final ProbeRecorder _recorder = ProbeRecorder(); @@ -33,6 +33,10 @@ class ProbeExecutor { _interceptUrlLauncher(); } + /// Updates the send function. Used by HTTP mode to route responses + /// to the current HTTP request's completer. + set sendFn(SendFn fn) => _send = fn; + /// Intercepts url_launcher platform channel to track external browser launches. void _interceptUrlLauncher() { const channel = MethodChannel('plugins.flutter.io/url_launcher'); @@ -233,10 +237,48 @@ class ProbeExecutor { final element = _requireElement(sel); final box = element.renderObject as RenderBox; final center = box.localToGlobal(box.size.center(Offset.zero)); + + // Check if the matched element is a Semantics wrapper — if so, the + // synthetic gesture may not reach the GestureDetector child. In that + // case, invoke onTap directly instead of using pointer events. + if (element.widget is Semantics) { + final tapped = _tryDirectTap(element); + if (tapped) return; + } + final gesture = await _createGesture(center); await gesture.up(); } + /// Walks down from [element] to find a GestureDetector or InkWell child + /// and invokes its onTap directly. Only used when the matched element is + /// a Semantics wrapper where synthetic pointer events are unreliable. + /// Returns true if onTap was invoked. + bool _tryDirectTap(Element element) { + bool found = false; + void visit(Element e) { + if (found) return; + try { + final widget = e.widget; + if (widget is GestureDetector && widget.onTap != null) { + widget.onTap!(); + found = true; + return; + } + if (widget is InkWell && widget.onTap != null) { + widget.onTap!(); + found = true; + return; + } + e.visitChildren(visit); + } catch (_) { + // Element may be disposed during tree walk — skip safely + } + } + visit(element); + return found; + } + Future _doubleTap(Map sel) async { final element = _requireElement(sel); final box = element.renderObject as RenderBox; diff --git a/probe_agent/lib/src/server.dart b/probe_agent/lib/src/server.dart index 7b87fab..7ad8f5d 100644 --- a/probe_agent/lib/src/server.dart +++ b/probe_agent/lib/src/server.dart @@ -1,4 +1,5 @@ import 'dart:async'; +import 'dart:convert'; import 'dart:io'; import 'dart:math'; @@ -10,20 +11,35 @@ import 'protocol.dart'; /// The probe CLI connects to this server after the app starts. /// Authentication is token-based: the server emits a one-time token /// to stdout/logcat which the CLI reads before connecting. +/// +/// Supports two transport modes: +/// - **WebSocket** (default): persistent connection at `ws://host:port/probe?token=` +/// - **HTTP POST** (fallback for physical devices): stateless `POST /probe/rpc?token=` class ProbeServer { final int port; + final bool allowRemoteConnections; HttpServer? _server; String? _token; ProbeExecutor? _executor; - ProbeServer({this.port = 48686}); + /// Shared executor for HTTP mode — persists state (mocks, URL tracking) + /// across stateless HTTP requests within the same session. + ProbeExecutor? _httpExecutor; + + /// Creates a ProbeServer. + /// Set [allowRemoteConnections] to true for WiFi testing (binds to 0.0.0.0 + /// instead of localhost). Only use in debug/profile builds — never in release. + ProbeServer({this.port = 48686, this.allowRemoteConnections = false}); Timer? _tokenTimer; /// Starts the WebSocket server and prints the session token. Future start() async { _token = _generateToken(); - _server = await HttpServer.bind(InternetAddress.loopbackIPv4, port); + final bindAddress = allowRemoteConnections + ? InternetAddress.anyIPv4 + : InternetAddress.loopbackIPv4; + _server = await HttpServer.bind(bindAddress, port); // Emit token so the CLI (via adb logcat / simctl log) can read it // ignore: avoid_print @@ -43,18 +59,26 @@ class ProbeServer { Future _serve() async { await for (final req in _server!) { - if (!WebSocketTransformer.isUpgradeRequest(req)) { + // Validate token (shared by both WebSocket and HTTP paths) + final queryToken = req.uri.queryParameters['token'] ?? + req.headers.value('x-probe-token'); + if (queryToken != _token) { req.response - ..statusCode = HttpStatus.badRequest + ..statusCode = HttpStatus.unauthorized ..close(); continue; } - // Validate token - final queryToken = req.uri.queryParameters['token']; - if (queryToken != _token) { + // HTTP POST fallback: stateless JSON-RPC over HTTP + if (req.method == 'POST' && req.uri.path == '/probe/rpc') { + await _handleHttpRpc(req); + continue; + } + + // WebSocket upgrade + if (!WebSocketTransformer.isUpgradeRequest(req)) { req.response - ..statusCode = HttpStatus.unauthorized + ..statusCode = HttpStatus.badRequest ..close(); continue; } @@ -64,6 +88,51 @@ class ProbeServer { } } + /// Handles a stateless HTTP POST JSON-RPC request. + Future _handleHttpRpc(HttpRequest req) async { + try { + final body = await utf8.decoder.bind(req).join(); + final probeReq = ProbeRequest.tryParse(body); + if (probeReq == null) { + req.response + ..statusCode = HttpStatus.badRequest + ..headers.contentType = ContentType.json + ..write('{"jsonrpc":"2.0","error":{"code":-32700,"message":"Parse error"}}') + ..close(); + return; + } + + // Use a shared executor so mocks and state persist across HTTP calls. + // Swap the send function per request to route the response back. + final completer = Completer(); + _httpExecutor ??= ProbeExecutor((msg) { + if (!completer.isCompleted) completer.complete(msg); + }); + _httpExecutor!.sendFn = (msg) { + if (!completer.isCompleted) completer.complete(msg); + }; + + await _httpExecutor!.dispatch(probeReq); + + final response = await completer.future.timeout( + const Duration(seconds: 120), + onTimeout: () => '{"jsonrpc":"2.0","id":${probeReq.id},"error":{"code":-32000,"message":"Timeout"}}', + ); + + req.response + ..statusCode = HttpStatus.ok + ..headers.contentType = ContentType.json + ..write(response) + ..close(); + } catch (e) { + req.response + ..statusCode = HttpStatus.internalServerError + ..headers.contentType = ContentType.json + ..write('{"jsonrpc":"2.0","error":{"code":-32603,"message":"${e.toString().replaceAll('"', '\\"')}"}}') + ..close(); + } + } + void _handleConnection(WebSocket ws) { // ignore: avoid_print print('ProbeAgent: CLI connected'); diff --git a/website/src/pages/index.astro b/website/src/pages/index.astro index 6854bb0..c8f1dcc 100644 --- a/website/src/pages/index.astro +++ b/website/src/pages/index.astro @@ -484,7 +484,7 @@

N Natural language

ProbeScript reads like English. No locators, no async/await. QA, product, and engineering can all read the tests.

⚡ Sub-50ms execution

Direct widget-tree access via the Dart agent. No XPath, no accessibility bridge. Command round-trips under 50 milliseconds.

↻ Watch mode

Re-runs on file save with hot-reload awareness. Iteration cycles under 2 seconds. No recompilation needed.

-

⊕ Cross-platform

Same tests on Android emulators and iOS simulators. Permissions, lifecycle, and platform quirks handled automatically.

+

⊕ Cross-platform

Same tests on Android emulators, iOS simulators, and physical devices. USB or WiFi. Permissions, lifecycle, and platform quirks handled automatically.

◉ Test recording

Record taps, swipes, and text input on a real device. FlutterProbe writes the .probe file as you interact.

△ Visual regression

Screenshot baselines with configurable pixel thresholds. Catch unintended UI changes before they ship.

⬡ HTTP mocking

Mock API responses directly in test files. No external packages, no separate mock server.

From 4a6f6c83943af43995a5a885a1c642d28b1630a2 Mon Sep 17 00:00:00 2001 From: Patrick Bertsch Date: Thu, 26 Mar 2026 02:17:52 -0600 Subject: [PATCH 4/7] feat: pre-shared restart token for WiFi mode reconnection --- internal/cli/test.go | 1 + internal/probelink/client.go | 5 ++++ internal/probelink/http_client.go | 5 ++++ internal/probelink/iface.go | 1 + internal/probelink/protocol.go | 3 ++ internal/runner/device_context.go | 49 ++++++++++++++++++++++++++++++- internal/runner/executor.go | 30 +++++++++++++++++-- probe_agent/lib/src/executor.dart | 30 +++++++++++++++++++ probe_agent/lib/src/protocol.dart | 1 + probe_agent/lib/src/server.dart | 47 ++++++++++++++++++++++++++++- 10 files changed, 167 insertions(+), 5 deletions(-) diff --git a/internal/cli/test.go b/internal/cli/test.go index 57f026c..95bb5aa 100644 --- a/internal/cli/test.go +++ b/internal/cli/test.go @@ -873,6 +873,7 @@ func runTests(cmd *cobra.Command, args []string) error { DevicePort: cfg.Agent.AgentDevicePort(), IsPhysical: isPhysical, UseHTTP: isPhysical, // physical devices use HTTP fallback + AgentHost: agentHost, // device IP for WiFi mode AllowClearData: autoYes, Confirm: promptUserConfirm, GrantPermissionsOnClear: autoYes || cfg.Defaults.GrantPermissionsOnClear, diff --git a/internal/probelink/client.go b/internal/probelink/client.go index d85f130..896ee96 100644 --- a/internal/probelink/client.go +++ b/internal/probelink/client.go @@ -453,3 +453,8 @@ func (c *Client) VerifyBrowser(ctx context.Context) error { _, err := c.Call(ctx, MethodVerifyBrowser, nil) return err } + +func (c *Client) SetNextToken(ctx context.Context, token string) error { + _, err := c.Call(ctx, MethodSetNextToken, map[string]string{"token": token}) + return err +} diff --git a/internal/probelink/http_client.go b/internal/probelink/http_client.go index 72dfd62..e2508d9 100644 --- a/internal/probelink/http_client.go +++ b/internal/probelink/http_client.go @@ -276,6 +276,11 @@ func (c *HTTPClient) VerifyBrowser(ctx context.Context) error { return err } +func (c *HTTPClient) SetNextToken(ctx context.Context, token string) error { + _, err := c.Call(ctx, MethodSetNextToken, map[string]string{"token": token}) + return err +} + // File helpers to avoid importing os/filepath in this file // (they delegate to the stdlib, matching client.go's Screenshot usage) func mkdirAll(path string) error { diff --git a/internal/probelink/iface.go b/internal/probelink/iface.go index 1c57a92..ab59871 100644 --- a/internal/probelink/iface.go +++ b/internal/probelink/iface.go @@ -35,4 +35,5 @@ type ProbeClient interface { CopyToClipboard(ctx context.Context, text string) error PasteFromClipboard(ctx context.Context) (string, error) VerifyBrowser(ctx context.Context) error + SetNextToken(ctx context.Context, token string) error } diff --git a/internal/probelink/protocol.go b/internal/probelink/protocol.go index 3ded2c6..8bb7432 100644 --- a/internal/probelink/protocol.go +++ b/internal/probelink/protocol.go @@ -184,6 +184,9 @@ const ( MethodPasteClipboard = "probe.paste_clipboard" MethodVerifyBrowser = "probe.verify_browser" + // Token management + MethodSetNextToken = "probe.set_next_token" + // Notification methods (agent → CLI, no response expected) NotifyRecordedEvent = "probe.recorded_event" NotifyExecDart = "probe.exec_dart" diff --git a/internal/runner/device_context.go b/internal/runner/device_context.go index d759cc5..183ae7a 100644 --- a/internal/runner/device_context.go +++ b/internal/runner/device_context.go @@ -28,6 +28,7 @@ type DeviceContext struct { DevicePort int // on-device agent port (default: same as Port) IsPhysical bool // true for physical devices (vs emulator/simulator) UseHTTP bool // if true, use HTTP POST instead of WebSocket for reconnection + AgentHost string // agent host IP (default "127.0.0.1"; set to device IP for WiFi) AllowClearData bool // if true, skip confirmation for clear app data (CI/CD mode) Confirm ConfirmFunc // interactive confirmation callback (nil = deny destructive ops unless AllowClearData) GrantPermissionsOnClear bool // if true, auto-grant all permissions after clearing data @@ -37,6 +38,14 @@ type DeviceContext struct { DialTimeout time.Duration // max time to establish WebSocket connection (default 30s) } +// agentHost returns the configured agent host or the default. +func (dc *DeviceContext) agentHost() string { + if dc.AgentHost != "" { + return dc.AgentHost + } + return "127.0.0.1" +} + // reconnectDelay returns the configured reconnect delay or the default. func (dc *DeviceContext) reconnectDelay() time.Duration { if dc.ReconnectDelay > 0 { @@ -379,7 +388,7 @@ func (dc *DeviceContext) Reconnect(ctx context.Context) (probelink.ProbeClient, } dialOpts := probelink.DialOptions{ - Host: "127.0.0.1", + Host: dc.agentHost(), Port: dc.Port, Token: token, DialTimeout: dc.dialTimeoutVal(), @@ -406,6 +415,44 @@ func (dc *DeviceContext) Reconnect(ctx context.Context) (probelink.ProbeClient, return client, nil } +// ReconnectWithToken reconnects using a pre-shared token (set via set_next_token +// before restart). This skips token reading from device logs — critical for WiFi +// mode where idevicesyslog is unavailable. +func (dc *DeviceContext) ReconnectWithToken(ctx context.Context, token string) (probelink.ProbeClient, error) { + // Wait for the app to restart and the agent to initialize + time.Sleep(dc.reconnectDelay()) + + dialOpts := probelink.DialOptions{ + Host: dc.agentHost(), + Port: dc.Port, + Token: token, + DialTimeout: dc.dialTimeoutVal(), + } + + // For WiFi: use the original host (not 127.0.0.1) + // The host is embedded in the HTTPClient's baseURL, but for reconnect + // we need to try multiple times as the app is still booting + deadline := time.Now().Add(dc.tokenTimeout()) + for time.Now().Before(deadline) { + var client probelink.ProbeClient + var err error + if dc.UseHTTP { + client, err = probelink.DialHTTP(ctx, dialOpts) + } else { + client, err = probelink.DialWithOptions(ctx, dialOpts) + } + if err == nil { + return client, nil + } + select { + case <-ctx.Done(): + return nil, ctx.Err() + case <-time.After(2 * time.Second): + } + } + return nil, fmt.Errorf("reconnect with pre-shared token: agent not reachable within %s", dc.tokenTimeout()) +} + // iosTokenPath returns the path to the agent's token file on the simulator. // It checks the app container first (where the agent actually writes), then // falls back to the device-level tmp path. diff --git a/internal/runner/executor.go b/internal/runner/executor.go index 7615939..b8dbe75 100644 --- a/internal/runner/executor.go +++ b/internal/runner/executor.go @@ -2,6 +2,8 @@ package runner import ( "context" + "crypto/rand" + "encoding/hex" "fmt" "io" "net/http" @@ -14,6 +16,13 @@ import ( "github.com/alphawavesystems/flutter-probe/internal/visual" ) +// generateToken creates a random 32-character hex token for pre-shared restart tokens. +func generateToken() string { + b := make([]byte, 16) + rand.Read(b) + return hex.EncodeToString(b) +} + // Executor walks an AST body and dispatches commands to a ProbeLink client. type Executor struct { client probelink.ProbeClient @@ -423,13 +432,28 @@ func (e *Executor) runAction(ctx context.Context, a parser.ActionStep) error { fmt.Println(" \033[33m⚠\033[0m Skipping restart (cloud mode — not supported without ADB/simctl)") return nil } + // Pre-share a token before restart so we can reconnect without + // reading device logs (critical for WiFi mode where idevicesyslog + // is unavailable). The agent persists it and uses it after restart. + nextToken := generateToken() + if err := e.client.SetNextToken(ctx, nextToken); err != nil { + // Non-fatal: fall back to normal token reading + fmt.Printf(" \033[33m⚠\033[0m pre-share token: %v (will read from logs)\n", err) + nextToken = "" + } e.client.Close() if err := e.deviceCtx.RestartApp(ctx); err != nil { return err } - newClient, err := e.deviceCtx.Reconnect(ctx) - if err != nil { - return fmt.Errorf("restart the app: %w", err) + var newClient probelink.ProbeClient + var reconnErr error + if nextToken != "" { + newClient, reconnErr = e.deviceCtx.ReconnectWithToken(ctx, nextToken) + } else { + newClient, reconnErr = e.deviceCtx.Reconnect(ctx) + } + if reconnErr != nil { + return fmt.Errorf("restart the app: %w", reconnErr) } e.client = newClient if e.onReconnect != nil { diff --git a/probe_agent/lib/src/executor.dart b/probe_agent/lib/src/executor.dart index 0b79e2f..412ddf9 100644 --- a/probe_agent/lib/src/executor.dart +++ b/probe_agent/lib/src/executor.dart @@ -83,6 +83,15 @@ class ProbeExecutor { ); return {'ok': true}; + case ProbeMethods.setNextToken: + final token = req.params['token'] as String? ?? ''; + if (token.length < 16) { + throw ProbeError(ProbeError.invalidParams, 'Token must be at least 16 characters'); + } + // Access server via global to persist the token + await _persistNextToken(token); + return {'ok': true}; + // ---- Navigation ---- case ProbeMethods.open: final screen = req.params['screen'] as String? ?? ''; @@ -696,6 +705,27 @@ class ProbeExecutor { _send(ProbeNotification(ProbeMethods.notifyRestartApp, {}).encode()); await Future.delayed(const Duration(milliseconds: 500)); } + + /// Persists a token to disk so the agent uses it after restart. + Future _persistNextToken(String token) async { + try { + String path; + if (Platform.isIOS) { + path = '${Directory.systemTemp.path}/probe/next_token'; + } else if (Platform.isAndroid) { + final cmdline = File('/proc/self/cmdline').readAsStringSync(); + final pkg = cmdline.split('\x00').first; + path = '/data/data/$pkg/cache/probe/next_token'; + } else { + path = '${Directory.systemTemp.path}/probe/next_token'; + } + final file = File(path); + await file.parent.create(recursive: true); + await file.writeAsString(token); + } catch (e) { + throw ProbeError(ProbeError.internalError, 'Failed to persist token: $e'); + } + } } // ---- Minimal gesture wrapper ---- diff --git a/probe_agent/lib/src/protocol.dart b/probe_agent/lib/src/protocol.dart index bfd336e..0c8b596 100644 --- a/probe_agent/lib/src/protocol.dart +++ b/probe_agent/lib/src/protocol.dart @@ -116,6 +116,7 @@ class ProbeMethods { static const copyClipboard = 'probe.copy_clipboard'; static const pasteClipboard = 'probe.paste_clipboard'; static const verifyBrowser = 'probe.verify_browser'; + static const setNextToken = 'probe.set_next_token'; // Notification methods (agent → CLI) static const notifyRecordedEvent = 'probe.recorded_event'; diff --git a/probe_agent/lib/src/server.dart b/probe_agent/lib/src/server.dart index 7ad8f5d..2967e2b 100644 --- a/probe_agent/lib/src/server.dart +++ b/probe_agent/lib/src/server.dart @@ -34,8 +34,11 @@ class ProbeServer { Timer? _tokenTimer; /// Starts the WebSocket server and prints the session token. + /// If a pre-shared token was persisted (via `set_next_token`), uses that + /// instead of generating a random one. This enables reconnection after + /// `restart the app` in WiFi mode where the CLI can't read device logs. Future start() async { - _token = _generateToken(); + _token = await _readPersistedToken() ?? _generateToken(); final bindAddress = allowRemoteConnections ? InternetAddress.anyIPv4 : InternetAddress.loopbackIPv4; @@ -210,6 +213,48 @@ class ProbeServer { ProbeExecutor? get executor => _executor; + /// Persists a token to disk so the agent uses it after restart. + /// Called by the CLI via `probe.set_next_token` before `restart the app`. + Future setNextToken(String token) async { + try { + final file = File(_nextTokenPath()); + await file.parent.create(recursive: true); + await file.writeAsString(token); + // ignore: avoid_print + print('ProbeAgent: next token persisted for restart'); + } catch (e) { + // ignore: avoid_print + print('ProbeAgent: failed to persist next token: $e'); + } + } + + /// Reads a persisted next-token from disk (set before restart). + /// Deletes the file after reading so it's only used once. + Future _readPersistedToken() async { + try { + final file = File(_nextTokenPath()); + if (await file.exists()) { + final token = (await file.readAsString()).trim(); + await file.delete(); + if (token.length >= 16) { + // ignore: avoid_print + print('ProbeAgent: using pre-shared token from restart'); + return token; + } + } + } catch (_) {} + return null; + } + + String _nextTokenPath() { + if (Platform.isIOS) { + return '${Directory.systemTemp.path}/probe/next_token'; + } else if (Platform.isAndroid) { + return '/data/data/${_resolvePackageName()}/cache/probe/next_token'; + } + return '${Directory.systemTemp.path}/probe/next_token'; + } + static String _generateToken() { const chars = 'abcdefghijklmnopqrstuvwxyz0123456789'; final rng = Random.secure(); From 05290698c3d6194a40b4652c3d2a0cd02c0f8b69 Mon Sep 17 00:00:00 2001 From: Patrick Bertsch Date: Thu, 26 Mar 2026 02:23:34 -0600 Subject: [PATCH 5/7] chore: bump version to 0.5.1, update docs and CLAUDE.md --- CHANGELOG.md | 21 ++++++++ CLAUDE.md | 85 ++++++++++++++++++++++++++++++ docs/wiki/Home.md | 2 +- docs/wiki/iOS-Integration-Guide.md | 2 + vscode/package.json | 2 +- website/src/pages/index.astro | 2 +- 6 files changed, 111 insertions(+), 3 deletions(-) create mode 100644 CLAUDE.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 1ff7c45..c055724 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,27 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). ## [Unreleased] +## [0.5.1] - 2026-03-26 + +### Added + +- Pre-shared restart token (`probe.set_next_token`) — CLI sends a token to the agent before `restart the app`; agent persists it and uses it after restart, enabling WiFi reconnection without `idevicesyslog` +- `--host` flag for WiFi testing — connect directly to device IP, no iproxy needed +- `--token` flag to skip USB-dependent token auto-detection +- `PROBE_WIFI=true` dart-define — binds agent to `0.0.0.0` for network access +- HTTP POST fallback transport (`POST /probe/rpc`) — stateless per-request communication for physical devices +- `ProbeClient` interface — both WebSocket and HTTP clients satisfy it for transport-agnostic execution +- `tap "X" if visible` ProbeScript syntax — silently skips when widget is not found; works with tap, type, clear, long press, double tap +- Direct `onTap` invocation fallback for `Semantics`-wrapped `GestureDetector` widgets on physical devices +- `take screenshot "name"` now accepts name directly (no `called` keyword needed) +- Physical device E2E test suite for FlutterProbe Test App (12 tests covering all 10 screens) + +### Fixed + +- `clear app data` on physical iOS now skips immediately (before confirmation prompt) to avoid killing the agent +- Connection error detection in `if visible` — propagates connection errors for auto-reconnect instead of silently swallowing them +- Screenshot parser accepts `take screenshot "name"` without requiring `called` keyword + ## [0.5.0] - 2026-03-26 ### Added diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..79bb477 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,85 @@ +# CLAUDE.md — FlutterProbe Development Guide + +## Project Overview + +FlutterProbe is a Go CLI + Dart agent E2E testing framework for Flutter apps. Tests are written in ProbeScript (`.probe` files) — natural language syntax. The CLI orchestrates device connections, parses tests, dispatches commands via WebSocket/HTTP to the on-device Dart agent. + +## Architecture + +``` +Go CLI (cmd/probe) <-- WebSocket/HTTP --> Dart Agent (probe_agent) + | | + ├── parser/ AST from .probe files ├── server.dart WS + HTTP server + ├── runner/ Test orchestration ├── executor.dart Command dispatch + ├── probelink/ JSON-RPC 2.0 client ├── finder.dart Widget tree search + ├── device/ ADB + iproxy management ├── sync.dart Triple-signal sync + ├── ios/ simctl + devicectl └── recorder.dart Test recording + ├── config/ probe.yaml parsing + └── report/ HTML/JUnit/JSON output +``` + +## Build & Test + +```bash +go build ./... # Build CLI +go test ./... # Run all Go tests (115+) +go build -o /tmp/probe-test ./cmd/probe # Build binary for testing +``` + +## Key Development Rules + +### Git +- **Email**: patrick@alphawavesystems.com for all commits +- **Branches**: Always feature branches + PRs, never push to main +- **Docs**: Every PR must update CHANGELOG.md, wiki, and landing page version + +### Physical Device Testing +- **Profile mode + flavor**: Debug builds can't cold-launch from home screen. Use `flutter build ios --profile --flavor --dart-define=PROBE_AGENT=true` +- **WiFi > USB**: USB-C causes intermittent drops. Use `--host --token --dart-define=PROBE_WIFI=true` +- **No `clear app data`**: Unsupported on physical iOS. Tests must be self-contained with explicit login/logout +- **`tap "X" if visible`**: Use instead of verbose `dismiss dialogs` recipes +- **Semantics + GestureDetector**: Put `ValueKey` on the `GestureDetector`, not the `Semantics` wrapper + +### Flutter Widget Tree Gotchas +- **Navigator keeps back routes in tree**: `if "X" appears` matches widgets on background routes. Don't use it to detect which screen is active. +- **Visibility filtering**: `Offstage`/`Visibility` checks don't catch Navigator route hiding (uses opacity, not Offstage) + +### Test Patterns for Physical Devices +- Each test starts from known state (login screen) and cleans up explicitly +- Use `do logout` + `wait until "Usuario" appears` at end of tests that log in +- No `restart the app` in middle of WiFi test suites (new token needed after restart) +- `tap "Aceptar" if visible` for dialog dismissal (2 lines vs 20-line recipe) + +## Transport Modes + +| Mode | When | How | +|---|---|---| +| WebSocket | Simulators, emulators | Default — persistent connection, ping/pong keepalive | +| HTTP POST | Physical devices (USB) | Auto-selected — stateless, no connection to drop | +| HTTP POST | Physical devices (WiFi) | `--host --token ` — zero drops | + +## Release Checklist + +1. `CHANGELOG.md` — new version section +2. `vscode/package.json` — bump version +3. `website/src/pages/index.astro` — update version badge +4. `docs/wiki/Home.md` — update current version +5. `git tag v0.X.Y && git push origin v0.X.Y` — triggers release workflow + +## File Locations + +| What | Where | +|---|---| +| CLI entry | `cmd/probe/main.go` | +| Test command | `internal/cli/test.go` | +| Parser | `internal/parser/{lexer,parser,ast,token}.go` | +| Runner | `internal/runner/{executor,runner,device_context}.go` | +| WS Client | `internal/probelink/client.go` | +| HTTP Client | `internal/probelink/http_client.go` | +| Interface | `internal/probelink/iface.go` | +| Device mgmt | `internal/device/{manager,adb,permissions}.go` | +| iOS tools | `internal/ios/{simctl,devicectl}.go` | +| Dart agent | `probe_agent/lib/src/{server,executor,finder}.dart` | +| Config | `internal/config/config.go` | +| Landing page | `website/src/pages/index.astro` | +| Wiki docs | `docs/wiki/*.md` | diff --git a/docs/wiki/Home.md b/docs/wiki/Home.md index 97bc97b..efb6150 100644 --- a/docs/wiki/Home.md +++ b/docs/wiki/Home.md @@ -17,7 +17,7 @@ Welcome to the FlutterProbe wiki. This documentation covers architecture details ## Project Status -FlutterProbe is in active development. Current version: **0.5.0**. +FlutterProbe is in active development. Current version: **0.5.1**. ### Repository Structure diff --git a/docs/wiki/iOS-Integration-Guide.md b/docs/wiki/iOS-Integration-Guide.md index f98ca06..06955b5 100644 --- a/docs/wiki/iOS-Integration-Guide.md +++ b/docs/wiki/iOS-Integration-Guide.md @@ -165,6 +165,8 @@ probe test tests/ --host --token --device To find the token, check the app's console output for `PROBE_TOKEN=...` (printed every 3 seconds). +**`restart the app` over WiFi**: The CLI automatically pre-shares a token with the agent before restarting. After restart, the agent uses the pre-shared token instead of generating a new one — no manual token re-reading needed. + ### How It Works 1. **USB**: `iproxy` forwards port 48686; CLI reads token via `idevicesyslog`; HTTP POST transport diff --git a/vscode/package.json b/vscode/package.json index 158575a..a45bd7d 100644 --- a/vscode/package.json +++ b/vscode/package.json @@ -2,7 +2,7 @@ "name": "flutterprobe", "displayName": "FlutterProbe", "description": "High-performance E2E testing for Flutter apps — ProbeScript language support, local & cloud device testing (BrowserStack, Sauce Labs, AWS Device Farm, Firebase Test Lab, LambdaTest), visual regression, test recording, and Studio integration", - "version": "0.5.0", + "version": "0.5.1", "publisher": "flutterprobe", "icon": "resources/probe-icon.png", "engines": { "vscode": "^1.85.0" }, diff --git a/website/src/pages/index.astro b/website/src/pages/index.astro index c8f1dcc..5c9dfd2 100644 --- a/website/src/pages/index.astro +++ b/website/src/pages/index.astro @@ -429,7 +429,7 @@
-
v0.5.0 — BSL 1.1 open source
+
v0.5.1 — BSL 1.1 open source

Flutter E2E tests in
plain English

Write tests your whole team can read. Execute with direct widget-tree access — no accessibility layer, no WebDriver overhead.

From 71c5d4eb1fa7e49c8790294cdfd4e83a59436db8 Mon Sep 17 00:00:00 2001 From: Patrick Bertsch Date: Thu, 26 Mar 2026 02:26:09 -0600 Subject: [PATCH 6/7] fix: remove unnecessary nil check around range (staticcheck S1031) --- internal/device/manager.go | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/internal/device/manager.go b/internal/device/manager.go index 814d412..0a54dfe 100644 --- a/internal/device/manager.go +++ b/internal/device/manager.go @@ -87,10 +87,8 @@ func (m *Manager) List(ctx context.Context) ([]Device, error) { physicalUDIDs, err := ios.ListPhysicalDevices(ctx) if err == nil { simUDIDs := make(map[string]bool) - if sims != nil { - for _, s := range sims { - simUDIDs[s.UDID] = true - } + for _, s := range sims { + simUDIDs[s.UDID] = true } for _, udid := range physicalUDIDs { if !simUDIDs[udid] { // avoid duplicates if somehow listed in both From 4a2bb513b437f20ec48bbd3f610a94afbf1f9bc4 Mon Sep 17 00:00:00 2001 From: Patrick Bertsch Date: Thu, 26 Mar 2026 02:37:12 -0600 Subject: [PATCH 7/7] chore: remove CLAUDE.md from tracking (already in .gitignore) --- CLAUDE.md | 85 ------------------------------------------------------- 1 file changed, 85 deletions(-) delete mode 100644 CLAUDE.md diff --git a/CLAUDE.md b/CLAUDE.md deleted file mode 100644 index 79bb477..0000000 --- a/CLAUDE.md +++ /dev/null @@ -1,85 +0,0 @@ -# CLAUDE.md — FlutterProbe Development Guide - -## Project Overview - -FlutterProbe is a Go CLI + Dart agent E2E testing framework for Flutter apps. Tests are written in ProbeScript (`.probe` files) — natural language syntax. The CLI orchestrates device connections, parses tests, dispatches commands via WebSocket/HTTP to the on-device Dart agent. - -## Architecture - -``` -Go CLI (cmd/probe) <-- WebSocket/HTTP --> Dart Agent (probe_agent) - | | - ├── parser/ AST from .probe files ├── server.dart WS + HTTP server - ├── runner/ Test orchestration ├── executor.dart Command dispatch - ├── probelink/ JSON-RPC 2.0 client ├── finder.dart Widget tree search - ├── device/ ADB + iproxy management ├── sync.dart Triple-signal sync - ├── ios/ simctl + devicectl └── recorder.dart Test recording - ├── config/ probe.yaml parsing - └── report/ HTML/JUnit/JSON output -``` - -## Build & Test - -```bash -go build ./... # Build CLI -go test ./... # Run all Go tests (115+) -go build -o /tmp/probe-test ./cmd/probe # Build binary for testing -``` - -## Key Development Rules - -### Git -- **Email**: patrick@alphawavesystems.com for all commits -- **Branches**: Always feature branches + PRs, never push to main -- **Docs**: Every PR must update CHANGELOG.md, wiki, and landing page version - -### Physical Device Testing -- **Profile mode + flavor**: Debug builds can't cold-launch from home screen. Use `flutter build ios --profile --flavor --dart-define=PROBE_AGENT=true` -- **WiFi > USB**: USB-C causes intermittent drops. Use `--host --token --dart-define=PROBE_WIFI=true` -- **No `clear app data`**: Unsupported on physical iOS. Tests must be self-contained with explicit login/logout -- **`tap "X" if visible`**: Use instead of verbose `dismiss dialogs` recipes -- **Semantics + GestureDetector**: Put `ValueKey` on the `GestureDetector`, not the `Semantics` wrapper - -### Flutter Widget Tree Gotchas -- **Navigator keeps back routes in tree**: `if "X" appears` matches widgets on background routes. Don't use it to detect which screen is active. -- **Visibility filtering**: `Offstage`/`Visibility` checks don't catch Navigator route hiding (uses opacity, not Offstage) - -### Test Patterns for Physical Devices -- Each test starts from known state (login screen) and cleans up explicitly -- Use `do logout` + `wait until "Usuario" appears` at end of tests that log in -- No `restart the app` in middle of WiFi test suites (new token needed after restart) -- `tap "Aceptar" if visible` for dialog dismissal (2 lines vs 20-line recipe) - -## Transport Modes - -| Mode | When | How | -|---|---|---| -| WebSocket | Simulators, emulators | Default — persistent connection, ping/pong keepalive | -| HTTP POST | Physical devices (USB) | Auto-selected — stateless, no connection to drop | -| HTTP POST | Physical devices (WiFi) | `--host --token ` — zero drops | - -## Release Checklist - -1. `CHANGELOG.md` — new version section -2. `vscode/package.json` — bump version -3. `website/src/pages/index.astro` — update version badge -4. `docs/wiki/Home.md` — update current version -5. `git tag v0.X.Y && git push origin v0.X.Y` — triggers release workflow - -## File Locations - -| What | Where | -|---|---| -| CLI entry | `cmd/probe/main.go` | -| Test command | `internal/cli/test.go` | -| Parser | `internal/parser/{lexer,parser,ast,token}.go` | -| Runner | `internal/runner/{executor,runner,device_context}.go` | -| WS Client | `internal/probelink/client.go` | -| HTTP Client | `internal/probelink/http_client.go` | -| Interface | `internal/probelink/iface.go` | -| Device mgmt | `internal/device/{manager,adb,permissions}.go` | -| iOS tools | `internal/ios/{simctl,devicectl}.go` | -| Dart agent | `probe_agent/lib/src/{server,executor,finder}.dart` | -| Config | `internal/config/config.go` | -| Landing page | `website/src/pages/index.astro` | -| Wiki docs | `docs/wiki/*.md` |