CASE STUDY 01
Watches a phone basketball game's shot meter on a live screen feed and releases each shot on the green.
Watches a phone basketball game's shot meter on a live screen feed and releases each shot on the green.
The green release window is too tight to hit by hand every time, so I let software time the tap.
The phone only hardware-encodes its screen and receives touches. Everything in between runs on the PC.
adb screenrecord streams H.264 from the phone's hardware encoder.
PyAV turns the stream back into raw frames.
OpenCV finds the sweeping cursor and the green zone in the meter.
A frame-clock PLL, a clock bridge from encoder timestamps to PC time, linear cursor prediction and an adaptive latency term decide the exact frame to fire.
The release is written as evdev touch events.
A persistent root shell on the phone injects the touch.
Seeing the meter is the easy part. The shot has to fire on the correct frame despite capture latency, touch latency and frame jitter. The scheduler maps the phone encoder's timestamps onto the PC's clock, predicts where the cursor will be, and fires early by the measured latency so the cursor coasts onto the green.
It started as an on-phone Android app in Kotlin. Capture, analysis and timing on the device thermally throttled it and corrupted its own timing, so the loop moved to the PC. The phone now only encodes its screen and receives touches.
The original app's core logic sits behind functional interfaces, so it runs as plain JVM unit tests with no emulator. The Python shooter ships device-free self-checks that exercise the timing logic with no phone attached.
Reads the phone's H.264 screen stream and runs the release scheduler.
Finds the cursor and the green zone in the meter's bar row.
Carries both the screen capture and the injected touch to the phone.