Tactyle Touch
Your hand is the mouse. Pinch to click, hold to drag — the webcam does the rest.
Tactyle Touch watches one hand through the built-in camera and turns pinches and finger poses into real clicks, drags and scrolls anywhere on your Mac — moving the actual system cursor through Quartz, not just inside one window. No touchscreen, no glove, no extra hardware. It lives in the menu bar, and it's built and signed on your own machine, so there's nothing to notarize and nothing to wait for.
Open hand Move your open hand and the cursor follows the palm.
- Open hand → move the cursor
- Pinch & release (<250 ms) → click
- Pinch, hold & move → drag
- Two fingers, glide → scroll, with momentum
- Middle-finger-to-thumb pinch → right-click
- Three-finger swipe → switch Space (Ctrl+←/→)
- Thumbs-up, hold → start Dictation
- ILY 🤟, hold → stop tracking
A closed fist is a deliberate no-op — it also blocks a false pinch, so folding your hand never fires a click.
A webcam sees your hand shake. A mouse never does.
Turning a hand seen through a laptop camera into a usable pointer isn't a UI problem, it's a signal problem. Landmarks jitter frame to frame; smooth them too hard and the cursor lags your hand; and the thing you're aiming at — a 12-pixel checkbox, a traffic-light close button — is smaller than the noise. Most hand-tracking demos look magical for thirty seconds and then can't reliably click anything real.
Jitter
Raw landmarks wobble every frame. Map them 1:1 to pixels and the cursor never sits still long enough to click.
Latency
Kill the jitter with a heavy filter and the cursor drags behind your hand. Steady and responsive is the whole trick.
Small targets
The controls you want to hit are tiny. Without help, a free-hand cursor simply can't land on them — which is exactly what Precision Assist fixes.
A webcam, a hand model, and the same cursor everything else uses.
Nothing simulated and nothing sandboxed to one window: Tactyle Touch reads the camera, finds the hand, decides what pose it's in, and injects the resulting click or scroll as a real system event — the same CGEvent path a trackpad drives.
See the hand
MediaPipe Hand Landmarker returns 21 landmarks per frame at ~15–30 Hz. A One Euro filter smooths the jitter without adding lag you can feel.
Read the pose
A pure gesture state machine turns landmark geometry — pinch ratios, which fingers are extended — into eight distinct, debounced gestures. No ML in this layer, so it's fully unit-tested.
Move the Mac
The menu-bar app maps each gesture to a Quartz CGEvent — move, click, drag, scroll, Ctrl+arrow, Dictation — so it works system-wide, in every app.
A hand in the air can't hit a 12-pixel checkbox. So it doesn't have to.
Free-hand tracking is smooth but never pixel-perfect — the honest problem with every camera mouse. Precision Assist asks macOS's Accessibility API which elements on screen are actually clickable, then gently snaps the cursor to the nearest one as you approach. You aim at the button; it lands on the button.
Raw cursor drifts around a small target; assisted cursor is pulled onto it once it's close.
The targets aren't guessed. ax_targets.py walks the focused app's Accessibility tree for real clickable elements — buttons, checkboxes, links — and magnet.py applies a distance-weighted pull toward the closest one, strong up close and nothing at a distance, so normal movement never feels sticky.
It's a magnet for things you can actually click — not a grid, not a guess. Toggle it from the menu bar; it's off until you want it.
Small, honest, and yours to inspect.
One status item, a camera preview when you want it, and source you can read end to end.
Live camera preview
An optional menu-bar thumbnail shows the tracked hand skeleton, the pinch pair, and the active region — with mirroring and horizontal-invert toggles.
Momentum scrolling
Two-finger glides scroll with inertia that decays like a trackpad flick — the engine hands off a velocity and the app lets it coast.
Precision Assist
Magnetic snapping to real clickable elements via the Accessibility API. Off by default; one click in the menu to turn on.
Tunable, from env or JSON
Every threshold is a TOUCH_* knob — pinch hysteresis, debounce frames, scroll gain — read from settings.json or the environment.
Browser SDK too
A zero-dependency ES module renders a smoothed cursor and dispatches real pointer events on any web page — the same engine, no Mac permissions.
Private by construction
The hand model runs on-device; frames never leave your Mac. No account, no telemetry, no network at runtime.
One command, from a checkout.
The installer builds a self-contained TactyleTouch.app (embedded CPython), signs it, copies it to /Applications, and launches it.
$ git clone https://github.com/gokulmc/tactyle-touch.git && cd tactyle-touch && ./install.sh
First build takes a few minutes and downloads CPython (cached afterwards); the first camera run fetches the ~8 MB MediaPipe hand model to ~/.cache/tactyle-touch/. The finished .app weighs about 426 MB across 7,834 files — that's a whole embedded CPython plus the vision runtime, bundled so it never touches your system Python. Grant Accessibility (to move the cursor) and Camera when macOS asks — the two permissions it genuinely needs, and nothing more.