The problem
A desktop workflow can stall when a dialog moves, recognition is uncertain, or an application responds late. The operator needs to see what happened before deciding whether to continue.
Applied AI / 2026
Local Computer Use Agent
A Windows automation tool that reads the screen, logs each action, and stops for review when the workflow becomes uncertain.
My role
Windows automation, visual recognition, operator interface, and recovery logic.
What I built
- Windows operator interface
- Screen-recognition and action logs
- Dry-run and supervised recovery controls
The result
A local automation tool with tested components for screen recognition, action logging, and supervised recovery.
Statusactive
Evidencetested
Record reviewed
Implementation
How it works
The tool captures the screen, uses OCR and template matching to identify visible state, and chooses from a defined set of actions. Dry-run modes and recovery steps let the operator inspect the next action or stop when recognition is uncertain.
Technical context and methods
The Windows implementation includes an operator GUI, OCR and template recognition, local/OpenAI model routing, structured logs, and supervised recovery paths.
- Python
- Tkinter
- OCR
- Template matching
- Structured runtime logs
- Local/OpenAI routing
- Operator review gates