GUI automation

Zhentao Li will present "Sikuli: Using GUI Screenshots for Search and Automation" by Yeh, Chang and Miller.

Sikuli is a tool for automating repetitive tasks through GUIs. To automate a program, it makes use of screenshots and image recognition to decide where to click and type. Tasks are scripted in Python using Sikuli's IDE which makes taking and viewing screenshot easy. Using screenshots gives the user a more natural flow from manual task to automation, and provides the same API across multiple programs.

The paper also present a screenshot based search engine but the presentation will focus on the automation portion. I will also show how to reimplement a simple version of Sikuli.

Please note that we are in Café OSMO and not the Clark room this time. The event starts at 6:30pm (instead of 6pm) but Café OSMO is open to the public so feel free to go there earlier.