Epic fail: figuring out Selenium test results

Your Selenium test failed. Maybe your test environment has a glitch... or maybe your test isn't well designed... or maybe the application changed without you knowing it... or maybe, just maybe, your test found a bug.

Sometimes it's hard to tell what a browser test failure means. In this live, hands-on session we'll we will look at real examples running against real features in public test environments to take a close critical look at real browser test failures and what they mean.

We will present analysis techniques applicable to any test framework, but we will show specific sophisticated examples from our Cucumber and Page Object framework that uses RSpec assertions. Along the way we will examine how to set up builds of Selenium tests in Jenkins, and we'll look at test result steps archived at Sauce Labs with video playback of failed tests.

At the end of the session, participants will be able to reason about browser test failures with some authority, and will be familiar with how to work with the browser test failures that WMF makes public, in order to contribute to the discussion of quality at WMF: not only quality of applcations, but also of test environments and of test design.

Video stream: http://www.youtube.com/watch?v=fs2VmlQ_h7Q
IRC channel for questions: #wikimedia-dev (http://webchat.freenode.net/?channels=#wikimedia-dev) on Freenode

More: https://www.mediawiki.org/wiki/Meetings/2013-09-18