Testing in a Velocity Culture

Delivering business value to customers rapidly is on the forefront more than ever before. Technical practices such as continuous delivery and business approaches such as the lean startup movement continue to push this to the extreme. Testing in these “velocity” cultures presents unique challenges and requires implementing some non-conventional approaches to traditional testing methods and practices. How does testing function in this type of environment? What is required to be successful? John Esser, Director of Engineering Productivity at Ancestry.com, will share his insights and experiences from Ancestry’s move to continuous delivery and how testing and his test team had to evolve in their velocity culture.

Join or login to comment.

  • Tony G.

    The presentation was done by someone who clearly does not understand the role of quality. If the overall messages is that QA should become Developers I would hope we could find a better message. QA knowing how to code is a good message, but Developers replacing QA??? You might as well have the Developers gather requirements too. This is a message that has been presented since the 1990 when XP came out yet year after year the value of having QA wins. There are good reasons to have BA, QA and Dev roles.

    1 · January 23, 2013

    • Tony G.

      Thanks John. If you present again at the QA Meet up, I would be interested in hearing more of a deep dive in this area. Seeing the full picture of how Ancestry.com tests I could see how this shift could be effective.

      January 29, 2013

    • Tom

      This part scared me "the TEs also coordinate offshore testers that can do manual regressions as stop gap." I personally believe you need people who understand the purpose of the software in order to test effectively. Unless your offshore people are part of the team do they really understand the purpose of the software? Do they understand what the customer needs? Or are they told to click on a , select b, press c?

      February 9, 2013

  • Paul P.

    John's Velocity Powerpoint presentation is available on Meetup. Go to More | Files menu and download John Esser's Presentation

    1 · January 30, 2013

  • Annette A.

    Here is a really good blog that I believe is the missing link to this whole discussion. It comes from the Google Test engineers on what exactly does the Test Engineers do at Google. It aligns nicely with Lisa Crispin and Janet Gregory's description in their book Agile Testing. This is a very strong recommendation for everyone to read if they haven't already.

    2 · January 29, 2013

    • Chris O.

      Thanks for the link

      January 29, 2013

    • Tony G.

      Thanks Annette, this link answers a lot of questions.

      January 29, 2013

  • Annette A.

    1 · January 29, 2013

  • Kyle H.

    Can the presentation slides be posted where we can review them? I'd like to review and share some of the concepts with my team.

    1 · January 25, 2013

    • John E.

      I sent them to Paul for posting. Have fun!

      1 · January 28, 2013

    • Dean W.

      Did this ever get posted?

      January 29, 2013

  • Annette A.

    http:google testing.blogspot.com/2013/01/test-engineers-google.html

    1 · January 29, 2013

    • Bing

      Never mind, I found it. Thanks

      January 29, 2013

  • Bing

    This was the first time that I attended the meetup, very impressed with the presentation as well as the participation of others in their comments after the meetup. Great to be there.

    January 29, 2013

  • John E.

    In hindsight because I hit the topic hard about Devs owning test automation that somehow translated into devs becoming QA or not needing QA then that was my fault. I do believe "human testing" needs to be "high value." The manual testing should be what computers can't do like determining the quality picture, usability, and that we are building the right functionality. I have seen so many test folks relegated to running menial regressions, spending gobs of time maintaining tests and test frameworks because the devs think their only responsibility is unit level testing. Sometimes I think testers enable this poor behavior in devs because they have been conditioned to accept that their value-add is running regressions, finding bugs that developers should have prevented, and just cleaning up after developers in general. The whole "conventional" approach slows everything down and actually leads to poorer quality. So, yes, am I trying to redefine the "traditional" testing model/approach.

    2 · January 28, 2013

    • Tony G.

      John, I applaud your effort and desire to improve and redefine traditional testing. Challenging tried and true methods can’t be easy. This model challenges methods proven by testing legends like Dorothy Graham, Janet Gregory, Lisa Crispin and Lee Copeland. People who have published tried and proven methods which have stood up in many organizations loaded with case studies showing improved quality, including my own personal experiences, using their methods, to see improved quality. For me it would help to see how the velocity culture model improves or delivers better quality then these traditional methods if it is going to challenge them. Best of luck in your efforts and desires.

      1 · January 28, 2013

    • John E.

      Yes, this is all on a continuum. My objective was not only to improve quality, but to also improve/maintain it in the context of a velocity culture. I believe the traditional approach and the methods advocated by the folks you mentioned are effective at improving quality--if not, they wouldn't exist. My bet, however, is that if you look at a velocity organizations you will find a testing setup looking more like the one I describe--at least that is what I have observed. Thanks for the feedback.

      1 · January 29, 2013

  • David R.

    I am familiar with the material. He didn't share much of anything that wasn't already taught in star west 2011. He wasn't a great presenter, seemed really nervous.

    January 23, 2013

    • Dean W.

      I would agree that the presentation style made you appear somewhat nervous. Perhaps too many "uh"s and "um"s, but that may be your communication style. I knew one gentleman who tended to do that about every 5 words or so, and he never knew he was doing it. (Record and listen to yourself perhaps?)

      January 28, 2013

    • John E.

      Great suggestion. Thanks.

      January 28, 2013

  • John E.

    Btw, I thought it was a great meetup and I will make a point to attend more consistently in the future.

    1 · January 28, 2013

  • Mike V.

    I think some good ideas were brought up. I've read the how Google Tests Software and I think the approach for dev writing all tests takes Google's approach too far and I think it was misrepresented that this is the practice that Google uses.

    According to the book Software Engineers in Test(SET) 'refactor code to make it more testable and write unit testing frameworks and automation.

    Test Engineers (TE)
    'spend a good deal of their time writing code in the form of automation scripts and code that drives usage scenarios and even mimics the user.'

    A practice that I do like that Google follows is that if a developer writes new code that breaks any test, it is then the developers responsibility to fix the test, regardless if it's an unit test or automation test. This helps distribute the work and the developer that broke the test would be more familiar with what would be needed to fix the test.

    1 · January 23, 2013

    • Chris O.

      Thanks for the clarification Mike.

      I think your last paragraph and the comment that developers help repair automation code is part of the reasoning behind John's argument to have them write all automation code. Because if they're not invested in writing it to begin with it may be in a language/format they're not familiar with and even if they are, they may have to come up to speed on the architecture, thus slowing things down.

      Still, I personally prefer having the test team more involved than it sounds like they are at ancestry.

      January 23, 2013

    • John E.

      I didn't mean to imply that this is what Google does, but rather that I received a lot of inspiration from Google's model. Our TEs dont' write much automation code, but spend more time focusing on user concerns and overall quality risk. We continue to work on shifting our SET model because in the beginning our SETs were writing the test automation and would do it in a way that the Devs didn't understand, couldn't write tests for, etc. Once we changed SETs to be enablers of code testability and put ownership of actually writing the automated tests, the quality of test automation went up and the tests got faster.

      1 · January 28, 2013

  • Kyle H.

    Good presentation. Lots of ideas for improving my processes.

    January 23, 2013

  • Tom

    A good topic. Made me realize the trend towards even faster releases. I felt a 2 week sprint with a release afterwards was a good metric but the presenter made a good case where in time the pressures of more features even faster will drive release cycles to be even quicker than every 2 weeks. I then had to review our processes and look into what changes we would need to make in order to handle releases even faster. So it made me think and that means it was good.

    January 23, 2013

  • Chris O.

    Thanks for a great presentation. I loved the concept of moving beyond "Agility" to a "Culture" of velocity. While our current development team model is not structured to fully adopt all of the concepts presented, I look forward to presenting the "10 Commandments" and discussing how we can move towards the methodologies therein.

    January 23, 2013

  • Renee H.

    I really enjoyed it. It was a great reminder of things I had learned previously but had forgotten a bit.

    1 · January 23, 2013

  • Don B.

    Great presentation, thanks.

    January 22, 2013

  • Dan P.

    Great topic, interesting testing strategy done with the help of Development Engineers.

    January 22, 2013

  • Tim F.

    Great information was presented. I was enlightened and had an enjoyable time listening to this important topic. I will be attending again.

    January 22, 2013

  • Chris O.

    I haven't attended one of your events before and I'm unfamiliar with the IHC Center. Can you provide general directions to the "education center" and an idea of where would be best to park?

    1 · January 7, 2013

  • Paul P.

    Looking forward to this month's meeting. Great speaker and timely topic. Also, don't forget to enter the contest for a FREE Star Conference registration.

    1 · January 3, 2013

Our Sponsors

  • TEKsystems

    TEKsystems a long time sponsor providing food and drinks for meetings.

  • Software Quality Engineering

    A 10% discount to ANY 2015 SQE Conference. Promo Code: UTAQA

People in this
Meetup are also in:

Imagine having a community behind you

Get started Learn more
Bill

I started the group because there wasn't any other type of group like this. I've met some great folks in the group who have become close friends and have also met some amazing business owners.

Bill, started New York City Gay Craft Beer Lovers

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy