New Zealand experience report presented at Workshop on Performance and Reliability

The 20th Workshop on Performance and Reliability, New York, May 2013

In mid-May 2013 I joined colleague Richard Leeke for the 20th Workshop on Performance and Reliability (WOPR) in the Garment District of the Big Apple.

Richard is Principal Consultant, Performance Engineering Practice at Equinox IT. He’s a world recognised expert in performance testing, and previously served as a WOPR Content Owner. His experiential paper, Pushing the Limits of Testing Tools, was accepted into the WOPR20 programme.

About WOPR

WOPR is a series of peer workshops for performance engineers & testers, operations planners, development managers, and related professionals. In the workshops, subject matter experts and practitioners share experiences which are then used to inform group discussions about systems performance, reliability, scalability and recoverability.

The workshops emphasise mutual learning, sharing experiences and discussing problems and solutions.

With a 30-participant limit, each speaker at these twice-yearly WOPR events presents a project experience. A facilitated discussion follows to distil the experience and the issues that it raises. These discussions sometimes continue for several hours.

At WOPR20, participants from Belgium, the Netherlands and New Zealand joined attendees from North America. Ross Collard, Content Owner for WOPR20, selected the theme, The Leading Edge of Performance and Reliability.

Highlights of WOPR20

Presentations at WOPR20 included discussions about modelling performance at extremely large scales, testing in the Cloud, big data platforms, mobile analytics, and a report on the on-going development of a new tooling approach. Distilling those experiences led to conversations about tooling and instrumentation methodology, modelling reliability in software systems, and the role and nature of testing in a continuous-release world.

Papers I particularly liked included Facebook’s Capacity Planning Engineer, Goranka Bjedov, talking about how Facebook Heat Maps Pinpoint Network's Trouble Spots when for example Hurricane Sandy hit, and Julian Harty’s paper on using Mobile Analytics Results for Quality Improvement and the issues of privacy around data acquisition from apps.

Goranka kindly complemented Richard on his work, “… the problems Richard solves makes my job look easy!”, and she told us “New Zealand is a frequent test bed for new Facebook features.”

Richard Leeke - Pushing the Limits of Testing Tools

Richard’s experiential paper told the story of grafting performance testing onto a government website.

Equinox IT evaluated eight test tools on paper before doing a proof of concept that eventually resolved the desired direction.

Richard’s team within Equinox IT then built a framework to enhance the capability of Microsoft Visual Studio Load Testing in order to support the application being tested. An experienced performance tester could use this framework to design and develop test scripts.

The outcome was that the process reduced the time to create a test suite from 600 hours to 150, while the time to re-record for a new build dropped down to 15 hours or so. This means a test run can happen in days, not weeks and resulted in significant savings and improved performance.

Summary

WOPR is an intense, professionally satisfying, inspiring workshop with an open format and the stimulating company of peers. For those of us working as trusted advisers diagnosing performance problems and advising clients about risk mitigation, stability, capability and capacity, WOPR20 delivered highly-respected insights from expert practitioners.