I’m a member of an online community of testers called
uTest (the practitioner-facing side of
Applause). The company hosts a social network for testers as
well as offers short-term gigs to its users (the cliched Uber for testers). I was
called on this weekend for my first gig: testing a payment method in Taxis. It was pleasant, if stressful, and it made me
think of ways a company could take advantage of it to expand the perspective
on their products.
After the initial invite to the test cycle, I communicated with a project manager via Skype to ensure I
was able to carry out the test scenarios.
I brought to the table a specific model of phone and a verified ability
to pull debug logs from it
(thanks Verizon for turning off the op codes on my S4; I resorted to adb logcat). They
provided technical assistance and reimbursement for the transactions, but the primary
incentive was a reward for test cases completed.
Throughout,
I felt like a skilled secret shopper rather than a functional tester. I was asked more about the non-software
components of the launch than the app or phone functionality. I reported on the advertisement of the
feature, the education of the employees, and the capability of the equipment that supports the feature. In
spite of expectations from the participating companies that the advertisement would match hardware would match training, this happened 0% of the
time, and no employee had been informed why their equipment had been updated. I wasn’t the only tester in this project on the ground, and
the others testers saw related issues, and none had all their ducks
in a row. In all, the information they were most excited about was the boots-on-the-ground report I provided. It was fascinating to see a live product launch from that perspective, and doubly so considering my history in
this product space.
The final bulk of time spent on this gig was an experience report. Complete with detailed feedback, photos of the equipment, and other evidence, this is where I was able to
comment on the process as a whole.
From a testing perspective, I was able to provide detailed UI/UX
feedback, evaluate the consistency of the process, and help them understand how
much of their plans actually came off during the launch period. There was some repetition in these reports. One was a document with pictures, the other was a survey that linked to the doc and a third was a JIRA-in-spreadsheets for tracking testers. These reports were all submitted to the project manager for feedback, and I received an up/down "accepted" in less than 24 hours. While there is definitely room for improvement on the platform that would reduce tester overhead, it wasn't enough of a burden to avoid gigs.
Participation in this project taught me that coordination is
hard, education of low-skill workers is even harder, and successful coordinated nation-wide launches like this are next to impossible.
This mirrored my experience with other companies. There are always bits and pieces of projects that do not make their way down the chain to the workers that are face-to-face with customers, so having a boots-on-the-ground perspective is vital.
Overall, a company can leverage uTest services to add
layers of testing to their launch activities over and above internal
testing. Specific and targeted verification is where
they are the strongest. They can provide
feedback on as to whether an uneducated but motivated customer can make it through the new process right now. Specific errors and third-party verification can be farmed out for
feedback through such a relationship: things the company wants to do but can’t do efficiently given a small staff or tight timelines. This can provide a company with an enhanced view of
the product and help challenge their assumptions before their customers do.
Note: edited to be a little more coy about who I was testing for and what I was testing.
