Tuesday, March 20, 2018

Behat AfterScenario, PHP Garbage Collection, and Singletons

In Behat, I added a singleton to our contexts to store things across scenarios, but I ran into trouble when trying to keep separation between my tests.  The storage object allowed me to be creative with builders, validators, and similar ways of reducing repetition and making the PHP code behind easier to read.  There was a problem though: it would randomly be cleared in the middle of a test.

The only thing I knew was the object would get cleared at relatively the same time.  I had a set of about 50 different tests in a single feature.  This would call an API multiple times, run validations on the responses, and then move on to the next test.  All the while, it would put information into the storage object.  The test would not just fail in the middle of a scenario, it would generally fail near the same part of a scenario every time.  it was timing, an async process, or something was clearing a logjam.

While designing the storage object, I had the bright idea to clear it with every scenario.  The singleton acts like a global variable, and a clear after each one would ensure data from one test didn't pop up in another.  To make sure i was running this at the last possible moment, I put the clear into the __destruct() method of my context class.  By putting the clear in the destructor, I gave PHP permission to handle it as it saw fit.  In reality, it sometimes left my scenario objects to linger while running the next (due to a memory leak or similar in Behat itself, or a problem in my code; I couldn't tell).

/**
 * Destructor
 */
public function __destruct()
{
 ApiContextStore::clear();
}


I first stopped clearing the store and the bugs went away.  Whew!  But how could I make sure I wasn't contaminating my tests with other data and sloppy design?  I tried two things:

1) gc_collect_cycles() forces the garbage collector to run.  This seems to have the same effect of stopping the crashes, but it was kind of a cryptic thing to do.  I had to put it in the constructor of the Context rather than something that made more sense.

/**
 * FeatureContext constructor.
 */
public function __construct()
{
 /**
 * Bootstrap The Store
 */
 gc_collect_cycles();
 ApiContextStore::create(); // Creates an instance if needed
}
2) Putting in an @AfterScenario test provided the same protection, but it ran, purposefully, after every test was complete.  I'm not freeing memory with my clear, so relying on garbage collection wasn't a priority.  I just needed it to run last.


/**
 * @AfterScenario
 *
 * Runs after every scenario */
public function cleanUpStore()
{
 ApiContextStore::clear();
}

http://php.net/manual/en/function.gc-collect-cycles.php
http://docs.behat.org/en/v2.5/guides/3.hooks.html

Monday, March 5, 2018

Postman Master Class Pt. 1

I gave a talk on using Postman while testing. We covered the UI, creating a collection, working with environment variables, and chaining tests with JavaScript.

A big surprise from this talk was how few testers knew about Postman to begin with. When I first started testing websites, I wanted a more reliable way of submitting http requests. The ability to save requests got me out of notepad and command-line cURL. The move to microservices only made it more useful to me.

By far, the biggest discovery was how many testers there were that had never explored its signature features. Environments and scripting make the instrumentation of integration testing almost effortless. Organizations that want automation but don't want to give the time can turn simple tests into bare bones system tests for very little further expense.

I'm planning a Part 2 where I can talk about Newman, the command line collection runner. I also want to demonstrate the mocking and documentation features. If a company adopts their ecosystem, it has the potential to make a tester's life much easier.  Even if it's only a tester's tool, it can help them communicate better with developers and reach into the product with greater ease.

Slides

Monday, February 19, 2018

Kill orphaned processes with awk

I use Behat to create some ham-handed load scenarios fairly regularly.  PhpStorm can help me spin up to get concurrency if I want a crushing load.  But PhpStorm on Mac seems to disconnect from these running tests if I lock my computer or leave them to run too long.  Killing them is a pain.  I have to run `ps`, find the pid, then kill it.

When the list gets too long, I like to use awk to comb through the list and kill anything that is found.  It's easy to search and parse out the tokens like so:


ps -ef | awk '/behat/ {system("kill -9 " $2)}'

ps -ef delivers a verbose list of processes.  This is piped to awk where I can specify a command using awk's scripting language.  In this command, I first search for 'behat'.  Then I  run a command pulls the second token, the process ID from each line of the result and inserts it into the `kill` command.

Monday, January 8, 2018

Fedora 27 Upgrade Issue - Solved by Removing yum-utils

Had a #meltdown upgrade to Fedora 27 stymied by a package conflict:

Error: Transaction check error:
  file /usr/bin/debuginfo-install conflicts between attempted installs of yum-utils-1.1.31-513.fc27.noarch and dnf-utils-2.1.5-1.fc27.noarch
...

I got around this by removing yum-utils.  It then allowed me to move on to the upgrade.  Got this idea from similar threads online.  Hoping anyone that runs into this after me will find this useful.

Here's to hoping the jump from 23 to 25 will be as significant as the one from 25 to 27.

Sunday, September 24, 2017

Dan Learns How To Blog...

Here's a quick retrospective after 25,000 page views.  In 4 years, topics were all over the map.  Mainstream topics like hacking Skylanders brought organic traffic from search engines.  Posts in a niche subreddit has a way of bringing in links years on.  The Cards Against Mormonism and boardgame tables still get a few hits per week.

Advertising revenue was an interesting experience.  A free site has generated a whopping $10 in 4 years from Google, and I haven't seen a cent from Amazon referrals.  In all, it's interesting to see how hard it would be to make a living with just ad revenue.  Sponsorship makes a lot more sense in this light, but impartiality takes a hit.  The web runs on a sea of quid pro quo.

In terms of authoring content, I learned a few things that English class couldn't anticipate in 1999. Mentioning specific dates instead of relative dates keeps readers from having to calculate in their head and increases the possibility that they will stay for the whole article.  If you have a story idea that has been sitting in your drafts for more than 3 months, delete it.  Your enthusiasm to talk about it is probably not going to grow after that long.  Build logs and multi-part posts should be split into shorter posts.  This will ensure you stay on topic and don't ramble.  And don't talk about how long it's been since you last blogged.  I can assure you that no one it waiting with bated breath for your review of Behat's 3.4 release.

On to the next 25,000!  Hopefully it won't take me 4 years.

Sunday, September 17, 2017

Review: Women in Tech

I finished reading @tarah's book Women in Tech. What better way to celebrate its paperback release than with a quick review.

Five years ago, I found my life turned inside out. People asked me deeply personal questions and questioned my basic competence. In the center of the maelstrom, I found comfort in a book with stories of people like me who were successful in spite of the difficulty. The stories were also paired with advice on how others has survived, thrived, and moved past the traumatic events.

In my case, my spouse had come out and I was coming to grips with my future as a straight half of a mixed orientation marriage. The book that helped me through that was The Other Side of the Closet: The Coming-Out Crisis for Straight Spouses and Families. Just knowing that I was not alone had a powerful influence. Therapy had helped; family could be supportive; friends might be weird. Those stories gave me the strength to say, "This too shall pass." 

For me, Women in Tech knocks it out of the park in a similar fashion. Concise, varied, authoritative women have lined up to share their experience making it in tech. Some faced abuse while others encountered discrimination. In the end, most felt the creeping fear of being an imposter (poignant in light of the abuse hurled at Equifax's Music Major CISO). For marginalized groups, simply knowing you're not alone can be enough strength for the day-to-day challenges.

The practical advice made it particularly useful for me. Coming to tech by way of tech support, I had no tutelage in interviewing, technical CV's, and salary negotiation. To this end, I've rewritten my resumé, registered a domain for this humble blog, and continue to try to organize a testing meetup in this desert town of mine. I don't know if each step individually will bear fruit, but together they make me feel less vulnerable to a manager's whim. I have a presence online and a skill to sell independent of any one job.

Broadly, Women in Tech has helped me understand the journey many of my co-workers have made. A fantastic tester with 20 years of experience that is comfortable in OpenVMS still expresses a lack of confidence in interpreting 'man words'. A skilled project manager guided countless projects from C-suite dream to customer reality while being a betimes single mom.  Being so broadly defined, tech needs diverse voices at all levels, and it particularly needs women and their contributions supported wherever possible.

There are plenty of gems in the book that I can't begin to address. My heart broke when the advice had to find a balance between optimism and reality. Having my spouse, an engineer by trade, transition made me want to learn more about the trans-in-tech experience. The constant refrain of Impostor's Syndrome makes me want to look for research papers. It is clear that Tarah has captured experiences with a depth and variety unavailable elsewhere.

I wouldn't be a hacker if I didn't mention the brain testing crypto puzzles at the heading of each chapter. Themed on famous women in tech, the learning curve is steep. I am currently stuck and, as the book makes perfectly clear, progress can only be made with help from all sides.

Monday, October 26, 2015

Testing uTest: or How I Learned to Stop Worrying and Love the Gig Economy

I’m a member of an online community of testers called uTest (the practitioner-facing side of Applause).  The company hosts a social network for testers as well as offers short-term gigs to its users (the cliched Uber for testers).    I was called on this weekend for my first gig: testing a payment method in Taxis.  It was pleasant, if stressful, and it made me think of ways a company could take advantage of it to expand the perspective on their products. 

After the initial invite to the test cycle, I communicated with a project manager via Skype to ensure I was able to carry out the test scenarios.  I brought to the table a specific model of phone and a verified ability to pull debug logs from it (thanks Verizon for turning off the op codes on my S4; I resorted to adb logcat).  They provided technical assistance and reimbursement for the transactions, but the primary incentive was a reward for test cases completed.

Throughout, I felt like a skilled secret shopper rather than a functional tester.  I was asked more about the non-software components of the launch than the app or phone functionality.  I reported on the advertisement of the feature, the education of the employees, and the capability of the equipment that supports the feature. In spite of expectations from the participating companies that the advertisement would match hardware would match training, this happened 0% of the time, and no employee had been informed why their equipment had been updated.  I wasn’t the only tester in this project on the ground, and the others testers saw related issues, and none had all their ducks in a row.  In all, the information they were most excited about was the boots-on-the-ground report I provided.  It was fascinating to see a live product launch from that perspective, and doubly so considering my history in this product space. 

The final bulk of time spent on this gig was an experience report.  Complete with detailed feedback, photos of the equipment, and other evidence, this is where I was able to comment on the process as a whole.  From a testing perspective, I was able to provide detailed UI/UX feedback, evaluate the consistency of the process, and help them understand how much of their plans actually came off during the launch period.  There was some repetition in these reports.  One was a document with pictures, the other was a survey that linked to the doc and a third was a JIRA-in-spreadsheets for tracking testers.  These reports were all submitted to the project manager for feedback, and I received an up/down "accepted" in less than 24 hours.  While there is definitely room for improvement on the platform that would reduce tester overhead, it wasn't enough of a burden to avoid gigs.

Participation in this project taught me that coordination is hard, education of low-skill workers is even harder, and successful coordinated nation-wide launches like this are next to impossible.  This mirrored my experience with other companies.  There are always bits and pieces of projects that do not make their way down the chain to the workers that are face-to-face with customers, so having a boots-on-the-ground perspective is vital.

Overall, a company can leverage uTest services to add layers of testing to their launch activities over and above internal testing.  Specific and targeted verification is where they are the strongest.  They can provide feedback on as to whether an uneducated but motivated customer can make it through the new process right now.  Specific errors and third-party verification can be  farmed out for feedback through such a relationship: things the company wants to do but can’t do efficiently given a small staff or tight timelines.  This can provide a company with an enhanced view of the product and help challenge their assumptions before their customers do.

Note: edited to be a little more coy about who I was testing for and what I was testing.