How I Lost My Head With Headless Testing

Ghost design in coffee
Ghost in coffee photo by Toa Heftiba on Unsplash

When I started out on my crazy adventures in test automation, I stayed close to the browser. The first automation framework I worked in was WATIR and it was awesome. It is a clean and straightforward framework which is what drew me to it. I was introduced to my first headless automation tool by one of our UI Engineers. The tool set was CasperJS and PhantomJS. While they were great tools, I really wasn’t sure how to use them correctly, especially since I wasn’t watching my tests execute anymore. So I played with them for a while and that was it.

I continued with browser automation and even would recommend teams to keep to that form of testing for a few years. After all, our customers use browsers. As browsers have evolved they continue to become heavier and heavier and test servers are not always keeping pace. I recently worked with a test server that was weaker than the laptop I was issued. (The laptop barely supported Windows 7.) The resources of test server get maxed out while running automation. While this might sound like an IT problem, not every organization is able to or willing to spend on proper test environments and servers. We need to keep pace and evolve our testing.

Fast forward a few years and I love headless test automation!

It was during this time, that I came back to headless testing. I needed to find solutions to the issues of our test server maxing out (and crashing) during automation execution and our pages taking minutes to load. I finally saw the potential for headless test automation and began looking into how I could port my tests away from the browser.

Going faster because I lost my head

Headless test automation allows for faster test execution with lower resource requirements. Running headless test automation enables you to avoid the weight of the browser and most page load time issues that would slow down your testing otherwise. This reduces your overall testing footprint which can become quite large when you get in the hundreds of test cases being executed.

Yes, you need to adjust how you are building test script because you’re not watching the test as it runs. This is not a huge problem as you can incorporate screenshot capturing for failures (see my article Failed Tests Are Better With Screenshots) or at any point along the way. Do be careful on how many screenshots you take per execution though, as you will eat up disk space. I recommend you keep it to just failures, after all if everything passed, you probably don’t care to see each step of the test.

Recommendations for headless test automation tools

There are two that I like the most, PhantomJS and Capybara. PhantomJS works with several languages and has been the backbone of other tools. Capybara works with Ruby and I have been using it more and more in my automation efforts lately. There is also Chrome Headless, but I have not used it yet.

My recommendation is to test several options and see what makes the most sense for your testing needs, organization, and team. If you already have existing test automation, you want to know which is easiest to convert to, you will need to experiment. When I moved my last test automation framework off of Selenium I went with Capybara. Not because it was the easiest to port over, but because it resolved many of the resource issues I had. The underline classes were easier to move over, but the tests needed updates.

Try losing your head

If for no other reason than to explore the technology. Headless test automation is a great way to get through more testing while reducing stress on your test server. And who doesn’t love seeing test results coming in faster?

Let me know your thoughts and opinions on headless test automation. Have you had any success?

Additional reading

HackerNoon ran a comparison between Chrome Headless and PhatomJS in September 2017, here are their results.

Headless Chrome vs PhantomJS Benchmark

Leave a Reply