BDD practice with Beaf - next generation acceptable testing

Author:Xie,Forest

Background

Last year, our Gumtree Australia team started to use behavior-driven development in our agile development process. Initially, we searched for an open-source BDD framework to implement for BDD testing. Since we work on a web application, we wanted our automated BDD tests to exercise the application as a customer view, via a web interface. Of course, many web testing libraries do just that, including open-source tools such as Selenium and WebDriver (Selenium 2). However, we discovered that no existing BDD framework supports web automation testing directly.

We decided to simply integrate the open-source BDD tool JBehave with Selenium, but soon learned that this solution doesn’t completely fulfill the potential of automated BDD tests. It can’t drive the project development and documentation processes as much as we expected.

Different stakeholders need to view BDD tests at different levels. For instance, QA needs results to show how the application behaved under test. Upper managers are not so interested in the finer details, but rather want to see the number and complexity of the features defined and implemented so far, and if the project is still on track.

Beaf Coming!

We needed a testing framework that could let us express and report on BDD tests at different levels, manage the stories and their scenarios effectively, and then drill down into the details as required. And so we started to develop a product that matches our needs. We named this product the Behavior Automation Framework (Beaf) and designed it to make the practice of behavior-driven development easier. Based on JBehave as well as more traditional tools like TestNg, Beaf includes a host of features to simplify writing automated BDD tests and interpreting the results.

In beaf process,

1) The following system will be integrated: beaf Web Console, Jenkins service (CI pipeline), QA lab.

2) Developer/QA/PM can execute BDD case in local or Jenkins Job (Anyone, anytime, anywhere).

3) QA lab will support all kinds of browsers and emulators (Android/iPhone/iPad).

Generally, Beaf consists of two parts.

Beaf-Core: Supplying the methods to trigger and execute the BDD cases, including the conmmon functions and data structure, generating testing report and analysis module.

Beaf-Web: Supplying a web console which users can create/edit/delete stories/scenarios, demonstrating the statics/trends of BDD status by a dashboard.

How Beaf Works

A Product development team can use beaf easily to finish BDD automation testing process.

At first, Product manager need add a story and the scenarios into this story before QA team start to write automation cases. He can login to beaf Web and add a story in story manager page. Beaf Web supply a template which can help user create the scenarios, besides that, when user creates

steps for each scenario, beaf web will do syntax checking and supply autocomplete tips, ensure user can finish steps efficiently

After PM finished it, QA will get an email notification to show there is a new scenario has been created,

QA can start to add the automation testing code for each step in the scenario.

After QA finish the codes, then he can execute the BDD test and generate beaf report.

If the testing result is ok, QA will check in the testing codes into github, and then we can find the status of new scenario has been changed to "automated".

And in Bolt Dashboard, the chart or metrics also updated:

Beaf facilitates QA people joining projects in stages. It provides PD/QA with the detailed information required to test and update code, while giving business managers and PMs the more high-level views and reports that suit their needs. However, the greater potential of Beaf is the ability to turn automated web tests into automated web acceptance testing, in the true spirit of BDD.