Behavior Driven Testing

Developing software in most companies starts with a Product Owner who is responsible for identifying and communicating requirements to the Development team (including QA).  The process flow usually goes something like this:

A Product Owner (or Product Team)
-> Collects and writes Requirements in a prose format
-> Someone interprets and rewrites tasks for Development
-> Development implements the feature
-> Someone may even interpret and rewrite the requirements for QA
-> Per QA Plan’s Definition of Done, QA may write automated tests
-> When the feature is ready, QA executes tests
-> QA interprets results
-> QA either sends ticket back to Development to fix
-> QA generates a report for review.
-> When Definition of Done is met, the Product is released.

There are two areas in this process that have seen streamlining thanks to a number of open source technologies.  These areas are:

  • Improvements to the way a Product Owner formulates the requirements
  • Improvements to tools that automatically report and integrate with those requirements

Technology Stack

  • JIRA: Software project management tool
  • Xray for JIRA: Test Management tool for JIRA
  • Java + Groovy: Enables Domain Specific Language (DSL) creation and use.
  • Gradle: A Groovy DSL that simplifies building and running tests
  • Geb: A Groovy DSL that simplifies Selenium. Especially when you use the page object pattern.
  • Spock: A Groovy DSL that uses the Gherkin language for specifying behaviors (BDD) as an action and assertion combination. Similar to Cucumber except that the feature file information becomes the high level structure to contain the test details for implementing the behavior.

Sample Gherkin Specification

When user navigates to login page
And user enters valid user credentials
And user clicks on Login button
Then user is shown the home page

Spock can be used for framing unit, back-end, and integration testing. When combined with Geb, they can be used for UI and end-to-end workflow testing. The integration of Spock means that tests are aligned more closely with the Specifications (i.e. Requirements).

I’ve extended the test infrastructure with features so that each test will output results directly to JIRA tickets thanks to the Xray for JIRA plugin. The end result gives the Product Owner a better sense of confidence that the implemented feature capabilities have met the specifications.

Overview of the New and Improved Process

  • Product team writes easy to read user stories and scenarios (examples) using the Gherkin language. These acceptance tests are stored in JIRA as Xray tests.
  • Development and QA updates the scenarios (test cases) or adds new test cases.
  • As Development implements the features functionality, QA writes the detail to test the features functionality by utilizing the Gherkin scenario specification as an outline to each test.
  • When the implementation is ready from development, QA can utilize their automated test to verify functionality. QA becomes closely tied with Development by being in lock step.
  • Test results are sent back to JIRA and get reported within the ticket associated with each test; where the test results belong.