If you use automation testing in your project, you’ve probably often wondered about the best ways to support automated tests and write new ones. Usually, teams recruit new staff, but what if you were to offer manual testers the opportunity to write automated tests? In one GS Labs project, testers decided to experiment with this alternative approach, and simplify the writing of automated tests by applying Behavior-driven development (BDD).
Test-driven development, or TDD, involves setting up automation testing before writing the code. In other words, the first step involves writing a test to check the correct operation of code that has not yet been written. Naturally, the test fails. The programmer then writes code which performs the actions required to pass the test. Once the test has been successfully passed, the code can be developed further.
Behavior-driven development, or BDD, is a variation on or extension of TDD. However, the approaches are designed for different purposes, so different tools are used to implement them. BDD first and foremost helps to improve cooperation between stakeholders, i.e. to develop a shared understanding of the software’s behavior. It is this which determines the main purpose of the approach.
BDD has proven itself for use in integration testing (i.e. for checking that individual modules can work together) and end-to-end testing (as in our case).
From the Python BDD frameworks, it was decided to use the BDD approach for writing automated tests. The Behave library was chosen for this. Behave is a framework for Python-style behavior-driven programming. Behave uses tests written in a “natural style”.
The library can be installed using the command line: pip install behave
The folder structure is as follows:
The steps.py file contains the “steps” written in Python. The test.feature file contains the tests themselves, in “natural language.” To launch tests, you need to run the command:
To launch tests with tags, you need to execute the command:
Before the advent of the API, it was possible to use an infrared switch on a receiver (in place of the remote control used for manual testing), but test stability was not satisfactory, since commands were not always received correctly by the receiver. Switches like these were mainly used for load testing.
Later, infrared switches on receivers were replaced by APIs, which are regularly upgraded and improved. This promised to improve test stability, and we began to introduce automation. Python 3 was chosen as the language and flows were used as test data.
For example, let’s look at the test that checks messages are correctly displayed on the client’s receiver. We start the flow and wait 60 seconds (it is sent after 60 seconds). Automated tests were written in Python code:
For new functionality, it was decided to use automation for configuration. This makes it possible to work with the latest version of the environment. It also opens up space for the creation of any kind of test, even very complex tests. For functionality which is not being updated, it is better to use flow testing, since there is no need to update the transmitter part; it is sufficient to prepare the test environment once and record a flow in it.
Colleagues, including automators, sometimes confused the expected result with the result obtained, particularly if they did not have an adequate understanding of all the finer points of the test object. This ultimately led to releases being delayed, and so it became necessary to seek out new approaches to automation.
What Do “Manual Testers” Do?
What do our manual testers do and why do we need to get them involved at some point? Manual testers carry out tests which have not yet been automated, or for which automation is not appropriate. Additionally, they write test cases and review new features.
As automation progresses, the number of automated tests compared to manual tests steadily increases. This causes two problems:
· There are more automated tests—these need to be supported and there are not enough resources.
· There are fewer manual tests, and work must be found for manual testers or they will get bored.
There is, however, a solution to both of these problems: getting manual testers involved in automation. But how do you do this if the testers do not have the necessary programming skills? This is where BDD can help.
Automation Following the Introduction of BDD
Let’s consider the previously written automated test to check that messages are being correctly displayed on screen, but this time written in BDD style:
Given—describing the system as intended.
When—describing the user actions.
Then—describing how the system reacts.
The most important thing is that it should be easy to understand what has been written, even if you are not fully immersed in the functionality. Automators would still run and update some steps, and debug new automated tests.
There is no longer any need for timings relating to waiting for services to load or subscriptions to arrive. Writing tests becomes like building with Legos— and have you ever met a tester who doesn’t enjoy playing with blocks?!
Evaluating Test Results
The results can be immediately transferred into the project test plan. If, for some reason, a plan has not been created or tests need to be debugged, then the results are transferred to an interim test plan. These results can then be transferred into the required plan.
A quick point on analyzing results: if a test fails, it is vital to understand why.
There are several possible reasons:
In Conclusion
Ideally, BDD methodology should be introduced at the product development stage, but even if this is not the case, it offers benefits. For all the complexity and time-consuming work involved in implementing it, automation using BDD methodology definitely helps to improve product quality. When the key elements of automation, such as the framework, staff training, and business process adjustment, are properly organized, automation using BDD methodology can serve as a starting point for the full deployment of BDD across all stages of product development.
Automation using BDD methodology has its drawbacks, but overall, it has been a useful approach for our team. It has allowed us to involve additional resources in automation and, consequently, to increase the proportion of requirements covered by automated tests. And it has boosted the self-esteem of our manual testers.