A framework is a set of guidelines and rules designed for creating Test Cases. It is a set of tools and good practices that make Quality Assurance Professionals test more efficiently.
These rules can include coding standards, test data usage methods, object repositories and processes to store test results, among others.
Benefits of Using a Testing Framework
Some of the benefits of using a testing framework are:
- Improve efficiency
- Low maintenance costs
- Minimum manual intervention
- Maximum code coverage
- Source code reutilization
There are different types of testing frameworks, each has its own advantages and disadvantages. Some of the existing testing frameworks are:
Linear Framework: This is a basic framework that has a record and play utility. Testers don’t need to create complex functions, and the steps to follow are sequential. Instructions are stored and these sequential steps are reproduced one by one each time a test is executed.
Linear Framework Advantages
- There’s no need to write source code, therefore no automation experience is needed.
- Creating automated tests is really quick.
- It’s easy to understand because steps are detailed in a one by one sequence.
- It’s easy to learn a new tool. Most automation applications have this functionality available.
Linear Framework Disadvantages
- Code usually isn’t reusable. When a step in the test case sequence changes, old code is now obsolete and it isn’t easily modified.
- Maintenance is complicated, when something changes on the application testers need to record a new set of steps and usually this functionality change involve several test cases. Besides scalability isn’t possible.
Modular Framework: This framework requires that the QA Engineer divides the application into different units, sections or functionalities, which need to be tested independently. After the application has been divided into modules, source code is created to test each one of them. The Tester may also combine different modules to create bigger tests that will cover additional test cases.
Modular Framework Advantages
- If the application maintenance is simple, you just need to update the module or functionality that has been affected.
- Source code from modules or functions can be easily reused.
Modular Framework Disadvantages
- Some tools don’t allow data parametrization so modules aren’t reusable in these scenarios.
- Tester needs to have programing language knowledge.
Data Driven Framework: Frequently Quality Assurance Engineers need to test the same functionality or the same module using different sets of data. In this cases it’s needed that test data isn’t hardcoded into the source code like it would happen on the linear or modular frameworks.
The data driven framework allows the test engineer to store the test data in an external storage like Excel spreadsheets, text files, CSV files, SQL tables or ODBC repositories.
Test Case source code connects with these data repositories, it reads and fills out this data when required.
Data Driven Framework Advantages
- Tests can be executed using different sets of data without rewriting the source code.
- Multiple scenarios can be tested just by changing the data without writing several source code sets.
- Data isn’t written in the source, so updates on the data and the source code are independent.
- More tests are executed faster.
Data Driven Framework Disadvantages
- The QA Engineer needs to have programming experience to properly use this framework, since there’s interaction with the data layer the Engineer must be able to write the source code to perform actions involving data.
- Using this framework takes a lot of time to setup and writing the code is complex. A higher level of abstraction is needed.
Keyboard Driven Framework: This one is similar to the data driven framework, however on the data repository there isn’t any test data stored, however actions are stored which can be executed with input devices like the keyboard or mouse. This framework is used mainly to test the application’s UI.
The test cases are written as consecutive actions that need to be followed by the framework.
Actions table:
The Test Engineer then writes the necessary source code for each one of the actions that will be executed. When the test is executed the data reads each of the keywords and the framework executes the source code for each action.
Keyboard Driven Framework Advantages
- There’s no need for a lot of test automation experience.
- The same identifier can be used on several tests, so the code is reusable.
- Source code can be written independent to the testing application.
Keyboard Driven Framework Disadvantages
- The cost needed to configure the framework is high. It consumes a lot of time and it’s complex. The keywords must be stored on the object repository and the libraries that need to be used must be defined.
- Quality Assurance Engineer needs programming languages expertise.
- Keywords can be difficult to maintain. A constant maintenance of the repositories and of keywords is needed.
Hybrid Framework: Just like it happens with a lot of procedures nowadays, automated test frameworks have begun to integrate and intersect with one another. Like its name says a hybrid framework is a combination between any of the frameworks mentioned before to strengthen their advantages and minimize their weaknesses.
Every application is different, therefore the processes needed to test them must be different from application to application. As teams embrace an agile methodology, the fact that the testing framework is flexible becomes crucial. A hybrid framework can be adapted easier to obtain better results.
Behavior Based Framework: Its objective is to create a platform that allows all the parties involved on a project (Business Analysts, Developers, Quality Assurance Engineers, etc.) to actively participate on the test design. It requires a strong collaboration between the Development and Quality Assurance teams. This framework doesn’t need the users to have programming language knowledge. It uses natural language to design the test requirements. Some of the behavior based test development tools are JBehave and Cucumber.
The Quality Assurance Engineer needs to know which are the different framework options that are available to make an informed decision on which are the proper frameworks that need to be used on the project and also to make things as fast and as efficient as possible. Also it’s useful for the Quality Assurance Engineer to know how the automated testing tool and the framework entangle for them to know better which concepts will be applied and how the tool implements them.
