Automatic testing generally refers to the test method of replacing manual testing with programs or tools. Compared with manual testing, automatic testing plays a great role in ensuring product quality and shortening test time with higher accuracy and better reusability. Here, the idea and the technical practice of PlatON’s automatic testing will be introduced in two parts.

Why Implement Automatic Testing

In the world of PlatON, there are various statuses, such as staking status, locked status, proposal status, etc. Among them, the staking status of a node can be divided into three periods: hesitation period, locking period, and frozen period. These statuses begin to change automatically with the height growth of the PlatON block after an account makes the node into or out of staking status. In manual testing, it is difficult for testers to grasp the right time of status change. Once a mistake occurred, the environment can not satisfy the test requirements,. As a result, the testing can only be restarted.

While, in automatic testing, the status change difficult to grasp for manual testing will be controlled by code. After written by the testers, automatic testing programs can complete the whole testing and verification process conveniently and accurately according to the code logic, thus solving the problem of manual testing perfectly, and making the testing work more convenient with high automation and reusability.

Key points of automatic testing

Before implementing automatic testing, the key points of the automatic testing need to be sorted by extending and refining the testing requirements. These key points could help us avoid design deviation in the establishment of the test framework.

The three key points of PlatON’s automatic testing are as follows:

Supporting distributed request and verification — the distributed feature of the blockchain determine that distributed request and verification should be available when implementing testing. For automatic testing, it means sending transactions at any node and verifying transaction results at any node.

Support free transaction assembly — we mainly verify its program logic by sending transactions when testing PlatON. There should be completely different program logic for transaction assembly when there are various transaction types and various data in the transaction. The ability to assemble transactions freely is a must for automatic testing to construct testing scenarios in a more sophisticated way, especially when simulating abnormal and evil test scenarios.

Supporting automatic and custom deployment chain — when testing various network structure modes in the blockchain, such as node access and exit, the deployment chain needs to be customized in automatic testing. In addition, when the testing scenario needs to be restored (restoring the scenario to its initial state, after finishing the execution of a certain testing scenario), it is a better choice to redeploy the chain if it takes a long time to wait.

Introduction of Automatic Testing Framework

Developed in Python language, PlatON’s automatic testing framework is implemented based on Pytest and PlatON SDK, and is mainly divided into three layers: Case, Lib, and SDK.

Please visit the GitHub repository for detailed codes: https://github.com/PlatONnetwork/PlatON-Tests

Framework Diagram

Detailed Introduction of the Framework

Case

Automatic testing cases based on the framework and the rules of Pytest.

2. Fixture

Based on the Pytest framework, Fixture can support various functions, such as developing test cases, running parameters in custom testing, constructing testing environment, judging whether the environment meets the operation conditions of the cases, acquiring testing objects, implementing pre/post action of cases, and pulling exception logs with hooks.

When obtaining testing objects with Fixture, we will abstract each node into a Client object, and return the list of all Client objects for the cases to use. As a result, the case can freely carry out distributed transaction requests and verification according to the needs of the testing scenario.

We’ve defined a batch of Fixtures for environment verification, to verify whether the current chain meets the operation conditions of a certain type of testing scenario. The testing scenario will operate if the conditions can be met. And the chain will be redeployed, and then the testing scenario will be run again based on the redeployed chain if the conditions can not be met. In this way, we can ensure the continuous running of the automatic testing.

3. Lib

As a support layer for testing cases, Lib connects to Fixture, with some common methods, test tools, calculation formulas, configuration information, and so on wrapped in it. Client class, the main class of the Lib layer, assembles all objects in the whole Lib layer to facilitate the development of automatic testing code. Besides, the Client class is added with the main objects of Env SDK and PlatON SDK, to realize the ability of chain environment deployment and custom transaction sending.

4. Env

PlatON Chain Deployment abstracts the testing environment into Env, Server, Node, and other classes. And Supervisor will be used to manage the process and the log of the PlatON’s node. Based on multithreading, Env supports deploying, modify, query and clean to the chain and the node.

5. SDK

PlatON SDK provides abilities to assemble and send transactions freely. When we need to construct special transactions, it goes directly to the SDK layer for further construction.