Quality Assurance

Automatic System Tests for Smallworld Products Concept and Context

Modern development processes are characterized by strong customer focus, high flexibility and short development and delivery cycles. A test procedure must adapt accordingly and offer efficient mechanisms for effective quality assurance, which is impossible without automatically performed tests.

Test equipment is classified by the process level to which it is applied. Automatic test tools are typically tailored to the early stages of development and focus on the source code, a disadvantage from the perspective of test management and means that the developer typically develops, maintains, interprets and often also executes test cases. Checks by management are nearly impossible in the sense of measurement overlap, a lack of proof of completed tests, readability etc. so that the usefulness of test cases is difficult to quantify and evaluate. Additionally, early stage tests are barely suitable as system tests due to the necessary complexity of the test environment.

System tests regard the system under test as black box and evaluate externally visible reactions to stimuli from the outside. In Smallworld applications the interface is typically but not necessarily the GUI. Test scenarios consist of a significant number of individual steps such as cursor navigation, mouse clicks, input of parameters into fields, evaluation of results etc. Support of a testing tool is essential to create and maintain the described test scenario and to avoid their becoming overly complex and error-prone.

Tools based on the concept of hierarchical refinement of test steps, known as Action Words or Interactions, are suitable. Both concepts create test scenarios by means of logical high level steps that are subsequently refined into elementary, executable steps. For instance, the tool ALM by HP introduced the concept Business Process Testing with possible refinement via Business Components and Component Steps. The latter consist of the elementary handling of GUI elements or specifically programmed steps.


Interaction-Based Automatic Test

The test management tool TestBench by imbus uses the equivalent expression "interactions" - the core capability of the system. An "interaction" is defined as a test step at any level in the hierarchy that can be further refined down to an elementary test step. No difference is made between manual and automatic steps. Interactions are parametrized with test data, which is recorded in TestBench. The figure, which is taken from imbus slides, illustrates the procedure.

The test area covered by TestBench is marked in grey. It encompasses the entire high level specification of logic and concrete test cases including its parameters and is defined, read and maintained by a test designer (or else a customer). They are part of a complete test management tool so that an orderly test process can be set up.


The Test Execution Engine (TEE) is not part of TestBench and needs to be created for concrete target systems. TestBench supports its development by offering the export of the test specification and a DLL for its analysis and control. The DLL can write the test result to file and serve as input to TestBench. Further editing (analysis, statistical analysis and archiving) takes place in TestBench, exemplified in the figure.

The tester conducts exports, imports and handling of the TEE. An extension to include a Continuous Delivery Process is in principal possible.

iTEP-Wrapper is a product by ITS International Services, Germany.


The TEE reads the basic steps of test cases and passes them on to the target systems, typically to Magik images under test. The target system interprets and executes the commands. Each basic interaction requires a dedicated method in Magik. Many basic interactions operate the GUI elements of the Smallworld GIS for which drivers might exist to handle different GUI elements of the Smallworld GIS. The figure illustrates the architecture.

In addition to the TEE, the Magik components iTB Interface, interactions and drivers are part of the test framework (shown in light blue) developed by ITS International Services, Germany.

The test framework provided by ITS International Services, Germany, is based on the interaction principle of TestBench by which it is optimally supplemented while being in principle independent from TestBench.

Go to top
© 2018 ITS Informationstechnik Service GmbH, Germany