1 SMLC

Test development
This test system contains typical conformance test cases for a GSM network element. It is tested wher the system under tests sends the correct messages with the correct contents. The test cases are written manually. In addition to these conformance test cases there are further test cases to test whether the encoding of the messages by the system under test is correct and that it recognizes incorrectly encoded messages and reacts appropriately. A third class of test cases is available for load testing of the SUT, these test cases can also be used to measure the performance of the TTCN-3 tool.

Test execution
In the case study the actual system under test has been replaced by an emulator. The emulator has been again a TTCN-3 test system with test cases mirroring those in the test system. The system under test adapter has been a separate executable.

Demonstrator
The test system consists of the TTCN-3 executable and the separate system under test adapter. No specific test equipment is needed for test execution because the messages are exchanged using TCP/IPv4.

References
[D3.3.1]Description of execution environments and testing tool demonstrators to be shown in exhibitions.

[D4.5_Nok]Domain Specific Test Systems. Please click here.

2 CORBA

Test development
This test system contains test cases for functional black box testing of a CORBA application. The test cases are rewritten manually from an already existing test suite in a proprietary format. The test cases import the IDL definitions of the application and use procedure-based communication to access the system under test. A generic system under test adapter is used to communicate via CORBA with the system under test.

Test execution
The test execution has been integrated into the TTworkbench. Using CORBA as middleware, the test system and the system under test could be geographically separated. The test system has been executed in Bochum, whereas the actual system under test has been located in Finland.

Demonstrator
The demonstrator consists of the TTCN-3 executable. No specific test equipment is needed.

References
[D3.3.1]Description of execution environments and testing tool demonstrators to be shown in exhibitions.

[D4.5_Nok]Domain Specific Test Systems - Nokia, v3. Please click here.

[D4.7_Nok]Case Study Manual. Please click here.

3 SIP

Test development
Test cases in this case study have both been manually written as well as generated from a UML model. Both sets of test cases could be executed on the same test platform. The test cases are used for functional testing of a SIP application on a mobile phone. The test cases had been written for a setup where the system under test had been connected directly to the phone.

Test execution
The test cases are executed against the actual phone. In this case study the phone is accessed via the air interface, involving a base station and the simulation of a base station controller. The direct connection between the system under test and the test system has been replaced with the real connection, only few changes of the existing test case have been necessary.

Demonstrator
The demonstrator consists of the TTCN-3 executable, a simulation of the 2.5G and 3G base station controllers, and a 2.5G or 3G indoor base station. The tool to generate the test cases from the UML model is also part of the demonstrator.

References
[D3.3.1] Description of execution environments and testing tool demonstrators to be shown in exhibitions.

[D4.5_Nok] Domain Specific Test Systems – Nokia, v3. Please click here.

4 OBSAI

Test development
The objective of conformance test is to ensure that the OBSAI module under test fulfills the requirements defined in the OBSAI technical specifications. Compliance to the OBSAI technical specifications, demonstrated with means of the defined conformance test enables inter-working between OBSAI modules from different vendors.
OBSAI specifies a set of conformance test cases for testing compliance of OBSAI defined interfaces. The test cases are in textual format and the module vendor, the base station vendor or an authorized test house is responsible for implementing test executable. TTCN-3 is preferred when applicable.

Test execution
The OBSAI Conformance Test Catalogue provides a list of applicable conformance test cases that are required in order verify compliance of an OBSAI module or an interface of the module. Manufacturers will use the Conformance Test Catalogue to identify specific test cases required to determine compliance requirements for the module under test. There are three main types of conformance tests valid to demonstrate OBSAI compliance, Self Conformance Test, Customer Conformance Test and Independent Conformance Test. OBSAI does also support the possibility to use a mixture of the conformance test types to complete the conformance test.

Demonstrator
Figure 5.1 illustrates the conceptual architecture of an OBSAI conformance tester, which puts some extra requirement for OBSAI modules by defining test messaging between the test environment and the module agent so that test behaviour can be controlled in standardized way. In the realm of OBSAI, the test messaging covers test control functions needed for permitting OBSAI conformance tests for modules. This is essential, because OBSAI does not specify functionality of modules in detailed level. The System under test (SUT) can be any OBSAI module under test. Many OBSAI conformance test cases require that the SUT is able to execute specific test function. The module agent listens at the RP1 interface and responds to test messages as described in the OBSAI RP1 Test Messages Specification. A test manager is a test system specific entity, which controls the testing, records test log and makes test verdicts. The test manager controls the testing using test messages from one end and test system specific test equipment from the other end. The test system performs required test functionality and measurements in the interface under test

Figure 5.1 Conceptual architecture of test environment for OBSAI testing

5 Automotive

The systems under test have been in-car telematics systems, i.e. systems consisting of integrated in-car entertainment and infotainment components such as the Head Unit (central user interface for the driver), Navigation System, Radio, Audio Amplifier, CD Changer etc. Most of the telematics features are realised by the simultaneous interplay of these components, e.g. on receiving a telephone call, the CD Player is paused and the Audio Amplifier allocates audio channels to allow for hands free speech using microphones and speakers built into the dashboard. The various components of the system are connected via a MOST ring.

Test development

TTCN-3-based test cases developed for this telematics system are simulating user actions on the Head Unit as for example the switching between several telematics functions or devices (e.g. between the CD-Player and the Radio or between the Radio and the Navigation Device).

A tester (a special test component) connected to the telematics ring sends test data in form of messages and observes the behaviour of the telematics components. For accessing and observing the MOST ring two special devices, called Optolyzers, are used.

A demonstrator for the automotive case study consisting of the telematics components named above has been build up and shown during the ICSTest conference in Düsseldorf, Germany, 6th to 8th of April, 2005 and at the ITEA Symposium in Helsinki, 14th and 15th October, 2005. The demonstrator runs the test cases described which results in several observable switchings between telematics components. The results of these switchings are monitored by the test component and it is checked whether the reactions of the components are those which are expected.

6 Railway

We developed an approach for testing railway Interlocking systems with TTCN-3. Interlocking systems form the safety layer of a railway control system. We demonstrated our approach on the Vital Processor Interlocking (VPI) for the Dutch railway station Hoorn-Kersenboogerd.

Test development
The CENELEC standard EN 50126/50128/50129 describes a number of safety scenarios that should be tested for railway Interlocking systems. Such a general safety scenario consists of an initial situation, a number of events, and an expected final situation. The general scenarios must be mapped onto the infrastructure of the system under test. The mapped scenario is then represented by a TTCN-3 test case.

Test system implementation
In order to test the VPI software without actual railway hardware, a simulator for VPI acts as the system under test. We implemented a TTCN-3 test system (including system and platform adapters) for testing railway Interlockings. The main issue was to define and implement the right notion of simulated time. We developed a time handling solution, based on Dijkstra’s algorithm for distributed termination detection. This can be reused for other timed systems as well. Together with the TTCN-3 code for the test cases, this forms a test system, which can be executed against the VPI simulator, acting as SUT.

Demonstrator and Test execution.
The demonstrator consists of the test suite (A TTCN-3 implementation of a subset of applicable CENELEC standard scenarios), the implementation of the test system (platform adapter in Java and TTCN-3), our simulator for VPI code, and the code for the Dutch railway station Hoorn Kersenboogerd. The test system is run against the simulator with the VPI code. We used the μTTman and TTworkbench tools, for the actual test execution and test logging. With this test suite we have actually detected a violation of a safety requirement.

References
[BIP05a] S. Blom, N. Ioustinova, J. van de Pol, Testing Railway Interlockings with TTCN-3. TTCN-3 User Conference, Sophia-Antipolis.

[BIP05b] S. Blom, N. Ioustinova, J. van de Pol, A. Rennoch, N. Sidorova, Simulated Time for Testing Railway Interlockings with TTCN-3. 5th International Workshop on Formal Approaches to Testing of Software FATES 2005, July 2005, LNCS proceedings.

[Sym05] 6th ITEA Symposium, Helsinki, October 2005, system demonstrator.

[SimTime] URL for simulated time: http://www.cwi.nl/~ustin/stime.html.

7 Financial

Test development
The TestFrame Language has been mapped to TTCN-3, making it a presentation format for TTCN-3. That gives testers from the financial sector, who want to use TTCN-3 as a specification format that is more end-user-oriented, an approach familiar in this world.

Test execution
To interface with financial applications, TestFrame software modules have been integrated in the TTCN-3 architecture as codecs and system adapters. The focus for interfacing has been on GUI applications. With this integration of existing test ware, the financial domain is now in reach of automated testing with TTCN-3.

Demonstrator
The demonstrator shows the process of translating TestFrame test designs written in TFL to an abstract test suite (ATS) in TTCN-3, compilation to an executable test set (ETS), test selection and execution with a test manager tool and analysis of log files. The System Under Test (SUT) is GLOBUS, a banking application with a graphical user interface (GUI) that is typical for the financial domain.


References
[D1.1.2] Developing testing methodologies and integration of testing tools. Please click here.

[D4.5] Domain specific test system. Please click here.

[D4.7] Demonstrator for the financial domain. Please click here.