The following deliverables are available for download:

Work Package 1

The specification of the TT-Medal test infrastructure and architecture aims to provide a comprehensive overview and understanding of the test infrastructure elements and tools required to perform industrial TTCN-3 tests. An introductory information including a market viewpoint and an overview on the overall architecture of test infrastructure and platform components is given at the beginning. The major contents of this document is the explanation of test infrastructure elements, i.e. general TTCN-3 test system entities for a distributed test execution. In addition the document presents required features for test logging, the integration of IDL or XML based information on the SUT, and extensions regarding the adaptation to heterogeneous technical SUT interfaces in addition to Java or C++. Furthermore the document includes an introduction of different test architectures used in TT-Medal case studies and a catalogue of a basic terminology for testing that has been derived from various standardization bodies in order to support a common understanding on the TT-Medal infrastructure.

[D.1.2.1] Towards Model Based Test Generation and Validation for TTCN-3.

Software testing is a model based activity, since any test is designed using some "model" of a system under test. Last years, the use of explicit models for software development has noticeably increased. A lot of work was done to both formalize the testing activities and to develop tools allowing to automate production and execution of test suites. To enable reuse of testware, a well defined test notation TTCN was developed within a conformance testing framework.

This document provides an overview of the conformance testing framework and formal approaches to test derivation. Conformance testing is a model based approach, hence we give a survey on commercial and academic tools for model based test generation. In particular, we are interested in techniques and tools that allow to derive TTCN test cases from a (formal) specification and/or test purposes. We consider various definitions of "test purpose" and discuss their limitations and suitability for the goals of the project. We provide a high-level definition of test purpose that is not related to any particular model. We also describe directions for future work in Work Package 1 of the TT-MEDAL project and provide a motivation for the choice of tools and formalisms.

[D.1.2.2.1] Definition Of Methods for Automated Test Generation for TTCN3 Based Test Systems.

The objective of Task 1.2 of TT-Medal project is to develop techniques for automatic test generation for TTCN-3. As a starting point, we consider tools for automatic test generation that are already developed by industrial and academic communities. Existing model-based test generation tools are mostly based on state exploration of a system specification. The tools are completely automatic and deliver test cases that are already correct by construction. However, these tools suffer from certain limitations. They usually can only handle finite systems of rather limited size. Real systems are very large or even infinite, so these test generation tools cannot be applied directly to such systems.

In this document, we consider three issues: (1) using abstractions to enable enumerative test generation for large and infinite systems; (2) an integration of TTCN-3 with techniques for generating test data and (3) mapping UML diagrams to testing techniques.

We propose an approach that allows reusing existing enumerative test generation techniques for large and infinite systems. Our approach does not depend on a particular tool in use. It consists in (i) abstracting a large (infinite) system into a smaller (finite) system; (ii) generating abstract test cases from the obtained system by an enumerative test generator; (iii) concretizing abstract test cases. We demonstrate how the approach is working for TGV, a tool for the generation of conformance test suites for protocols and CTE (Classification Tree Editor).To complete abstract test cases, we have to instantiate abstract test cases with concrete input data. The choice of input data for tests is not a key issue and should be well considered. An approach to generating input data should be supported by tools and should enable the reuse of test data between products and different stages of the software development process. The idea is to combine existing test generation methods based on state exploration with methods for partitioning input data like equivalence partitioning, boundary values analysis or CTM (Classification Tree Method). In this document, we show how CTM providing automatic generation of data can be integrated with TTCN-3.

The third issue considered in this document is mapping UML diagrams to testing techniques. Namely, we are interested which of testing techniques described in previous deliverables D.1.2.1 is suitable for which kind of UML diagrams.
In this document, we define directions for further investigation of this matter.

[D.1.2.2.2] Definition Of Methods for Automated Test Generation for TTCN3 Based Test Systems.

The objective of Task 1.2 of the TT-Medal project was to develop techniques for automated test generation and validation of TTCN-3 tests. This document is focused on automated test generation. As a starting point, we have considered existing tools for test generation already developed by industrial and academic communities. The basis for the generation is usually a formal and semiformal model.

UML has gained a great impact in industry. In this document we discuss ways to combine UML with existing testing techniques. A subset of UML diagrams that are used by industry most frequently has been selected and mapped to test generation approaches.

The major part of model based test generation tools explores the state space of a model or a subset of it. Therefore, the application of these tools to the systems of real industrial size is limited. In this document, we propose two approaches to mitigate this problem. Both of them employ abstraction and constraint solving techniques.

[D.1.2.3] Requirements And Concepts For Test Validation Of TTCN3 Tests.

The objective of Task 1.2 of the TT-Medal project is to develop techniques for automatic test generation and validation for TTCN-3. This document is focused on validation. The manual development of test cases and implementation of test systems are time consuming and potentially error prone. Therefore there is a need to validate test cases derived by hand as well as test systems implemented by the programmers.

Here we first consider a methodology based on refinement that guides test designers in development of valid test cases. To guarantee validity of test cases from different sources, we further propose two approaches to validation of the test cases: one by constraint solving and another one by simulation against a system specification.

Complexity of a system under test is usually reflected by the complexity of a test system for these SUT. Therefore, we provide an approach to automated generation of codecs that are responsible for correct data exchange between an SUT and a test system. Finally we assess techniques for test quality examination using metrics.

[D.1.3.1.1] General Requirements Of Reusable TTCN-3 Tests.

Reusable tests and testing methodologies are the main themes in the TT-Medal project. In the project, reusability is studied on different levels: reuse aspects of abstract test specifications (the TTCN-3 level), reusability aspects of parts of executable tests (the test system level) and how reusability effects test processes (the test process level). This document studies reusability on TTCN-3 level. Test reuse is approached by looking at those methods and techniques which have been proved successful in producing reusable software code. Then some of the special characteristics of test reuse and viewpoints to test reuse have been identified. Based on this groundwork, general requirements for reusable TTCN-3 have been identified.

This document is structured as follows: The first chapter is a short introduction. The second chapter provides an overview to central elements of software reuse. The third chapter presents the viewpoints of test reuse as well as test reuse in general. The fourth chapter outlines the general requirements for reusable TTCN-3 tests. The fifth chapter presents language requirements that can be later on issued to ETSI as change requests to improve the TTCN-3 language features and create new ones promoting the reusability of TTCN-3 test scripts. The final chapter draws conclusions on issues covered in this document.

[D.1.3.1.2] Guidelines And Patterns For Reusable TTCN-3 Tests.

Reusable tests and testing methodologies are the main themes in the TT-Medal project. In the project, reusability is studied on different levels: reuse aspects of abstract test specifications (the TTCN-3 level), reusability aspects of parts of executable tests (the test system level) and how reuse is included in test processes (the test process level). This document presents guidelines for reusable TTCN-3 tests to address reuse and reusability (mainly) on TTCN-3 level, but also presents use of patterns to address reuse and reusability on design level. General requirements for reusable TTCN-3 tests are presented in deliverable General Requirements for Reusable TTCN-3 Tests (1.3.1.1) and based on these, a set of detailed guidelines have been identified and presented in this document. Finally, the use of test patterns is studied to cover briefly the design issues of test reuse.

This document is structured as follows: The first chapter is a short introduction. The second chapter presents guidelines with illustrative examples. The third chapter presents test patterns and the final chapter draws conclusions on this document.

[D.1.3.2] A Process Model For Developing And Utilizing Reusable Test Assets.

While systematic reuse promises to shorten software development time, testing may become a bottleneck of otherwise efficient software development process. Development and utilization of reusable test assets could provide similar efficiency gains as expected from software reuse. Expectations towards test asset reuse could be even higher, as in addition to reusing test assets in testing different products (e.g. in a product family or different product generations), tests could be reused in different testing levels (e.g. unit and integration testing) or between different testing types (e.g. functional and performance testing).

Unlike software reuse, test reuse is a fairly new and unexplored field of study. No existing process approaches seem to be proposed that combines testing and reuse aspects. This document presents two general test development process models that take reuse activities into account. The two models offer the choice between revolutionary and evolutionary application of reuse issues in the test development process. They incorporate development for reuse and development with reuse aspects into test development process. The process models are generic by nature and thus are likely to require adaptations when put into practice.

These process models are one of the innovative contributions planned in section 1.3 of the Full Project Proposal of TT-MEDAL.

[D.1.3.3] Requirements Specification Of Test System Supporting Reuse.

Reusable tests and testing methodologies are the main themes in the TT-Medal project. In the project, reusability is studied on different levels: reuse aspects of abstract test specifications (the TTCN-3 language level), reusability aspects of parts of executable tests (the test system level) and how reusability effects test processes (the test process level). This document describes requirements to develop TTCN-3 test systems that support reuse on TTCN-3 test system and language level.

The requirements are investigated in two steps. First, we approach test reuse by looking at those software engineering approaches, techniques, and methods which have been proved successful in producing reusable software code. This step provides general requirements of TTCN-3 test system supporting reuse. Second, we study the architectural structure of TTCN-3 test system to produce more specific requirements for parts of executable tests. These requirements not only address test system reuse but also facilitate reuse of TTCN-3 tests. The second step investigates the reusability from vertical, horizontal, and historical viewpoints.

This document is structured as follows. The first chapter gives a short introduction. The second and third chapters provide an overview to central elements of software reuse. The fourth chapter presents general requirements for reusable software components and architecture. In chapter five, test reuse at the TTCN-3 test system level is defined. Reuse from vertical, horizontal, and historical viewpoints is investigated and more detailed requirements are provided in chapter six. Chapter seven and eight concentrate on surveying means to address the requirements presented in the previous chapter. The final chapter draws conclusions on reuse aspects on the TTCN-3 test system level.

[D.1.4.1.1] Testing Profiles.

In this document, we have presented various usage of UML 2.0 and TTCN-3 profiles. We outlined the differences of both profile definitions and presented several domain areas where these profiles can be applied to.

[D.1.4.1.2] Testing Profiles.

In this document, we have presented various usage of TTCN-3 profiles. We presented several domain areas where these profiles can be applied to, including telecommunication, financial and railway domains.

Work Package 2

• Test case definition using LogicaCGM’s TestFrame tool
• Syntax based derivation of valid test cases
• Codec validation

This document provides analysis and review of the deliverable “Definition of methods for automated test generation for TTCN-3 based test systems” (D.1.2-2) [1], which is a document that has been created within the TT-MEDAL industrial co-operation project. Three different approaches are analysed: automatic test generation based on the TGV tool and constraints; the integration of the Classification Tree Method to TTCN-3; and ideas on UML-based testing.

[D.2.4.2] Test Profile.

This report is based on the work package WP1 suggestions on methods and processes for implementing the UML test profile. Within this task the partners will specify the requirements on the tool platform from a tool vendor specific point of view.

The document describes the implementation of the transformator of U2TP to TTCN-3 as a plugin for Eclipse. Additionally, a graphical front-end for visualizing U2TP diagrams has been developed and its implementation documented.

[D.2.6.2] Demonstrator of the test infrastructure.

This document describes how to use the TFLtoTTCN3 application. This is a tool that generates Testing and Test Control Notation version 3 (TTCN-3) files out of TestFrame Language (TFL) files.

Work Package 3

Work Package 4