WTSA Resolution 76 – Studies related to conformance and interoperability testing, assistance to developing countries, and a possible future ITU Mark programme – resolves that ITU-T Study Group 11 coordinates the sector’s activities related to the ITU Conformance and Interoperability (C&I) programme across all study groups and reviews the Recommendations in the C&I business plan for the long term implementation of the C&I programme.

ITU-T is producing a large number of Recommendations. To achieve interoperability and conformity, one of the important aspects of the ITU C&I Programme concerns the development and maintenance of testing frameworks and methodologies.

It is essential that conformance and interoperability testing methodologies used by all study groups engaged in testing are aligned and consistent with each other. To achieve interoperability on a global scale, the ITU-T Recommendations must be developed and maintained with conformance and interoperability in mind according to the relevant methodology.

The objective of conformance testing is to determine how completely and correctly the requirements stated in the Recommendation have been met by the implementation. On the contrary, in interoperability testing, the objective is to determine if two or more implementations of the same Recommendation communicate and correctly exchange information with each other. It is generally assumed that the conformity of an implementation has been tested prior to perform an interoperability testing assessment.

The recent trends (such as IMT-2020, IoT technologies) may lead to many changes with regards to the extant network architecture that will require higher network performance. This in turn would impact on the specifications of the terminal unit as in CPE, Mobile Units, and Phones etc.

Most of the telecom operators are implementing various emerging technologies and migrate from circuit-switched to the packet-switched networks, trying to provide their services using “all over IP” concept. As a result, operators face some issues which are in general related to the compatibility and interoperability of the used ICT equipment and the interconnection of IP-based networks (e.g. 4G, IMT-2020), which, among others, will be used for roaming/nomadic services. For example, providing voice and video over LTE (VoLTE/ViLTE) roaming services pose some challenges to the operators due to the lack of agreed roaming procedures among stakeholders, various available VoLTE/ViLTE implementation scenarios and other not standardized issues (e.g. ENUM, emergency call, etc.). The conformance and interoperability testing of network-network interfaces (NNI) against ITU-T Recommendations may help operators to be sure that their VoLTE/ViLTE solutions are ready for interconnection. This interconnection approach may be also used for future packet-based networks, e.g. IMT-2020 network and beyond.

In general, benchmarking is a common approach used for the measurement and testing of signalling parameters against the performance design objectives, which should help to provide the end to end service delivery and ensure network reliability.

Benchmarking in the case of IMT-2020 is not limited only to the transport layer and includes performance, quality and reliability of virtual platforms.

The issues of finding the values of network performance and productivity for services with required quality of service (QoS) is an important question for operators and their users.

For example, the Internet-related performance measurement systems, which are publicly available on the Internet, do not provide reliable and comparable measurements. Beyond the lack of standardized metrics for Internet speed in ITU-T Recommendations, the obtained test results, which can be achieved by an existing testing method, may vary from the results achieved by other methods. Obviously, the testing results depend on the network segments which are used during e2e connection. Particularly, it is not possible to guarantee that the e2e connection is based on the telecom operator’s network only and does not include other network segments which may belong to other operators.

The development of a unified approach to Internet-related performance measurement is important for all ICT stakeholders.

Also, this standardized approach will support the development of the “Framework for ICT service delivery with the guaranteed QoS and requested bitrate on fixed & mobile data networks, for development of efficient economic mechanisms and models of interaction in the "operator-provider-user" chain”.

NOTE − The network performance QoS/QoE, including the networks and services KPIs, parameters and requirements, are defined by ITU-T SG12 and related SDOs. This Question is responsible for Q.3900-Q.4099-series (testing for next generation networks), Q.1912.x-series, X.290-series (except X.292), X.Suppl.4, X.Suppl.5 and Z.500-series.