This document specifies the framework, concepts, methodology for testing, and criteria to be achieved to claim conformance to multiple parts of the ISO/IEC 21122 series. It lists the conformance testing procedures.

This document specifies:

—    Conformance testing procedures for decoders implementing ISO/IEC 21122-1.

—    Tests to check which conformance point an ISO/IEC 21122-1 decoder conforms to, that is, whether a decoder satisfies the error bounds required for strict or relaxed conformance.

—    Conformance testing procedures for decoders implementing ISO/IEC 21122-3.

—    Tests to check codestreams for conformance to ISO/IEC 21122-1. As such, it provides means to test whether encoder implementations generate syntactically correct codestreams, and whether codestreams generated by such implementations follow the requirements of a particular profile, level and sublevel, and the buffer model implied by them.

—    Tests to check files for conformance to ISO/IEC 21122-3.

—    Conformance testing procedures that allow testing whether codestreams conform to any of the profiles specified in ISO/IEC 21122-2.

—    Conformance testing procedures that allow testing whether codestreams conform to the buffer model specified in ISO/IEC 21122-2 as part of a profile, level and sublevel.

—    Codestreams, decoded images, and error metrics to be used within the decoder testing procedures.

—    A buffer model test.

—    Abstract test suites.

NOTE: This document does not specify:

—     Testing the reconstruction of a full resolution image from a subsampled image format. In particular, upsampling from 4:2:2 or 4:2:0 to 4:4:4 sampling is a non-normative extension and as such its testing is beyond the scope of this document.

—     Testing the conversion of the sample values reconstructed by an ISO/IEC 21122-3 decoder to the target colour space by means of the colour specification box of ISO/IEC 21122-3.

—     Testing of the composition of background and foreground for images reconstructed from ISO/IEC 21122-3 files or codestreams that contain auxiliary channels carrying opacity information.

—     Testing of the interpolation of a colour filter array image to a full scale colour image; this process is not normatively defined and beyond the scope of this document.

—     Acceptance testing: the process of determining whether an implementation satisfies acceptance criteria and enables the user to determine whether or not to accept the implementation. This includes the planning and execution of several kinds of tests (e.g. functionality, quality, and speed performance testing) that demonstrate that the implementation satisfies the user requirements.

—     Performance testing: measures the performance characteristics of an implementation under test (IUT) such as its throughput, responsiveness, etc. under various conditions.

—     Robustness testing: the process of determining how well an implementation is able to conceal problems from attempting to reconstruct an image from an ill-formed codestream.