User manual OPTICOM PEVQ

[. . . ] 10 Contrast . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 PSNR. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Distortion Indicators. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 7 PRODUCT AVAILABILITY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 7. 1 STAND-ALONE OPERA PRODUCTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . [. . . ] Subjective tests require a number of test persons, which have to listen to an audio file or watch a video sequence, while they have to score them. Often a separate room needs to be installed were the tests can take place; appointments have to be made with the test persons, and then it often takes even days until the first test results become available. In some applications subjective tests are even prohibitive. However, subjective tests are a crucial task before objective measurement algorithms can be developed at all. This is because objective measurement algorithms model subjective tests. Only with a big fund of subjective measurement data objective algorithms can be made robust enough to accurately predict the human quality scoring. At OPTICOM we conducted such subjective measurements of video quality in cooperation with the Institute for Mobile Communications at the University of Erlangen-Nuernberg. The picture below shows the test room, the scoring device and the display where the video sequences were played back. Of course all the used equipment and the test procedure were selected and prepared to match the required standards of the ITU-T and the VQEG. Figure 4-1 Viewing test room at the University of Erlangen where OPTICOM conducted subjective tests. The test subject watches a scene on an adjusted TFT screen from a well defined distance. By pressing one of the keys, the test subject will be able to score the scene on the 5 point opinion scale. Page 6 of 12 ADVANCED PERCEPTUAL EVALUATION OF VIDEO QUALITY 5 INTRUSIVE TESTING Intrusive test methods insert a reference signal into the device under test1 (DUT) (near end side) while the processed and therefore possibly impaired file is recorded at the far end site (receiving side). Like in a subjective test the used video signals are natural2 video samples (audio is optional) with a length of several seconds. Using natural signals for the measurement is superior to applying artificial test signals, as they do not properly model a typical operation. After the coding/transmission procedure the corresponding reference file and the processed (impaired) file are compared to each other. Figure 5-1 A typical setup for an intrusive test: The tester outputs a reference stimulus that is inserted into a network connection at the `near end side' (origin), while the received signal at the `far end side' (termination) is fed back to the tester for difference analysis. However, due to the fact that the reference signal has to be inserted into the DUT, such measurements are called 'intrusive'. The DUT can include a wide variety of codecs, networks, bandwidth, service types, etc. The device under test can be anything, e. g. a video codec, a wired/wireless network transmission line, `natural' means in this context no artificially generated test samples, but samples of video content that is or will be most likely transmitted in such networks or coded by the appropriate codecs. 2 1 Page 7 of 12 ADVANCED PERCEPTUAL EVALUATION OF VIDEO QUALITY 6 PERCEPTUAL EVALUATION OF VIDEO QUALITY This chapter discusses OPTICOM's new video quality measurement algorithm in detail. Furthermore the activity of the motion in the reference signal provides another indicator representing the temporal information. This indicator is important as it takes into account that in frame series with low activity the perception of details is much higher than in frame series with quick motion. The third block in figure 1 classifies the previously calculated indicators and detects certain types of distortions. Finally, in the fourth block all the appropriate indicators according to the detected distortions are aggregated, forming the final result - the mean opinion score (MOS). The representation of the objective quality by a MOS is already known from speech quality models like PESQ and 3SQM. The MOS value describes the video quality on a range from 1 for very bad quality, to 5 for excellent quality. [. . . ] Under the mission statement "quality is our business", OPTICOM focuses on top-notch developments to gain for its customers improved quality in audio and video communications. With the new OPERATM family of perceptual analyzers, the company proves it's worldwide reputation for state-of-the-art solutions to improve the audio, video and multimedia quality of new media. Its President Michael Keyhl founded OPTICOM in 1995 as a "spin-off" company of the Fraunhofer-Institute, Germany's leading organization for applied research. OPTICOM's developers benefit from their broad experience in the research and development of perceptual based coding and evaluation techniques, such as MP3 and NMR, lasting back to the late 1980's. [. . . ]