Video Quality Monitor

Solution for no-reference measurement and monitoring of perceived video quality, perceived audio quality, audio loudness, blockiness, blur, contrast, flatness, jerkiness, black frames, noise, bloom, silences, saturations, video bitrate, audio bitrate

with dedicated perceived video quality metrics for HEVC (H.265), MPEG-4/AVC (H.264) and MPEG-2

In November 2016, Video Quality Monitor received this award:

 

Video Quality Monitor (VQM) is a precise and very flexible solution to measure and monitor audio video quality perceived by end-users (also called QoE: Quality of Experience).

VQM produces perceived video quality scores expressed on a MOS (Mean Opinion Score) scale. VQM's perceived video quality metrics are dedicated to HEVC (H.265), MPEG-4/AVC (H.264) and MPEG-2 video encoding formats. These video quality metrics have been optimized to produce quality scores which are highly correlated with human jugdments collected during subjective quality assessment tests in normalized conditions (ITU BT.500 and ITU BT.710).

VQM enables to perform:

  • perceived audio video quality measurement of audio video files, without reference
  • perceived audio video quality monitoring of live audio video sources in real time, without reference
  • perceived audio video quality monitoring of streaming services (like youtube, dailymotion or any other streaming service) using any web browser or player
  • measurement and monitoring of blockiness (sometimes called "blocking effect"), blur (sometimes called "blurring"), contrast, jerkiness (sometimes called "image freezing"), black frames, silences, saturations, video bitrate, audio bitrate
  • audio or video signal loss detection
  • audio loudness measurement and monitoring according to recommendations ITU BS 1770-2 and EBU R128 (as required by the CALM: Commercial Advertisement Loudness Mitigation Act)
  • video decoders (Set Top Box) test and validation
  • monitoring of file-based encoding (example: encoding for VOD)
  • measurement of the impact of rebuffering events on perceived video quality (rebuffering is when streaming video is paused for a while in order to buffer more data)
  • video cameras benchmarking and comparison using the camera noise metric
  • LCD screens benchmarking and comparison using the bloom metric
  • alerts triggering when problems happen
  • detection of the most complex scenes in long videos (to prepare complex videos for video encoders benchmarking)

VQM supports lots of different sources: video files, capture cards (SDI, Composite, etc.), external capture devices, desktop capture and IP streaming in UDP, RTP, RTSP, RTMP, HTTP or HLS (HTTP Live Streaming). VQM includes no reference, parametric and hybrid video quality metrics. It means that when you want to measure the perceived video quality of a distorted video, you just have to provide this video. For parametric and hybrid metrics, the video must be in a compressed format.

For monitoring of file-based encoding (example: encoding for VOD): VQM can process several files in parallel. This is particularly useful to process a large number of files (using the Watch Folder for example). VQM can even test the decodability of audio and video at about 40x speed (on a simple PC) and count the number of decoded audio samples and video frames. And if these values are different than the expected values (based on header information), VQM can send you alerts.

Starting VQM

At start up, VQM displays a window showing a preview frame of the tested video (the video that you want to measure). To start using VQM, you generally use the "offline mode", click the "Browse..." button in order to select a video file and load it.

But VQM can also measure in real time the quality of video being captured live: that's the "online mode". In this case, VQM can process audio and/or video from IP streaming, from desktop capture or from audio video capture devices (like a Blackmagic Design card for example).

VQM supports HEVC (H.265), MPEG-4/AVC (H.264) and MPEG-2 and the most common containers: AVI, TS, MP4, PS, ES, etc...

Videos loading

Once a video is loaded, its first frame is displayed. The parameters of its video and audio streams are also displayed: codec, frame dimensions, frame rate, number of frames, duration, audio sampling rate, audio duration, etc.

For real time monitoring, the "online" mode enables to choose the source between IP streaming, desktop capture and DirectShow compatible capture devices/cards.

VQM's graphical user interface also displays many useful information. In fact, all the parameters that you could need are displayed and are editable.

Measuring perceived video quality

During video quality measurement, several windows can be displayed: measured video frame, quality curves and video bitrate curve.

In parallel of quality measurement, the instant video bitrate is measured for each frame.

Quality measurement uses multithreading. So the more CPU cores you have, the faster VQM works.

Web server

So far, we've only described the main features of the main interface which is used for:

  • selecting a video quality metric
  • loading a video
  • starting the quality measurement step and waiting its completion
When these steps are done, the perceived video quality score is displayed, expressed in MOS (the signification of MOS values is fully explained in the help).

That's good but the best is yet to come...

VQM includes its own web server. There is no need to install a separate server (like Apache): VQM's web server is already fully included in the application. This web server enables you to connect from anywhere and get the measurement results, statistics, curves and even quality analysis reports (in several formats, including HTML format so that you can copy/paste it or load it in your favorite word processor, like Microsoft Word for example).

Results

For offline measurement results: In the main menu, the most frequently selected choice displays the list of processed videos. In this list, each video is represented by a unique number, its test campaign (that enables to gather several results), its file name, its full path, the description of its video and audio streams (codec, dimensions, frame rate, duration, etc...), its video quality score and statistics on video quality (min, max, standard deviation).

For each video, several links enable you to display:

  • the quality curves
  • the bitrate curve
  • the curves of blockiness, blur, contrast, jerkiness and camera noise
  • .

Some other links permit to draw various representations of the perceived quality versus the bitrate or to display the characteristic frames. At last, some links allow to generate perceived video quality analysis reports in various formats (CSV, TXT, HTML).

For online monitoring results: Results (quality curves, bitrate curve, curves of blockiness, blur, contrast and jerkiness) can be drawn between two user-chosen dates and times. The curves can also be drawn in real time, permitting to remotely see VQM's monitoring activity.

Curves

In the web interface, audio and video curves are interactive.

Just click on a point of the curve and it will put this point at the center and zoom by a factor of 2. And select a portion of the curve (by dragging your mouse), it will zoom on this selection.

If the frame corresponding to the center of the curve is a characteristic frame, then this frame will be displayed below the curve. And if you have processed several versions of the same video (at different bitrates, for example), you can draw all their curves on a single figure to easily compare them.

You can also get graphical representations of "perceived quality versus bitrate".

This will enable you to determine the bitrate which is required to get a given quality level.

Audio

VQM can also measure:

  • audio quality (based on bandwidth estimation, audio breaks, saturations and silences)
  • loudness, according to international recommendations ITU BS 1770-2 and EBU R128
  • audio bitrate

The measurements are made on the decoded audio waveforms and VQM supports the most common audio formats. To use a particular format, you'll just have to install the appropriate audio decoder.

In the main interface, a button enables to manually reset the audio breaks counter.

Loudness

VQM can measure audio loudness according to international recommendations ITU BS 1770-2 and EBU R128. This loudness measurement computes 5 different values:

  • Momentary loudness (denoted "M" in "EBU mode"): instant value of loudness.
  • Short term loudness (denoted "S" in "EBU mode"): mean value of loudness over 3 seconds.
  • Medium term loudness: mean value of loudness over 15 seconds.
  • Gated loudness: mean value of loudness over 15 seconds, ignoring the values which are below two thresholds (first threshold is absolute and equal to -70 LKFS, second threshold is relative and equal to mean value of remaining loudness measurements after first thresholding minus 10 LKFS).
  • Integrated loudness (denoted "I" in "EBU mode"): mean value of loudness over all the duration of the audio signal (even if the values proposed by EBU are recommended).

Loudness can be computed in both online and offline modes. This permits to monitor loudness in real time but also to measure loudness from captured signals (like audio video recordings).

Like the other measured data, all measured loudness values are saved and can be retrieved thanks to the web interface.

VQM can also trigger alerts if loudness is greater than a user-defined threshold.

At last, the integration durations for short term, medium term and gated loudness can be modified by the user.

Scripts and infrared support

VQM also supports scrips execution and infrared codes learning and sending.

This enables to control a Set-Top Box (STB) in order to test and validate a STB or to monitor several TV channels one after the other.

Each script contains actions enabling to control the 40 most used functions of VQM. Among these functions, VQM can send infra-red codes which were learnt beforehand.

Scripting is available for both online and offline modes. A script can even mix these two modes: during the online mode, when live video is captured, the video frames can be dumped to a file. Then, the recorded file can be processed in offline mode.

Other features

VQM contains many other features, among which:

  • Tasks Manager: when you click on a button, you don't directly perform an action but in fact you add a task in a queue. This enables you to continue loading files and measuring them, even if VQM is already working.
  • Watch Folder: it enables to detect every file appearing (copied or moved) in a folder (or its subfolders). When a file is detected, its quality measurement is launched. The Watch Folder enables to monitor a file-based encoding process using VQM (for VOD, for example)
  • Automation methods: to facilitate the use of VQM with an important number of files, VQM can load scripts ou be used in command line (with arguments).
  • Magnifying glass: to explore the image at pixel level and view the Y, U and V values of each pixel, a magnifying glass is available. This window shows the image content in the neighborhood of the mouse cursor's position.

Conclusion

Enabling to work in both offline and online (real time) modes and with its elaborate quality metrics, VQM is definitely the solution that will adapt itself to answer your needs for video quality measurement and monitoring. Ask for an evaluation version today!

If you are interested by VQM but if you have only a punctual need or if you don't have the necessary manpower to fully exploit one license, please note that we also propose consulting services based on the use of VQM.