Measurement and monitoring of perceived video quality, perceived audio quality, audio loudness, blockiness, blur, contrast, flatness, jerkiness, rebuffering, black frames, zapping times, time to start video, noise, bloom, silences, saturations, video bitrate, audio bitrate
with dedicated perceived video quality metrics for HEVC (H.265), MPEG-4/AVC (H.264) and MPEG-2
VQM can even record GPS coordinates to measure video quality with respect to localization for field testing of 5G, 4G, 3G or Wifi networks
And VQM can also measure lipsync (synchronization between audio/video) using audio/video test signals which can be customized
VQM is scriptable. VQM can control Set-Top Boxes, and much more...
In 2016, 2017 and 2019, Video Quality Monitor received these awards:
Video Quality Monitor (VQM) is a precise and very flexible solution to measure and monitor audio video quality perceived by end-users (also called QoE: Quality of Experience).
VQM produces perceived video quality scores expressed on a MOS (Mean Opinion Score) scale. VQM's perceived video quality metrics are dedicated to HEVC (H.265), MPEG-4/AVC (H.264) and MPEG-2 video encoding formats. These video quality metrics have been optimized to produce quality scores which are highly correlated with human jugdments collected during subjective quality assessment tests in normalized conditions (ITU BT.500 and ITU BT.710).
VQM enables to measure and monitor perceived video quality and audio quality from:
VQM measures/detects/monitors in real time:
VQM is useful for:
VQM can also:
VQM supports lots of different sources: video files, capture cards (HDMI, SDI, Composite, etc.), external capture devices, desktop capture and IP streaming in UDP, RTP, HTTP, HTTPS, HLS (HTTP Live Streaming), RTSP or RTMP. VQM includes no reference, parametric and hybrid video quality metrics. It means that when you want to measure the perceived video quality of a distorted video, you just have to provide this video. For parametric and hybrid metrics, the video must be in a compressed format.
For monitoring of file-based encoding (example: encoding for VOD): VQM can process several files in parallel. This is particularly useful to process a large number of files (using the Watch Folder for example). VQM can even test the decodability of audio and video at about 40x speed (on a simple PC) and count the number of decoded audio samples and video frames. And if these values are different than the expected values (based on header information), VQM can send you alerts.
VQM is a software product. It works under any Windows version (XP, Vista, Seven, 8, 8.1, 10 and Server editions) and is easy to install (especially if you just want to use the no reference metrics, in this case you just have to run the installer and click on "Next..." several times). It can run on virtually any PC. It can even run on a laptop. If possible, the recommended configuration is: 4-core CPU, 3 GB of RAM, 10 GB of free hard disk space.
At start up, VQM displays a window showing a preview frame of the tested video (the video that you want to measure). To start using VQM, you generally use the "offline mode", click the "Browse..." button in order to select a video file and load it.
But VQM can also measure in real time the quality of video being captured live: that's the "online mode". In this case, VQM can process live video and/or audio from an HDMI or SDI capture device (or an external capture device), from IP streaming (using UDP, RTP, HTTP, HTTPS, RTSP or RTMP) or from Windows desktop capture.
VQM supports the most common encoders (HEVC (H.265), MPEG-4/AVC (H.264), MPEG-4/SP, MPEG-2, ...)) and the most common containers: AVI, MP4, TS, MOV, WMV, Raw YUV, PS, ES, ...
In fact, VQM can decode the images from nearly any codec and you can choose between using the integrated decoders or using DirectShow decoders. So if you can play your video files in a media player (like Windows Media Player, or Media Player Classic, etc...), then you can use VQM with these files. Videos can have any duration, any frame rate and any frame size. VQM also supports uncompressed YUV formats (4:2:0 planar, 4:2:2 planar, 4:2:2 UYVY, 4:2:2 YUYV, 4:4:4 planar).
And if your videos are Multi Program Transport Streams (MPTS), you can select the audio and video streams that you want to process.
Once a video is loaded, its first frame is displayed. The parameters of its video and audio streams are also displayed: codec, frame dimensions, frame rate, number of frames, duration, audio sampling rate, audio duration, etc.
For real time quality monitoring, the "online" mode enables to choose a live video/audio source : an HDMI or SDI capture card (or an external capture device), IP streaming (using UDP, RTP, HTTP, HTTPS, RTSP or RTMP) or Windows desktop capture (to process a YouTube window for example).
If one or several parameters were badly parsed from the file (this happens sometimes with non-standard compliant applications or broken files), all the parameters can be manually forced. And if you need to crop a video in order to remove black bands or to select only a part of the image, it can be done manually or automatically.
The interface also supplies a slide bar to seek to a particular frame. You also have buttons that enable you to view a given frame or play the video.
VQM's graphical user interface also displays many useful information. In fact, all the parameters that you could need are displayed and are editable.
Since VQM only needs the video under test (and doesn't need its reference video), it is very simple to use. You just have to select the appropriate metric, load your file/start your live source and start the video quality measurement.
During video quality measurement, several windows can be displayed: measured video frame, quality curves and video bitrate curve.
The estimated remaining time is also displayed, along with a progress bar. Of course, measurement can be paused or stopped. On the presented screenshot, three quality curves are displayed. Don't worry: all is explained in the help and in the generated analysis reports. Just click the "help" button.
In parallel of quality measurement, the instant video bitrate is measured for each frame.
Quality measurement uses multithreading. So the more CPU cores you have, the faster VQM works.
When quality has been measured, the shape of the perceived video quality curve (versus time) is known. So the minimum and maximum quality scores are known. Then the "characteristic frames" (the most disorted frames and the less distorted frames, determined thanks to adaptive thresholds that can be defined by the user) are saved in uncompressed format so that they can be consulted later, even if the video file has been moved or deleted. And don't worry: you won't get out of disk space due to the saved images. In fact, all file saving operations in VQM are controlled by size quotas on folders: when the size quota is reached for a folder, its older files are deleted.
So far, we've only described the main features of the main interface which is used for:
That's good but the best is yet to come...
To provide detailed reporting, VQM includes its own web server. There is no need to install a separate server (like Apache): VQM's web server is already fully included in the application. This web server enables you to connect from anywhere and get the measurement results, statistics, curves and even quality analysis reports (in several formats, including HTML format so that you can copy/paste it or load it in your favorite word processor, like Microsoft Word for example).
To connect to the web server, all you have to do is opening a web browser (Internet Explorer, Mozilla Firefox, Chrome, Safari, Opera...) and go to URL "http://IPADDRESSOFTHEMACHINE" (replace IPADDRESSOFTHEMACHINE by the IP address of the machine which runs VQM). On the local machine (that runs VQM), you can use the URL "http://127.0.0.1" to display the web interface. By default, the web server uses port 80 but if you already have a server listening on port 80, you can change this port number. The first page of this web interface is the main menu which proposes different choices.
For offline measurement results: In the main menu, the most frequently selected choice displays the list of processed videos. In this list, each video is represented by a unique number, its test campaign (that enables to gather several results), its file name, its full path, the description of its video and audio streams (codec, dimensions, frame rate, duration, etc...), its video quality score and statistics on video quality (min, max, standard deviation).
For each video, several links enable you to display:
Some other links permit to draw various representations of the perceived quality versus the bitrate or to display the characteristic frames. At last, some links allow to generate perceived video quality analysis reports in various formats (CSV, TXT, HTML).
For online monitoring results: Results (quality curves, bitrate curve, curves of blockiness, blur, contrast, jerkiness, rebuffering durations...) can be drawn between two user-chosen dates and times. The curves can also be drawn in real time, permitting to remotely see VQM's monitoring activity.
In the web interface, audio and video curves are interactive.Just click on a point of the curve and it will put this point at the center and zoom by a factor of 2. And select a portion of the curve (by dragging your mouse), it will zoom on this selection. If the frame corresponding to the center of the curve is a characteristic frame, then this frame will be displayed below the curve. And if you have processed several versions of the same video (at different bitrates, for example), you can draw all their curves on a single figure to easily compare them.
You can also get graphical representations of "perceived quality versus bitrate".
This will enable you to determine the bitrate which is required to get a given quality level.
VQM can also measure:
The measurements are made on the decoded audio waveforms and VQM supports the most common audio formats. To use a particular format, you'll just have to install the appropriate audio decoder.
In the main interface, a button enables to manually reset the audio breaks counter.
VQM can measure audio loudness according to international recommendations ITU BS 1770-2 and EBU R128. This loudness measurement computes 5 different values:
Loudness can be computed in both online and offline modes. This permits to monitor loudness in real time but also to measure loudness from captured signals (like audio video recordings).
Like the other measured data, all measured loudness values are saved and can be retrieved thanks to the web interface.
VQM can also trigger alerts if loudness is greater than a user-defined threshold.
At last, the integration durations for short term, medium term and gated loudness can be modified by the user.
VQM also supports scrips execution and infrared codes learning and sending.
This enables to control a Set-Top Box (STB) in order to test and validate a STB or to monitor several TV channels one after the other.
Each script contains actions enabling to control the 40 most used functions of VQM. Among these functions, VQM can send infra-red codes which were learnt beforehand.
Scripting is available for both online and offline modes. A script can even mix these two modes: during the online mode, when live video is captured, the video frames can be dumped to a file. Then, the recorded file can be processed in offline mode.
Sending and learning infra-red codes are performed using a device like the USB-UIRT emitter/receiver.
VQM can use a USB GPS antenna (like a bu-353s4 receiver) to record positions during video quality and/or audio quality measurement.
Then each measure is associated to its GPS information (latitude, longitude, altitude, number of used satellites, GPS clock).
VQM can generate maps showing video quality and/or quality with respect to GPS position.
This map indicates the video quality during the playing of a 4K video from YouTube in a car driving around the city of Nantes, France. One can notice a zone (in red, on top) which is very badly covered by the operator under test.
Put VQM in a backpack or in a car and measure your video quality depending on your localization!
VQM can generate audio/video test signals and use them to measure the synchronization between audio and video (lipsync).
The user can customize many parameters of the generated audio/video test signals:
You can even insert your own logo in a test signal.
VQM then saves audio/video tests signals as AVI files containing uncompressed YUV 4:2:2 video frames and uncompressed 16-bit audio samples (PCM).
The audio and video streams contain synchronization marks which are designed to be robust regarding audio and video encoding distortions.
So you can encode this file using the audio/video codecs you want to test. And you can pass the audio/video signal through any system or broadcasting chain you must test.
Thanks to the sample-accurate and frame-accurate synchronization marks contained in the audio/video streams, VQM can compute the skew (misalignement between audio and vidéo) and express it in milliseconds, from a captured file or in real time from an HDMI or SDI capture device.
VQM contains many other features, among which:
VQM can be used as a measurement probe which sends in real time its measures to a central server called "VQM Centralization Server" (VQM CS).
VQM CS enables to gather on a single machine the measures of several VQM instances. Like VQM, VQM CS includes a web server which enables to generate the measurement reports. VQM CS offers the same types of reports as VQM but for several VQM probes in the same graph or table.
VQM CS is included for free with VQM.
Enabling to work in both offline and online (real time) modes and with its elaborate quality metrics, VQM is definitely the solution that will adapt itself to answer your needs for video quality measurement and monitoring. Ask for an evaluation version today!
If you are interested by VQM but if you have only a punctual need or if you don't have the necessary manpower to fully exploit one license, please note that we also propose consulting services based on the use of VQM.