Full-reference solution to measure perceived video quality, audio quality and audio/video synchronization (lipsync) from live sources: HDMI, SDI, IP streaming (UDP, RTP, HTTP, HTTPS, HLS, MPEG-DASH, RTSP, RTMP, NDI®), desktop capture, etc.
And also: measures of blockiness, blur, contrast and flatness for both the reference video and the measured video.
Also includes VMAF.
The ideal tool to monitor video encoders and broadcasting, in real time and with reference.
In 2023, Full Reference Video Quality Monitor received this award:
Full Reference Video Quality Monitor (FRVQM) is the ideal software solution to precisely measure and analyze video quality and audio quality perceived by end-users (also called QoE: Quality of Experience) using a reference video/audio signal.
FRVQM also enables to measure the skew between audio and video (lip sync offset).
FRVQM is mainly designed for:
FRVQM takes live audio/video signals as input:
FRVQM is a software product for Windows. It is very easy to install. It can run on virtually any PC. It can even run on a laptop. If possible, the recommended configuration is: 8-core CPU (or more), 8 GB of RAM.
FRVQM is based on full reference video quality metrics. It means that when you want to measure the perceived video quality of a distorted video, you have to provide the reference video of this distorted video. The reference video is the video which doesn't contain the distortions for which you want to measure the impact of perceived video quality. Typically, for video encoders benchmarking, the distorted video corresponds to the output of the encoder and the reference video corresponds to its input (so the distortions taken into account are the ones brought by the video encoder).
At start up, FRVQM displays a window showing two video preview frames: one for the reference video and one for the distorted video (that you want to measure).
FRVQM supports the most common encoding formats: MPEG-2, MPEG-4/AVC (H.264), HEVC (H.265), uncompressed YUV, etc.
The main window enables to trigger all the actions. The main actions are:
To start using FRVQM, you are going to select an audio/video source (audio/vidéo file, HDMI or SDI capture device, audio/vidéo source device, IP streaming, desktop capture).
Then you are going to detect the settings of this source, select the settings you want (for example the HDMI capture resolution, or which IP stream should be used) and run this source. You can then start capturing video frames and audio samples coming from this source. Then you'll be able to start synchronizing or measuring (or both).
These operations must be done for the 2 audio/vidéo sources: the reference signal and the signal under test.
Some buttons enable to do all these operations in one click.
FRVQM's graphical user interface also displays lots of other useful information, mainly for synchronization and measurement. In fact, all the parameters that you could need are displayed and are editable.
To avoid making errors, several values can interact together and if you modify one of them, the others will be updated automatically. The most difficult task is to determine the synchronization parameters (see below) but luckily they can be detected automatically. And anyway, most of the time, you won't have to manually enter or modify any parameter since the detected values and the default values will work in most cases.
Maybe you're already aware of that but measuring the quality of a signal using a reference signal requires to temporally and spatially synchronize them. Synchronization is also sometimes called "alignment".
Synchronization is necessary because, for example, the frame number 1 of the distorted video can correspond to the frame number 5 in the reference video. In this case, a temporal offset of 4 frames must be taken into account when comparing the two videos. Also, the video content (the objects in the image) may have been shifted, for example, by 1 pixel horizontally and 2 pixels vertically. If such a displacement has occured, it must also be taken into account when comparing pixels or regions between the two videos.
To detect the temporal and spatial offsets, you just have to define the size of search windows. Then FRVQM will automatically detect the offsets.
During synchronization, monitoring windows can display the comparisons performed between the two videos signals or the two audio signals.
When the synchronization step completes, the temporal and spatial offsets are detected (and copied in the appropriate fields of the main window), you can start audio/video quality measurement.
FRVQM contains several video quality metrics (concerning the audio quality metrics, see hereafter) :
With the perceptual metrics, FRVQM measures video quality indicators like blockiness, blur, contrast and flatness, and combine them to produce video quality scores highly correlated with human judgment. To achieve that, FRVQM's video quality metrics having been trained using the results of subjective quality assessment tests performed with human observers.
FRVQM produces video quality scores expressed on a DMOS scale (Differential Mean Opinion Score).
During video quality measurement, many windows can be displayed: measured video frame, reference video frame, seamless view (left part of the reference frame and right part of the measured frame), errors map, video and audio quality curves, video quality indicators curves and bitrate curve.
On the presented screenshot, three quality curves are displayed. Don't worry: all is explained in the help and in the generated analysis reports. Just click the "help" button.
In parallel of quality measurement, the instant video bitrate is measured for each frame.
Both the synchronization step and the quality measurement step use multithreading. So the more CPU cores you have, the faster FRVQM works.
When you've done that (starting the 2 audio/video sources, synchronizing and starting quality measurement), the perceived video quality score is displayed, with other information about the measurements being performed. But you can retrieve even more information using the web interface.
FRVQM includes its own web server. There is no need to install a separate server (like Apache): FRVQM's web server is already fully included in the application. This web server enables you to connect from anywhere and get the measurement results, statistics, curves and even quality analysis reports (in several formats, including HTML format so that you can copy/paste it or load it in your favorite word processor, like Microsoft Word for example).
All you have to do is to open a web browser (Internet Explorer, Mozilla Firefox, Chrome, Safari, Opera...) and to go to URL "http://IPADDRESSOFTHEMACHINE" (replace IPADDRESSOFTHEMACHINE by the IP address of the machine which runs FRVQM). On the local machine (that runs FRVQM), you can use the URL "http://127.0.0.1" to display the web interface. By default, the web server uses port 80 but if you already have a server listening on port 80, you can change this port number. The first page of this web interface is the main menu which proposes different choices.
The main menu enables you to have access to the results of each monitoring session (a monitoring session represents the amount of results measured between the moment you started quality monitoring and the moment you stopped it).
Above each curve, several links enable you to display the quality curves, the bitrate curve, the characteristic frames. Some other links permit to draw various representations of the perceived quality versus the bitrate. At last, some links allow to generate perceived video quality analysis reports in various formats (CSV, TXT, HTML).
In the web interface, all curves are interactive: if you click on one point of the curve, it will put this point at the center and zoom by a factor of 2. And if you drag your mouse to select a portion of the curve, it will zoom on this selection. If the frame corresponding to the center of the curve is a characteristic frame, then this frame and its reference frame will be displayed below the curve. And if the distorted frame contains areas that have been distorted beyond an acceptability threshold defined by the user, then these areas will be boxed, so that you can find locate easily. The color of the box indicates the level of distortion (the clearer the color, the higher the distortions). And if the boxes are annoying, you can make them invisible using a single click.
If several distorted videos have the same reference video, then you can draw all their curves on the same figure, in order to compare the effects of different encoders or different bitrates on a given video content. You can also get graphical representations of the perceived quality versus the bitrate. This will enable you to determine the bitrate which is required to get a given quality level.
FRVQM can also measure audio quality and loudness (according to international standards ITU BS.1770 and EBU R.128).
FRVQM enables you to choose between quality metric between the MSE and PSNR or PSNR + (which is an "improved PSNR", the improvement being due to audio gain compensation).
When audio processing is enabled, FRVQM performs audio synchronization, audio quality measurement, instant audio bitrate measurement and mean audio bitrate computation.
Apart from video and audio quality, FRVQM can also measure skew (sometimes called lip-sync offset) which indicates how audio and video are synchronized.
To do that, FRVQM uses the same technology as our other product Audio Video Sync & Quality Monitor (AVSQM). FRVQM measures skew for files whereas AVSQM measures it for live sources.
Like in AVSM, the skew is expressed in milliseconds. Perfectly synchronized audio video signals have a skew of 0 ms (no delay between audio and video). The skew is positive when audio arrives before video (for example, you hear a voice before you can see someone talking) and negative when video comes before audio (for example, you can see someone starting to talk but it takes a while before you can hear the voice).
Measured skew values can be used to trigger warnings or errors, thanks to user-defined thresholds.
To measure skew, audio and video signals from the measured file and the reference file are transformed in audio and video fingerprints. These fingerprints describe the audio signal and each video frame under the form of a quasi-unique signature. Fingerprints are robust to encoding, re-encoding, transcoding and resizing. Each fingerprint also contains the timestamp at which it was computed. FRVQM compares audio and video fingerprints between the measured file and the reference file. This enables to precisely measure the time offsets (for both video frames and audio signals) between the two files. Then, these time offsets enable to compute the skew (also called lip-sync offset) between the two files.
FRVQM includes AccepTV's full-reference video quality metrics but also Netflix's VMAF metric.
By using VMAF in FRVQM, you can compute the mean VMAF value of each processed video but you can also:
With all these features (and more, like scripting), FRVQM is a perfect tool to use VMAF!
Thanks to its elaborate metrics, FRVQM is the ideal tool to measure video quality, audio quality and synchronization between audio and video (lipsync) from live audio/video signals and with reference.
Ask for an evaluation version today!