Measuring VoIP Quality with MOS Score (Mean Opinion Score)
VoIP Quality is highly reliant on network performance, which means that many network problems like packet loss, latency, and jitter can cause high levels of VoIP degradation. To avoid embarrassing choppy voice calls, or lagginess during your next client meeting, we’re running you through how to measure VoIP Quality with MOS Score (Mean Opinion Score).
- Measuring with MOS (Mean Opinion Score)
- MOS Score vs VoIP Call Quality
- What is QoE for VoIP?
- QoS for VoIP Call Quality
- MOS Score Calculation from Network Performance Measurements
- Measuring MOS Score with Network & Device Monitoring
- Start Measuring VoIP Quality
Figuring out why VoIP calls are making you sound like a robot can be a challenge. Luckily Obkio’s VoIP monitoring solution gives you full visibility into your LAN, WAN, Internet and Service Provider Network, while keeping track of the metrics that affect VoIP Quality the most.
The Mean Opinion Score, usually referred to as MOS Score, has been the most commonly-used metric to measure the overall voice call quality for decades. Standardized by the International Telecommunications Union (ITU-T), MOS Score refers to:
A numerical measure of the human-judged overall quality of voice and video sessions.
Voice Quality MOS Score is most often judged on a scale from 1 (bad) to 5 (excellent) of the perceived quality of a voice call. Although originally Mean Opinion Scores were derived from surveys of expert observers, today a MOS Score is often produced by an Objective Measurement Method approximating a human ranking.
The MOS Score for Voice Quality was originally developed for traditional voice calls but has been adapted to Voice over IP (VoIP) in the ITU-T PESQ P.862.
The standard defines how to calculate MOS Score for VoIP Quality based on multiple factors such as the specific codec used for the VoIP call. Each VoIP codec (ex: G.711, G.722, G.723.1, G.729) behaves differently. Some codecs such as G.711 are uncompressed for higher quality but use more bandwidth than compressed codecs such as the G.729.
The MOS Score we measure is for the G.711 codec, which is by far the mostly used codec for VoIP calls. The maximum MOS Score for a G.711 call is 4.4.
When deciding how we would be measuring VoIP Quality in the Obkio app, we decided to measure it based on the MOS Score. Here are the details for how we measure VoIP MOS Score. The table below lists the different qualities and the lowest MOS Score limit for each of them. The limit values are from the ITU-T G.107 and ITU-T G.109 standards.
We also realized that most of our customers were not very familiar with the MOS Score and the interpretation of MOS in VoIP Quality. To help all our users understand more easily, we redesigned the MOS Score graph to create the VoIP Quality graph.
Obkio’s MOS VoIP Quality graph categorizes, for every minute, the MOS Call Quality as
- or Poor
The exact MOS Voice Quality Score is always available in the graph tooltip.
This feature is a great measure of the Quality of Experience (QoE) for users using VoIP applications over their network and helps IT Pros evaluate the impact of Network Performance on VoIP Applications.
When it comes to measuring the performance of any technologies or application, the user-perspective is your most important metric. Users identify good performance because essentially, good performance is the ability to meet a user’s expectations.
Quality of Experience (QoE) is a metric that allows you to measure performance from the end-user perspective to gain a better understanding of human quality metrics and be able to actually measure whether performance met a user’s expectations.
QUALINET (The European Network on Quality of Experience in Multimedia Systems and Services), describes QoE as:
“The degree of delight or annoyance of the user of an application or service. It results from the fulfillment of his or her expectations with respect to the utility and / or enjoyment of the application or service in the light of the user’s personality and current state.”
QoE allows you to measure the user’s perception of the effectiveness and quality of a system or service to essentially give you a performance standard. In fact, users base their opinions about the network exclusively on their perception of QoE.
QoE and VoIP Quality are two metrics that really work hand-in-hand. You can actually use VoIP Quality as a way to measure QoE, because after all, if poor VoIP Quality is leading to choppy calls, and robot voices, you can be pretty certain that your user is not having a pleasant experience.
Nowadays, IP networks are used to transport various types of applications which are a lot more sensitive to network performance and quality. One of those applications is VoIP.
This is why network engineers implement QoS (Quality of Service) to prioritize some traffic on the network in order to reduce latency, jitter and packet loss. In case of a network congestion, this will ensure that performance sensitive applications are always running without degradation and that only the less critical applications (such as web browsing) are impacted.
QoS (Quality of Service), is important for all VoIP Quality, because, at some point, voice and data will ‘mix’ on your network, or on the Internet. VoIP Call Quality can be measured using a variety of metrics, but it is more so perceived in regard to clarity of the voice quality heard at both ends of a call. The human ear is very sensitive, and will pick up on anything that may sound abnormal.
QoS allows us you set different priorities for different types of data Services on your network. To ensure that phone calls are good quality, we need to give VoIP traffic a higher priority than, for example, a download of a Windows 10 upgrade. That way, even with several computers doing upgrades on your network, calls are still crystal-clear.
VoIP Quality is about being able to get high-fidelity audio at each end of the phone call, without unwanted distortions. VoIP QoS will be affected by:
- Packet Loss
Obkio allows you to continuously monitor QoS with DSCP features - specifically the DSCP code located in the IP header.
For geeks that want to understand how we calculate the MOS Score from the Network Performance metrics, the standard ITU-T G.107 defines the algorithm we use to calculate the G.711 MOS Score based on the Network Performance metrics (latency, jitter and packet loss). The exact algorithm is available below in pseudo-code.
# VoIP Codec Processing Delay in milliseconds (ms) codec_delay = 10.0 # Effective Latency is the Average Latency plus the doubled # Average Jitter plus the VoIP Codec Delays effective_latency = average_latency + 2* average_jitter + codec_delay # Calculate the R-Value (Transmission Rating Factor R) based on # Effective Latency. The voice quality drops more significantly # over 160ms so the R-Value is penalized more. if effective_latency < 160: r = 93.2 - (effective_latency / 40) else: r = 93.2 - ((effective_latency - 120)/ 10) # Reduce R-Value 2.5 times the Packet Loss percentage r = r - 2.5 * packet_loss # Based on ITU-T G.107 if r < 0: mos = 1.0 else: mos = 1 + 0.035*r + 0.000007*r*(r-60)*(100-r)
When looking at how to calculate MOS Score, network metrics give you the most value and detail when they’re combined to give you a full overview.
Network Device Monitoring is a feature that allows you to monitor the performance of networking devices such as firewalls, routers, switches and wifi access points is crucial for IT teams. The Device Monitoring feature inside the Obkio Software is a fast and easy solution to get detailed information about the health of devices using Ultra-Fast Polling (every 30 seconds).
It’s important to find out what network problems are affecting VoIP Quality to be able to begin network troubleshooting. As I mentioned earlier, there are a variety of network problems that can affect VoIP performance, such as packet loss, jitter, and bandwidth.
Modern networks are vast infrastructures that can span over a variety of locations. When a problem arises, it is important to pinpoint where VoIP issues are located along your network so you know where to focus your troubleshooting efforts.
Your business may only have one IT specialist, or you may have a large IT department with different employees responsible for different parts of the network. Identifying who is responsible for dealing with a problem affecting VoIP Quality (user, application, network, or ISP), will help you decide who is responsible for fixing it.
Once you’ve collected all the information from the three previous points, you can assess the data to come to a resolution. It’s a given that having as much information as possible will lead to a quick and efficient solution.
You can check out the Network Performance Monitoring case study with Station 22, an international beverage manufacturer and distributor, to find out how they pinpointed undetected network problems on their firewall using MOS Score, Network Performance Monitoring, and Network Device Monitoring.
Learn how Station 22 solved their network issue from a 15-minute installation to a resolution in 48-hours with Obkio’s Network Performance Monitoring tool.Learn more
He’s a little summary about their problem and their solution.
Station 22 began experiencing VPN network application slowness and instability for their Microsoft Teams and GoToMeeting application.
Using Obkio’s Network Performance Monitoring software, they identified that their network latency was constant but the packet loss kept increasing. Specifically, there was a lot of packet loss occurring over the Internet as the bandwidth usage increased during the day, likely due to increased activity from users. They pinpointed the exact percentage of packet loss in the Obkio app with the graph tooltip.
They looked into the MOS Score chart to find that it reported thresholds well above what is normally acceptable.
Finally, they moved to Obkio's Network Device Monitoring feature on their firewall and saw that the CPU usage was well above the 40% threshold shown on their Firewall's GUI.
By using all three features, Station 22 was able to get a complete overview of all of the metrics affecting their network performance, and realized that the type of problem they were facing was known and could be fixed with a firmware update of the firewall. They immediately performed emergency maintenance and the problem was resolved in no time.
Network performance, specifically VoIP Quality, can be affected by a variety of different factors, which is why it’s important to get a complete, end-to-end overview of all these factors to truly understand what is wrong, and how to fix it.
VoIP issues make up 50% of IT problems!
VoIP travels a long distance from your network, through the Internet, and up to your Service provider. That means that, when a problem occurs, no matter where it is, you need to manage it.
Try Obkio’s VoIP Monitoring solution for free today to measure VoIP Quality with MOS Score and start fixing network problems before your users even know they exist!