Search

Telecom Decision CRTC 2018-241

PDF version

Reference: 2018-241-1

Ottawa, 13 July 2018

Public record: 8663-C12-201503186 and 8621-C12-01/08

CISC Network Working Group – Non-consensus report on quality of service metrics to define high-quality fixed broadband Internet access service

With this decision, the Commission further defines the universal service objective by establishing the broadband quality of service (QoS) that should be provided to all Canadians. The Commission determines that to meet the broadband portion of the universal service objective, fixed broadband Internet access service is defined as a high-quality service if it provides the subscriber with a smooth experience when using real-time QoS-critical applications, as described in this decision. Specifically, the Commission establishes a round-trip latency threshold of 50 milliseconds, and a packet loss threshold of 0.25%, both based on measurement during peak times. The Commission is launching, today, a separate proceeding to establish an appropriate QoS metric for jitter.

Introduction

  1. In Telecom Regulatory Policy 2016-496, the Commission determined that the availability of fixed broadband Internet access service offerings that meet certain levels of speeds, data allowance, and quality of service (QoS) will help ensure that Canadians are receiving services that meet their needs and enable them to participate in the digital economy. Accordingly, the Commission established a universal service objective: Canadians, in urban areas as well as in rural and remote areas, have access to voice services and broadband Internet access services, on both fixed and mobile wireless networks. To measure the successful achievement of this objective, the Commission established several criteria, including,
    • Canadian residential and business fixed broadband Internet access service subscribers should be able to access speeds of at least 50 megabits per second (Mbps) download and 10 Mbps upload, and to subscribe to a service offering with an unlimited data allowance; and
    • the latest generally deployed mobile wireless technology should be available not only in Canadian homes and businesses, but also on as many major transportation roads as possible in Canada.
  2. The Commission also determined that the QoS levels for latency,Footnote 1 jitter,Footnote 2 and packet lossFootnote 3 need to be established to define high-quality fixed broadband Internet access service and measure the successful achievement of the broadband portion of the universal service objective, in addition to the above-mentioned criteria. The Commission considered that the CRTC Interconnection Steering Committee (CISC) would offer an opportunity for many different parties with technical expertise to provide input on appropriate QoS metrics and measurement methodology.
  3. Accordingly, the Commission requested that CISC review and make recommendations on appropriate metrics for latency, jitter, and packet loss to define high-quality fixed broadband Internet access service. These recommendations were to include (i) technical specifications, (ii) the identification of points of interconnection in the Internet service providers’ (ISPs) networks where these metrics would apply, and (iii) the methods by which data on the service metrics could be collected and reported by ISPs in a consistent manner. The Commission expected that the QoS metrics would reflect the objective that broadband Internet access services in rural and remote areas be of similar high quality as those in urban areas.

Report

  1. The CISC Network Working Group (NTWG) submitted the following non-consensus report, dated 29 November 2017, for the Commission’s consideration:
    • Develop recommendations as to the appropriate metrics and reporting to define high-quality fixed broadband Internet access service (NTRE061) [the NTWG Report]
  2. The NTWG Report can be found under the “Reports” section of the NTWG page, which is available under the CISC section of the Commission’s website at www.crtc.gc.ca.

Issues

  1. The Commission has identified the following issues to be addressed in this decision:
    • What constitutes an ISP’s broadband Internet access network?
    • What measurement methodology should be used?
    • What are appropriate metrics to define high-quality fixed broadband Internet access service?

What constitutes an ISP’s broadband Internet access network?

Positions of parties

  1. NTWG participants (hereafter, “parties”) noted that typically, the ISP supplies a modem or gateway at the customer premises, or customers purchase their own modem.Footnote 4 They indicated that the modem is the starting point of a customer’s home wireless (Wi-Fi) or wired local area network. The customer’s home network directly connects to the customer’s computers, laptops, smartphones, tablets, video game consoles, and potentially many other devices. The parties noted that the devices that customers use could affect QoS, but since these devices are not supplied by the ISPs, they are not within the ISPs’ control. As a result, the parties submitted that the customer’s home network is not part of an ISP’s fixed broadband Internet access network.
  2. SSi Micro Ltd. and the Eeyou Communications Network submitted that the transport networksFootnote 5 should be included in QoS measurement. This is because ISPs providing broadband Internet access service far from Canadian Tier 1 citiesFootnote 6 have to purchase Internet Protocol (IP) transit services, typically from larger ISPs, to connect to the appropriate Internet exchange point (IXP).Footnote 7 These parties also submitted that their choice of IP transit service provider is largely based on balancing the quality and cost of the service; hence, the transit network could have a significant impact on QoS performance and the end-customer experience.
  3. Bell Canada noted that not all ISPs are connected to IXPs in Canadian Tier 1 cities and that some Canadian ISPs are connected to United States-based IXPs. Parties indicated that some ISPs exchange traffic with each other at private interconnection points in Canadian Tier 1 cities.
  4. Parties agreed that including the QoS of the global Internet beyond Canadian Tier 1 cities would not be appropriate, since this does not constitute Canadian ISPs’ fixed broadband Internet access network. Parties noted that it would be impossible for Canadian ISPs to measure QoS beyond Canadian Tier 1 cities into the global Internet. Consequently, parties agreed that broadband QoS should be measured in Canadian Tier 1 cities.  
  5. The Canadian Internet Registration Authority (CIRA), along with Fenwick McKelvey (Concordia University), the Cree Nation / Eeyou Communications Network, and Herb Charles (independent consultant) [collectively, CIRA et al.], Rogers Communications Canada Inc. (RCCI), Clearcable Networks, and SamKnows Ltd. (SamKnows)Footnote 8 recommended that the IXPs in Canadian Tier 1 cities should be the end-point of ISPs’ fixed broadband Internet access network where QoS should be measured.

Commission’s analysis and determinations

  1. The primary purpose of an ISP’s broadband Internet access network is to connect broadband service subscribers to the Internet. Once connected, subscribers can access various Internet-based services and applications, hosted in Canada and globally.
  2. A typical ISP network starts from the customer premises to an IXP or a private interconnection point in Canada.
  3. The Commission considers that a customer’s home network and devices beyond the modem are not part of ISPs’ fixed broadband Internet access network, since they are typically not supplied by or within the control of the ISPs but could affect QoS measurements. As such, it would not be appropriate for broadband QoS measurement to include the performance of customer devices and the customer’s home network. Accordingly, the Commission determines that QoS measurement starting from the customer premises should take place at the modem.
  4. ISPs that use an IP transit service typically do so to carry their customers’ Internet traffic to or from an IXP. The ISP is responsible for providing or choosing the IP transit service provider or routes; therefore, the IP transit service is part of ISPs’ fixed broadband Internet access network.
  5. It was not the Commission’s goal to determine the QoS of the global Internet or United States-based IXPs. The Commission’s goal is to measure the QoS of Canadian ISPs’ fixed broadband Internet access service, and since the IXPs in Canadian Tier 1 cities are well-established points of interconnection where ISPs typically interconnect to exchange Internet traffic within Canada and with the rest of the global Internet, these IXPs are appropriate end-points for QoS measurement. The record indicates that ISPs that do not connect to the IXPs in Canadian Tier 1 cities can establish these connections specifically for the purpose of QoS measurement.
  6. In light of all the above, the Commission determines that the parts of an ISP’s fixed broadband Internet access network to which QoS measurement should apply include all network elements from the modem at the customer premises to a point of interconnection at an IXP in a Canadian Tier 1 city.

What measurement methodology should be used?

Background

  1. The measurement methodology identifies (i) the points of interconnection in the ISPs’ networks where QoS is measured, and (ii) the method by which QoS performance data should be collected and reported by ISPs across Canada, in a consistent manner.
  2. The Commission launched the Broadband Measurement Project in 2015 to objectively measure broadband Internet performance in Canadian homes. This Project is a collaboration between the Commission and major Canadian ISPs. SamKnows conducted the broadband QoS measurement study in Canada on behalf of the Commission, and submitted two broadband measurement reports (one in December 2016 and one in April 2016) [hereafter, “the broadband measurement reports”].
  3. While the primary focus of the Broadband Measurement Project is to measure actual Internet connection speeds, ISPs’ performance data (latency, packet loss, and jitter) are also measured. In its CRTC Three-Year Plan 2017-2020, the Commission indicated that it would continue to collect performance data from participants and expand the Project to include more ISPs and performance measurement parameters.

Positions of parties

  1. Parties proposed the three QoS measurement options below.
Option 1: From the customer premises to an IXP
  1. The Canadian Network Operators Consortium Inc. (CNOC), CIRA et al., Clearcable Networks, RCCI, and SamKnows submitted that broadband QoS measurement from the customer premises, either at the modem or the customer’s computer/device, to a measurement serverFootnote 9 located off-netFootnote 10 at the IXP is the only way to ensure that the full extent of the ISP’s network is measured and is the best practice in broadband QoS measurement.
  2. CIRA submitted that to support broadband QoS measurement, it has deployed an Internet Performance Test in the form of off-net measurement servers at the IXPs in all the Canadian Tier 1 cities, and that these servers can be used for free by any ISP in Canada in the manner proposed in Option 1.
  3. CIRA et al. recommended Option 1 since it aligns with third-party broadband QoS measurement initiatives, such as the measurement study conducted by SamKnows, and CIRA’s Internet Performance Test. These initiatives rely on established measurement standards that are already used in regulatory contexts globally and have clear explanations of their methodology. CIRA et al. also submitted that the Commission’s Broadband Measurement Project should be part of an ongoing QoS monitoring system by the Commission.
  4. SamKnows noted that it uses Option 1 exclusively in almost all the work it carries out for telecommunications regulators globally, including for the Commission’s Broadband Measurement Project. SamKnows supported Option 1 since it ensures that the same measurement software is used consistently across all measurement probes.Footnote 11
  5. Bell Canada, Cogeco Communications Inc., Quebecor Media Inc., and TELUS Communications Inc. (collectively, Bell Canada et al.) and CNOC expressed the concerns that deploying third-party probes in customers’ homes in every community throughout Canada is not cost effective, and that it would be difficult to recruit volunteers who would permit such devices to be installed in their homes. Bell Canada et al. also noted that such measurement would not be completed within reasonable time frames.
  6. The parties that supported Option 1 argued that they did not propose deploying third-party probes in the manner that Bell Canada et al. and CNOC were concerned about, since Option 1 involves a sample-based approach.
Option 2: From the access aggregation pointFootnote 12 to the on-net border routerFootnote 13
  1. Bell Canada et al. and the Independent Telecommunications Providers Association (ITPA) proposed that broadband QoS measurement take place at or near the access aggregation point to the on-net border router. However, they indicated that if broadband QoS measurement of the access network falls within 20 milliseconds (ms) of the latency threshold to be established by the Commission, ISPs should continue to conduct measurements at the modem in the customer premises to confirm whether or not the established latency threshold is met. If the result is more than 20 ms below the latency threshold to be established by the Commission, the households served by that central office or cable head-end should be assumed to fall below the established threshold.
  2. Bell Canada et al. acknowledged that this measurement methodology was not appropriate for some ISPs, and proposed that the Commission give ISPs the choice of using Option 1 or 2, based on the resources available to them. CNOC supported giving ISPs this choice.
  3. Bell Canada et al. and the ITPA proposed partial network measurements, based on the cost and effort associated with measurement and reporting. The ITPA noted that the Policy DirectionFootnote 14 requires that any new regulatory measure imposed by the Commission be efficient and proportionate to its purpose and interfere with the operation of competitive market forces to the minimum extent necessary.
  4. SamKnows noted that on-net measurement servers are typically not used in public reporting of ISPs’ performance measurement.
  5. CIRA et al. did not support Option 2. They submitted that this option does not reflect or accurately capture an ISP’s broadband QoS performance, ignores the performance of critical parts of an ISP’s network, or factors those parts with unsupported estimates. CIRA et al. added that Option 2 lacks transparency and does not enable the Commission to use an independent third party for broadband QoS measurement.
  6.  CIRA et al. noted that the parties that supported Option 2 did not address basic design components, such as the measurement protocols to be used, the sampling approach and scheduling, and the calculation of average, maximum, and minimum performance. They added that Option 2 would require significant elaboration before being put into practice.
Option 3: From the access aggregation point to an off-net server at an IXP
  1. Shaw Communications Inc. (Shaw) proposed that if the purpose of broadband QoS measurement is to determine ISPs’ QoS performance nationally, the company supports measurement from the access aggregation point to an off-net server at an IXP. Shaw submitted that this option provides a transparent and fair measurement platform to objectively compare end-to-end services between ISPs.
  2. Shaw also submitted that if the measurement results at the access aggregation point are close to the high-quality latency threshold to be determined by the Commission, a sufficient number of customer premises should be measured to confirm that the broadband Internet access service meets the threshold. As well, Shaw argued that ISPs should have the flexibility to conduct broadband QoS measurement based on the resources available to them, which may include remotely initiated measurement from the ISP-provided router gateway in a customer premises, if technically feasible, or testing by a technician on-site.

Commission’s analysis and determinations

  1. The Commission considers that the chosen methodology for broadband QoS measurement should accurately capture ISPs’ actual broadband network access QoS performance, as well as subscribers’ actual real-world experience. It should also take into account factors such as consistency (i.e. the broadband QoS results from different ISPs must be comparable or equivalent so that they can be aggregated to give a national assessment of QoS performance), accessibility (i.e. the measurement points or equipment should be accessible to all ISPs that use a given network and to third-party measurement organizations, and not require them to install their own equipment), fairness (i.e. the measurement methodology should be neutral and prevent more favourable QoS results from one ISP over another), and the burden on ISPs.
  2. The parties that proposed partial network measurements in Options 2 and 3 did so primarily to reduce the cost and effort of conducting measurements. These factors should be considered in the assessment of the potential burden of broadband QoS measurement on ISPs. However, the Commission considers that for the broadband QoS measurement methodology to be accurate and fair, ISPs would have to measure their entire broadband Internet network. Option 1 is the only option through which (i) the entire network is measured, and (ii) all ISP performance is measured from the same network points. Under Options 2 and 3, some ISPs would measure only parts of their network, while other ISPs would measure their entire network, creating unfairness and inconsistency in the measurements. In addition, Options 2 and 3 are based on making assumptions regarding the parts of the network that were not being measured.
  3. As such, Options 2 and 3 would lead to unfairness among ISPs in demonstrating their broadband QoS performance, since different ISPs’ access networks have significantly different broadband QoS performance depending on technology and design. The use of one estimate for all ISPs would be inaccurate in measuring broadband QoS performance. Furthermore, the appropriateness and feasibility of using estimates to determine ISPs’ packet loss and jitter QoS performance have not been demonstrated.
  4. As well, Options 2 and 3 would not enable competitive ISPs, or those that use IP transit service, to measure their broadband QoS, since the measurement points would be in network locations to which these ISPs do not have access. Further, if these ISPs were given access to those network locations, they may incur costs to purchase, install, and maintain their measurement equipment, rather than using existing and shared equipment provided by third parties, as would be the case with Option 1.
  5. Since broadband QoS measurement is a long-term activity, it is important that ISPs’ performance be measured repeatedly over time. The parties that supported Options 2 and 3 noted that these methodologies can be conducted on a one-time basis as required, but not regularly, due to the burden on ISPs and completion time.
  6. Furthermore, Option 1 is the only option that supports the use of an independent third party to measure broadband QoS. The Commission considers that the use of a third party ensures fairness and consistency in broadband QoS measurement, since the same measurement probes, servers, software, protocols, and algorithms would be used by all ISPs. The use of an independent third party also ensures that QoS measurement is implemented in a symmetrical and competitively neutral manner, in line with subparagraph 1(b)(iii) of the Policy Direction. The parties that supported Options 2 and 3 did not indicate which probes, servers, software, protocols, or algorithms would be used. In addition, Options 2 and 3 would not ensure consistency, symmetry, or the implementation of the QoS measurements in a competitively neutral manner.
  7. In addition to determining where in an ISP’s network broadband QoS should be measured, it is important to determine when these measurements should be taken. When congestion occurs, subscribers may not receive high-quality broadband Internet access service. Therefore, the period that best indicates overall QoS performance of broadband Internet access service is during peak usage times. As such, the Commission determines that all QoS measurements should be based on performance at peak times, i.e. from 7 p.m. to 11 p.m. local time on weekdays.
  8. The Commission notes that CIRA et al. recommended the continuation of the Commission’s Broadband Measurement Project to measure broadband QoS performance. The Project uses a measurement methodology that aligns with Option 1 and measurement during peak times. The methodology is well established and has proven to be suitable for accurate, efficient, fair, and continual broadband QoS measurement. This is evident from its use by regulators in countries that measure broadband QoS performance, including those in the United States and the United Kingdom.
  9. In light of all the above, the Commission determines that the measurement methodology used in the Commission’s Broadband Measurement Project is the appropriate measurement methodology to determine ISPs’ broadband QoS performance. Specifically, broadband QoS is to be measured using a sample-based approach, during peak times (i.e. from 7 p.m. to 11 p.m. local time on weekdays), and using a measurement probe at the modem in the customer premises to an off-net measurement server connected to an IXP in a Canadian Tier 1 city.
  10. In addition, in its CRTC Three-Year Plan 2017-2020, the Commission indicated that it would, in every year up to 2020, continue to collect broadband performance data and publish it as part of its Broadband Measurement Project. As such, the continued use of the Project is the most efficient and least burdensome option for ISPs to collect and report broadband performance data, since it is already being used for participating ISPs (which represent over 80% of Internet subscribers). The Commission’s goal is to increase ISPs’ participation in the Project.
  11. Consequently, the Commission determines that its Broadband Measurement Project is an appropriate means to collect and report on ISPs’ broadband QoS measurements for latency, packet loss, and jitter to measure the successful achievement of the broadband portion of the universal service objective.
  12. The Commission considers that, in line with subparagraph 1(a)(ii) of the Policy Direction, use of the Broadband Measurement Project for continued broadband QoS measurement is efficient and proportionate to its purpose and interferes with the operation of competitive market forces to the minimum extent necessary.

What are appropriate metrics to define high-quality fixed broadband Internet access service?

Background

  1. In Telecom Regulatory Policy 2016-496, the Commission noted that real-time applications – particularly those with audiovisual functionalities – are sensitive to any degradation of the connection and require low levels of latency, jitter, and packet loss to provide a smooth experience to the Canadians who use them. High latency could result in an unsatisfactory user experience for real-time communications services, such as telephone calls or video conferencing. Similarly, high packet loss or jitter causes visible effects, such as video pixilation, sound distortion, or delays in loading Web pages.

Definition of high-quality fixed broadband Internet access service

Positions of parties
  1. The NTWG noted that “high-quality” is a subjective term. It defined broadband QoS as “the collective effect of service performance which determines the degree of satisfaction of a user of the service.” It defined quality of experience as “the overall acceptability of an application or service, as perceived subjectively by the end-user.” 
  2. The NTWG stated that the best approach to developing QoS metrics objectively was to determine a “basket” of online applications that Canadians commonly use, categorize these applications based on their sensitivity to broadband QoS, and attempt to determine the broadband QoS metrics that would generally lead to a good quality of experience for end-users.
  3. Based on this approach, the NTWG set out the categories of applications according to their sensitivity to QoS metrics (i.e. QoS critical, sensitive, or tolerant). Examples of QoS-critical applications are multi-player interactive games and cloud-based applications; examples of QoS-sensitive applications are conversational voice applications, conversational video applications, and Web browsers; and examples of QoS-tolerant applications are file transfers, downloads, high-quality audio streaming, and one-way video streaming.
  4. The NTWG reviewed numerous standards, reports, and studies that referred to broadband QoS requirements for various applications. These materials did not identify a common threshold for a “high-quality” broadband Internet connection for specific applications. However, they did indicate a threshold below which the QoS was unacceptable.
  5. While the NTWG agreed that low latency, jitter, and packet loss are desirable to provide a high-quality broadband QoS, they were unable to reach consensus on appropriate thresholds.
Commission’s analysis and determinations
  1. The Commission considers that broadband QoS thresholds should reflect high-quality fixed broadband Internet access service, similar to the 50 Mbps download and 10 Mbps upload speed thresholds it established in Telecom Regulatory Policy 2016-496. In that decision, the Commission recognized that meeting the universal service objective will take time and significant investment to achieve. The above-mentioned speeds do not reflect the minimum speeds that are achievable today in all of Canada.
  2. By requesting industry stakeholders to develop QoS metrics, the Commission aimed to establish thresholds that would represent a high benchmark, similar to the 50/10 Mbps speeds, such that the thresholds for latency, jitter, and packet loss, in conjunction with the 50/10 Mbps speeds, would define high-quality fixed broadband Internet access service.
  3. It would be contrary to the Commission’s determinations in Telecom Regulatory Policy 2016-496 for broadband QoS metrics to be based on the minimum acceptable or adequate QoS required to support the various online applications used by Canadians.
  4. For a fixed broadband Internet access service to be considered high quality, it must provide the subscriber with a smooth experience without any degradation of the connection when using a wide variety of real-time applications with audiovisual functionalities, which are commonly used today and will continue to be used in the future.
  5. The NTWG reached consensus on the categorization of applications as QoS critical, QoS sensitive, or QoS tolerant. It further agreed on the representative types of applications that would fall within each category. The Commission accepts both this categorization and the examples of representative applications identified in the respective categories.
  6. The Commission considers that high-quality fixed broadband Internet access service should be able to support QoS-critical applications. These applications are important in today’s digital economy, in which most online and even some offline services are being offered using an online cloud-based model. In addition, fixed broadband Internet access service that supports QoS-critical applications can support important services, such as e-health, remote surgery, online education, teleconferencing, and teleworking through virtual private network access.
  7. The QoS metrics required to support QoS-critical applications should therefore serve as the minimum thresholds to define high-quality fixed broadband Internet access service. Accordingly, the Commission determines that fixed broadband Internet access service is defined as a high-quality service if it provides the subscriber with a smooth experience when using real-time QoS-critical applications.

Latency

Positions of parties
  1. Various parties proposed different latency metrics to define high-quality fixed broadband Internet access service.
  2. CIRA et al. submitted that poor latency can degrade the quality of experience of delay-sensitive applications, even with a relatively high bandwidth connection. CIRA et al. recommended 50 ms as the latency threshold that would provide a high quality of experience for QoS-critical applications, based on measurement by SamKnows, as well as a report commissioned by the United Kingdom’s communications regulator, Ofcom (the Ofcom Report).Footnote 15
  3. Bell Canada provided results from broadband QoS measurements in Southern Ontario on its fixed wireless network showing average latencies below 50 ms and none greater than 100 ms. Bell Canada also cited research showing that a good-quality multi-player game experience requires a latency of less than 70 ms, while 200 ms provides an adequate or acceptable experience. Bell Canada stated that remote surgery can be safely conducted with latency levels of less than 200 ms.
  4. Bell Canada et al. proposed that instead of establishing QoS metrics for “high-quality” broadband Internet access service, the Commission should establish such metrics for the broadband Internet access service that is presently attainable in all or most parts of Canada. Bell Canada et al. also noted that a latency of up to 750 ms could form part of the definition of a high-quality fixed broadband Internet access service, since it provides adequate-quality voice service.
  5. RCCI proposed a latency threshold of 150 ms, stating that this threshold takes into account the broad geographic and technological challenges the telecommunications industry faces in delivering consistent fixed broadband Internet access service throughout Canada.
  6. CIRA et al. and Distributel Communications Limited disagreed with Bell Canada et al.’s proposed approach since it would support only an adequate-quality voice service and not QoS-critical applications.
  7. CIRA provided latency measurements and maps showing the latency between certain locations in Canada (e.g. from Fort Smith, Northwest Territories, to Calgary, Alberta, [53 ms], to Toronto, Ontario [93 ms], and to Montréal, Quebec [102 ms]), which confirmed that CIRA’s proposed latency metrics can be met. Clearcable Networks provided actual latencies measured between various locations in CanadaFootnote 16 to Toronto, Ontario, and concluded that all of these locations have a latency of less than 100 ms to Toronto. Bell Canada also provided data on the latency between Inuvik, Northwest Territories, and Montréal, Quebec, in the range of 100 to 200 ms.
Commission’s analysis and determinations
  1. In Telecom Regulatory Policy 2016-496, the Commission indicated that it expected the 50/10 Mbps target speeds for fixed broadband Internet access service to be reached in an incremental manner within 10 to 15 years. Similarly, if the QoS metrics to define high-quality fixed broadband Internet access service were based on the present attainability of those metrics in all or most parts of Canada, the result would be that the lowest QoS attainable would define high-quality services. Therefore, these QoS metrics should be based on the quality of experience that subscribers receive or expect when using high-quality fixed broadband Internet access service.
  2. The proposed 200 ms to 750 ms latency thresholds are based on evidence that these thresholds are for only an acceptable- or adequate-quality fixed broadband Internet access service. RCCI’s proposed latency threshold of 150 ms takes into account the broad geographic and technological challenges the telecommunications industry faces in delivering consistent fixed broadband Internet access service throughout Canada. As a result, this threshold also represents only a medium- or adequate-quality service. Accordingly, setting the latency threshold at these levels would be contrary to the Commission’s objective of defining high-quality fixed broadband Internet access service.
  3. The Commission considers that CIRA et al.’s recommended threshold of 50 ms most closely aligns with the Commission’s intentions based on evidence showing that this threshold is reasonable and achievable, and that it can support QoS-critical applications. As well, the broadband measurement reports indicate that the highest average peak period latency measured from subscribers of the major ISPs in Canada was below 22 ms for digital subscriber line (DSL), cable, and fibre-to-the-home (FTTH) technologies.
  4. In light of all the above, the Commission establishes a round-trip latency threshold of 50 ms to define high-quality fixed broadband Internet access service and to measure the successful achievement of the broadband portion of the universal service objective. As mentioned above, this threshold is based on measurement during peak times (i.e. from 7 p.m. to 11 p.m. local time on weekdays), and from the modem in the customer premises to an IXP in a Canadian Tier 1 city.

Packet loss

Positions of parties
  1. CIRA et al. noted that latency and packet loss are as important metrics to define high-quality fixed broadband Internet access service as speed. In addition, CIRA submitted evidence that for some QoS-critical applications, the subscriber’s quality of experience is affected more by packet loss than by latency. CIRA et al. submitted that the Ofcom Report is the most useful resource for establishing a packet loss threshold, and recommended a threshold of 0.25% for both QoS-sensitive and QoS-critical applications, which corresponds with the latency threshold they recommended.
  2. The ITPA recommended a packet loss threshold of less than 1%, but did not provide rationale. Shaw recommended a packet loss threshold of less than 0.5% based on its internal best practice for voice over Internet Protocol (VoIP) services.
  3. Bell Canada et al. and RCCI did not propose a specific packet loss threshold, and instead recommended that such a threshold not be established at this time. These companies indicated that only the latency threshold should define high-quality fixed broadband Internet access service.
Commission’s analysis and determinations
  1. The Commission considers packet loss to be an important metric for defining high-quality fixed broadband Internet access service, since high packet loss prevents many applications from performing at a satisfactory level. The factors that can influence packet loss, such as network design and choice of technology, are under an ISP’s direct control.
  2. The broadband measurement reports indicate that in 2016, FTTH services yielded the lowest levels of packet loss, averaging 0.04%, while cable services averaged 0.13%, and DSL services averaged 0.17%. These packet loss levels were noted as being extremely small and imperceptible to any common Internet application.
  3. CIRA et al.’s recommended packet loss threshold of 0.25% supports QoS-critical and QoS-sensitive applications. Furthermore, the evidence on the record indicates that ISPs’ networks should perform at a much lower packet loss level than 0.25%, and that a threshold of 0.5% or 1% for packet loss would fall within the range of a medium-quality broadband Internet access service.
  4. In light of all the above, the Commission establishes a packet loss threshold of 0.25% to define high-quality fixed broadband Internet access service and measure the successful achievement of the broadband portion of the universal service objective. As mentioned above, this threshold is based on measurement during peak times (i.e. from 7 p.m. to 11 p.m. local time on weekdays), and from the modem in the customer premises to an IXP in a Canadian Tier 1 city.

Jitter

Positions of parties
  1. The ITPA was the only party that proposed a threshold for jitter (i.e. less than 5 ms), and did not provide any supporting evidence for this proposal.
  2. CIRA et al. did not provide a threshold for jitter since they submitted that jitter could be compensated through buffering and better latency. Bell Canada et al., CNOC, and RCCI recommended that a specific threshold for jitter not be established at this time.
  3. Valve, a major multi-player online interactive game provider, indicated that excessive jitter results in packets being out of order, which negatively impacts end-users’ experience when using multi-player online interactive games.
Commission’s analysis and determinations
  1. The Commission considers that, consistent with its determination in Telecom Regulatory Policy 2016-496, it is necessary and important to establish a QoS threshold for jitter, in addition to the latency and packet loss thresholds.
  2. Even with low latency, high jitter can lead to a poor experience for subscribers with real-time applications, such as videos, audio calls, e-health, and multi-player interactive online games. While many of the effects of jitter can be managed by applications that buffer the data packets, buffering may itself negatively affect the subscriber’s experience. Therefore, the use of buffering does not eliminate the need to establish a threshold for jitter, since low jitter reduces or eliminates the need for buffering.
  3. There is insufficient data on the record for the Commission to make a determination on what threshold for jitter is appropriate. As well, the broadband measurement reports did not include any statistics regarding jitter that demonstrate Canadian ISPs’ performance in this respect.
  4. Accordingly, the Commission is launching a separate proceeding to establish an appropriate QoS threshold for jitter to define high-quality fixed broadband Internet access service, through Telecom Notice of Consultation 2018-242, also being issued today. As well, to ensure consistency with the established latency and packet loss QoS thresholds, the jitter threshold to define high-quality fixed broadband Internet access service must be based on the ability to support QoS-critical applications, and jitter performance during peak times (i.e. from 7 p.m. to 11 p.m. local time on weekdays), and from the modem in the customer premises to an IXP in a Canadian Tier 1 city.

Secretary General

Related documents

Date modified: