SamKnows Analysis of Broadband Performance in Canada - March & April 2016
Download this report in PDF.
Dated: September 1st, 2016
Modified: December 1st 2016
View most recent report, June 2020
Revisions to the Broadband Measurement Report – December 2016
- Executive Summary
Paragraph of the Executive summary starting with “ISPs also largely met or exceeded…” has been modified to address the discrepancy between the paragraph and the revision to the figures presented in the “Upload Throughput” section. - Key Performance Indicator (Download Throughput)
The paragraph starting with “Figure 6 to 8 depict…” has been modified to reflect the revision to Figure 6. - Key Performance Indicator (Upload Throughput)
The paragraph starting with “Figure 10 depicts…” and the following paragraph have been modified to reflect the revision to Figure 10. - Key Performance Indicator (Upload Throughput)
The paragraph starting with “The majority of DSL did not…” and the following paragraph have been modified to reflect the revision to Figure 11. - Key Performance Indicator (Upload Throughput)
The paragraph starting with “Figure 13 through 16 depict…” has been modified to reflect the revision to Figure 13. - Conclusion
Paragraph of the Conclusion starting with “Upload speeds generally also met or exceeded…” has been modified to address the discrepancy between the paragraph and the revision to the figures presented in the “Upload Throughput” section. - Appendix
The Sample Plan has been revised to address error in the upload rate of some plans. - Figure 1
Figure 1 has been revised since the publication of the Report to reflect the revision to Bell Canada upload speeds. - Figure 2
Figure 2 has been revised since the publication of the Report to reflect the revision to Bell Canada upload speeds. - Figure 6
Figure 6 has been revised since the publication of the Report to reflect the revision to Bell Canada upload speeds. - Figure 7
Figure 7 has been revised since the publication of the Report to reflect the revision to Bell Canada upload speeds. - Figure 8
Figure 8 has been revised since the publication of the Report to reflect the revision to Bell Canada upload speeds. - Figure 9
Figure 9 has been revised since the publication of the Report to reflect the revision to Bell Canada upload speeds. - Figure 10
Figure 10 has been revised since the publication of the Report to reflect the revision to Bell Canada upload speeds. - Figure 11
Figure 11 has been revised since the publication of the Report to reflect the revision to Bell Canada upload speeds - Figure 12
Figure 12 has been revised since the publication of the Report to reflect the revision to Bell Canada upload speeds - Figure 13
Figure 13 has been revised since the publication of the Report to reflect the revision to Bell Canada upload speeds. - Figure 14
Figure 14 has been revised since the publication of the Report to reflect the revision to Bell Canada upload speeds - Figure 15
Figure 15 has been revised since the publication of the Report to reflect the revision to Bell Canada upload speeds - Figure 16
Figure 16 has been revised since the publication of the Report to reflect the revision to Bell Canada upload speeds - Table 1
Table 1 has been revised since the publication of the Report to reflect the revision to Bell Canada upload speeds -
Table 2
Table 2 has been revised since the publication of the Report to reflect the revision to Bell Canada upload speeds
Contents
About The Project
The Canadian Radio-Television and Telecommunications Commission (CRTC) has commissioned SamKnows to conduct a study of the performance of broadband services sold to Canadian consumers. SamKnows is a global leader in broadband measurement and has been working with governments, Internet Service Providers (ISPs), content service providers, application developers, consumer groups, and academics to accurately measure Internet performance since 2009. In doing so, SamKnows has built a global Internet measurement platform, which now spans five continents and conducts many millions of measurements each day.
Data presented in this report was collected between 15th March 2016 and 14th April 2016.
4,808 Whiteboxes were deployed to Canadian volunteers as a part of this study. Data from 3,056 of these Whiteboxes was used in this report.Footnote 1 For purposes of reporting, graphical representations of speed tier groups are split into four “buckets”: 5-9Mbps, 10-15Mbps, 16-39Mbps, and 40Mbps+. The ISPs participating in this project included all the major wireline service providers in Canada other than Sasktel. Specifically Bell Canada, Bell Aliant, Cogeco, Eastlink, MTS, Northwestel, Rogers, Shaw, TELUS and Videotron all participated on a voluntary basis and measurements covered all geographic regions of Canada in a mix of urban and rural settings. These ISPs use the following technologies to provide Internet access service: digital subscriber line (DSL)Footnote 2, hybrid-fibre co-axial cable (Cable / HFC)Footnote 3 and fibre to the home (FTTH)Footnote 4. Testing has not included any ISPs using satellite or fixed wireless technologies, nor did it include resellers of these ISPs’ networks.
The test methodology employed is the same as the one SamKnows uses around the globe with other regulators and ISPs. A full description of the test methodology can be found at https://goo.gl/piIKqA.
SamKnows recommends that a minimum sample of 40 measurement probes should be reporting data per stratum in order to provide sufficient statistical accuracy in the results. This is the approach taken in this report. Additional information on sample size methodology can be found at https://goo.gl/xvwa9T.
Any comments on the analysis in this document should be directed to Roxanne Robinson (roxanne@samknows.com).
Executive Summary
This report presents the findings of the measurement study that SamKnows is conducting in Canada on behalf of the CRTC. This report looks at data collected between the 15th March 2016 and the 14th April 2016 and investigates performance on an ISP and plan level. A preliminary report that focused on performance by region and access technology was released in March 2016 (based upon results collected in October and November 2015). The overall results in this report remain consistent with those in the Preliminary report.
4,808 SamKnows Whiteboxes have been deployed in Canadian homes across a range of ISPs and products. Each Whitebox conducts end-to-end performance measurements 24 hours a day, 7 days a week to destinations representative of Canadian consumers’ Internet usage.
In this report, results are reported by technology, ISP and speed bucket. Technologies have been identified as DSL, Cable/HFC (including all DOCSIS variants) and FTTH.
During data collection for the previous reportFootnote 5, which showed preliminary findings of broadband performance in Canada, we experienced issues with a third party that carried some of the measurement traffic, affecting the reliability of some measurement results. This was taken into account when writing the report and only unaffected data was used to create the resulting charts and tables. These issues have since been resolved and did not resurface at any point in this study's measurement period.
Unless otherwise stated, all results presented in this report are taken from the peak period, which is defined as 7-11pm local time on weekdays. Moreover, all reported figures are subject to a minimum sample size of 40 Whiteboxes. These provisions are consistent with those used in SamKnows’ other projects around the globe. The key findings contained in this report are as follows:
- Download speed across all ISPs was consistent between peak and off-peak hours, with the majority of ISPs delivering speeds above their advertised rates, regardless of the access technology in use. Bell Aliant’s 7x0.64Mbps DSL service, however, underperformed, delivering speeds at 77% of advertised. The vast majority of individual plans also exceeded their advertised speeds, with most others performing just below them. TELUS’ 6x1Mbps DSL plan and Bell Aliant’s aforementioned 7x0.64Mbps DSL plan were the only ones to reach less than 90% of their advertised rates, achieving averages of 85% and 77% respectively.
- All access technologies met or exceeded the advertised download speed on average, demonstrating that the access technologies themselves are capable of supporting the advertised services. FTTH services delivered 121% of advertised download speed during peak hours on average, with Cable/HFC services delivering an average of 105% and DSL services 103%. However, as noted in the previous paragraph, there were isolated exceptions for DSL services.
- ISPs also largely met or exceeded their advertised upload speeds. Bell Aliant and Bell Canada DSL underperformed, achieving 81% and 85% of their advertised upload rates. Most individual plans slightly exceeded their advertised speeds, including two plans each from TELUS and Bell Canada. TELUS’ 6x1Mbps DSL plan and Bell Aliant’s 7x0.64Mbps DSL plan delivered 81% of their advertised upload speed. Bell Canada’s 5x1Mbps, 15x10Mbps and 25x10Mbps plans also underperformed, with values between 60% and 86% of advertised speed. On average, all access technologies exceeded 100% of advertised upload rates, although as noted earlier there were individual DSL plans that did not meet this.
- Latency was comparable across all access technologies and the observed variances would be imperceptible to common Internet applications. DSL services yielded the highest latency, with results ranging between 17.80ms and 51.50ms depending on the ISP and product. Higher latencies with DSL products are an expected by-product of the access technology. The fact that DSL is also more widespread in rural areas contributed to its latency performance.
- Between the peak and off-peak measurement periods, DSL services yielded slightly more stable latency measurements than Cable/HFC products, with an average of 21.23ms in the off-peak period, increasing by 0.54ms to 21.77ms in the peak period. Cable/HFC displayed lower but less stable latencies, increasing by 0.72ms from 19.62ms to 20.34ms. FTTH services were generally more consistent and displayed lower latencies than all other access technologies on average as well as across various speed tiers, achieving an average of 9.16ms in the off-peak period, rising to 9.23ms during peak hours.
- Packet loss, which describes how likely it is that a data packet sent from point A will not reach point B, was generally very low, although there were exceptions. FTTH services yielded the lowest levels of packet loss, averaging 0.04%. Cable services average 0.13% and DSL services 0.17%. These levels of packet loss are extremely small and would be imperceptible to any common Internet application.
- Web page loading times to a selection of websites popular in Canada improved as download speeds increased. However, this improvement is not linear. Some services below 10Mbps took up to 2.3 seconds to load web pages on average. The fastest service in the sample plan, Rogers’ 250x20Mbps plan, loaded pages in just 0.8 seconds. However, this is virtually no different from the performance of all 100Mbps products, provided by Rogers, Eastlink and Bell Aliant, with some 50Mbps plans even displaying lower loading time, including Bell Canada’s 50x50Mbps plan with a loading time of 0.5 seconds. As has been found in studies in other marketsFootnote 6, improvements in page loading time tail off after 10Mbps (at which point latency becomes the dominant factor). Latency between the end-user and destination (which is related to physical distance) will impact web browsing performance, resulting in higher load times in rural areas.
Figure 1 below shows a comparison of download and upload speed by technology, expressed as a percentage of advertised speed. As stated previously, all technologies exceeded their advertised rates on average.
Figure 1: Download Speed vs Upload Speed as a percentage of Advertised Speed by technology
This bar chart illustrates a comparison of download and upload speed by technology, expressed as a percentage of advertised speed. The percentages for each speed range by technology are as follows:Long Description
Technology
Download Speed Throughput
Upload Speed Throughput
DSL
103%
111%
Cable/HFC
105%
104%
FTTH
121%
120%
Figure 2 below shows a comparison of download and upload speed by ISP as a percentage of advertised speed. As stated previously, the majority of ISPs met or exceeded their advertised download and upload speeds. Bell Aliant’s 7x0.64Mbps and TELUS’ 6x1Mbps plans in the 5-9Mbps bucket were shown to perform below advertised speeds for both download and upload.
TELUS’ 15x1Mbps plans significantly exceeded its advertised upload speed. Bell Canada, Bell Aliant and and TELUS also overprovisioned some of their FTTH services.
Figure 2: Download Speed vs Upload Speed as a percentage of Advertised Speed by ISP and technology
Long Description
This bar chart illustrates a comparison of download and upload speed by ISP as a percentage of advertised speed as follows:
| Technology-ISP | Download Speed Throughput | Upload Speed Throughput |
|---|---|---|
| DSL-Bell Canada | 112% | 93% |
| DSL-Bell Aliant | 77% | 81% |
| DSL-MTS | 103% | 95% |
| DSL-TELUS | 99% | 151% |
| Cogeco-Cable/HFC | 101% | 101% |
| Cable/HFC-Eastlink | 101% | 105% |
| Cable/HFC-Northwestel | 98% | 94% |
| Cable/HFC-Rogers | 108% | 108% |
| Cable/HFC-Shaw | 109% | 104% |
| Cable/HFC-Videotron | 106% | 103% |
| FTTH-Bell Canada | 124% | 113% |
| FTTH-Bell Aliant | 125% | 111% |
| FTTH-TELUS | 105% | 174% |
Methodology
Whiteboxes
SamKnows deployed 4,808 Whiteboxes to Canadian consumers for the purposes of this project. The Whitebox is a consumer-grade device that is installed in a user’s home network between their home modem/router and their devices. The Whitebox’s core function is to measure the quality of the user’s Internet connection.
The measurements are conducted autonomously by the Whitebox to a variety of destinations on the Internet. No user interaction is required to conduct measurements; they are executed automatically according to a test schedule. The Whitebox does not base any of its measurement results on end user activity.
End-user cross-trafficFootnote 7 is monitored continuously by the Whitebox. If cross-traffic exceeds a certain threshold then measurements are not conducted until the cross-traffic subsides. This ensures that the Whitebox’s measurements are not distorted by end-user activity, and that the Whitebox’s measurement traffic does not interfere with a user’s experience of the Internet.
A full description of the Whitebox and its features can be found at https://goo.gl/JVMnBn.
Measurements
The Whiteboxes run a suite of active performance measurements according to a pre-defined test schedule. These include the following network measurements: download speed, upload speed, latency and packet loss, as well as application measurements such as web browsing performance.
A full description of the methodology underpinning each test can be found at https://goo.gl/fruqy6.
An overview of the test schedule used for the Measuring Broadband Canada project can be found in Appendix A.
Test destinations
The measurements carried out by the Whiteboxes are conducted against two different types of destination servers.
Firstly, there are dedicated measurement servers. These are installed at major peering and Internet exchanges at the following locations:
- Montreal
- Halifax
- Winnipeg
- Vancouver
- Toronto
Each server met minimum specifications set out by SamKnows, with the exception of the one in Winnipeg. The server provisioned in Winnipeg has a 100Mbps connection rather than the standard 1Gbps as this was the highest connection available for a dedicated server in Winnipeg. This did not affect the results as only Whiteboxes based in Winnipeg were able to test to this server and these were configured in a manner that ensured the 100Mbps link would never be saturated. Full details of the hardware, software, network connectivity and monitoring requirements are available at: https:// goo.gl/RUu8QK.
The dedicated measurement servers are utilized for the download speed, upload speed, latency and packet loss measurements.
Secondly, there are real applications / content providers that measurements are carried out against. For the purposes of the Measuring Broadband Canada project, this is limited to the web browsing measurements only. The websites tested against were as follows:
- facebook.com/policies
- ca.yahoo.com
- live.ca
- cbc.ca
- google.ca
- ebay.com
- theweathernetwork.com
- ici.radio-canada.ca
- meteomedia.com
Sample plan
SamKnows constructed a sample plan to govern the distribution of Whiteboxes amongst Canadian Internet consumers. This sample plan was built using subscriber data provided by the ISPs. This considered which plans and technologies met the minimum sample size requirement of 5% of the ISP’s overall wireline subscriber base. It also considered which provinces the product was available in, and what market share each province represented for the product.
Full details of the sample plan are available in Appendix A.
Recruitment and validation
The ISPs assisted in the recruitment of panelists for the project by sending emails soliciting volunteers to their customer base. These emails directed users to sign up at a SamKnows-controlled signup website. Users were then given the opportunity to sign up to the project to receive a free Whitebox, which would also provide them access to their own measurement results.
Once a user completed the signup process, their name, address and telephone number were shared with their ISP (with the user’s consent) in order for the ISP to confirm the plan that they were subscribed to. The validated ISP, product and geographic location was then checked against the sample plan to ensure that the volunteer was eligible.
Assuming so, the volunteer was sent a Whitebox and data collection began. The validation process continued to run at weekly intervals with the ISPs, thus ensuring that if the user changed ISP, product or geographic location then SamKnows would be aware and it could be accounted for in the report.
All collaborative ISPs involved in the program and the CRTC signed a ‘Code of Conduct’ which protects against ISPs intentionally changing or affecting test results as well as governing what ISPs may do with the data they have access to. While the identity of each panelist was made known to the applicable ISP as part of the speed tier validation process, the actual Unit ID for the associated Whitebox was not released to the ISP and specific test results were not directly assignable against a specific panelist.
The Code of Conduct is included in Appendix A.
A note on reported regions
This report contains multiple charts showcasing performance verticals by region. For the purposes of this report, regions are comprised of the following provinces:
- West & North: British Columbia, Alberta, Saskatchewan, Yukon, Northwest Territories, and Manitoba.
- Central: Ontario and Quebec.
- East: New Brunswick, Nova Scotia, Prince Edward Island, and Newfoundland and Labrador.
Key Performance Indicators
Download Throughput
Download throughput is the measure of the capacity of the user’s broadband connection. Higher speeds are more desirable, as it can allow the user to retrieve data (be it a web page, music file, or anything else) more quickly.
To characterize the user’s maximum access link capacity, measurements were conducted between panelists’ homes and the nearest test server. Test servers were deployed in multiple metropolitan areas throughout Canada.
It is common for broadband providers around the world to differentiate their product offerings by an advertised access speed, and this is a key part of their advertising. Speeds are typically expressed in megabits per second (commonly abbreviated to ‘Mbps’ or ‘Mbit/s’). To enable comparability between different products and technologies which may feature vastly different speeds, most regulators conducting measurement studies around the world report on the percentage of advertised speed that products and technologies achieve. This report follows the same convention. For purposes of reporting, the data was split into four speed tier “buckets”: 5-9Mbps, 10-15Mbps, 16-39Mbps, and 40Mbps+.
Figure 3 below shows download speed as a percentage of advertised speed, broken down by region and access technology. Download speed in almost all regions met or exceeded the advertised rates, regardless of access technology. The one exception was DSL in the East, which achieved 75% on average.
Figure 3: Peak vs Off-Peak Download Speed as a percentage of Advertised Speed by technology and region
Long Description
This bar chart illustrates download speed as a percentage of advertised speed, broken down by region and access technology as follow:.
| Technology-Region | Download Speed Throughput –Off-Peak | Download Speed Throughput-Peak |
|---|---|---|
| DSL-Central | 111% | 108% |
| DSL-East | 79% | 75% |
| DSL-West & North | 105% | 101% |
| Cable/HFC-Central | 108% | 105% |
| Cable/HFC-East | 102% | 101% |
| Cable/HFC-West & North | 107% | 105% |
| FTTH-Central | 124% | 123% |
| FTTH-East | 123% | 123% |
| FTTH-West & North | 117% | 116% |
Figure 4 shows download speeds as a percentage of advertised speeds, broken down by ISP and access technology during the peak and off-peak time periods. The vast majority of ISPs met or exceeded advertised rates. Only Northwestel, TELUS (DSL only) and Bell Aliant (DSL only) products fell below 100% of their advertised rates during peak hours. FTTH services delivered an average of 121% of advertised speed, and all ISPs providing FTTH services delivered in excess of the advertised speed. Cable/HFC services delivered an average of 105%, and DSL services achieved an average of 103%.
DSL services showed more variance between ISPs. Bell Aliant fell short of the advertised rate, achieving 77% of advertised speed. This increased variability for DSL is not surprising given the effect copper loop lengths (i.e. how far a customer is from the nearest central office or node) have on line performance.
Figure 4: Peak vs Off-Peak Download Speed as a percentage of Advertised Speed by technology and ISP
Long Description
This bar chart illustrates download speeds as a percentage of advertised speeds, broken down by ISP and access technology during the peak and off-peak time periods as follows:
| Technology-ISP | Download Speed Throughput –Off-Peak | Download Speed Throughput-Peak |
|---|---|---|
| DSL-Bell Canada | 114% | 112% |
| DSL-Bell Aliant | 79% | 77% |
| DSL-MTS | 109% | 103% |
| DSL-TELUS | 102% | 99% |
| Cable/HFC-Cogeco | 105% | 101% |
| Cable/HFC-Eastlink | 104% | 101% |
| Cable/HFC-Northwestel | 100% | 98% |
| Cable/HFC-Rogers | 110% | 108% |
| Cable/HFC-Shaw | 111% | 109% |
| Cable/HFC-Videotron | 107% | 106% |
| FTTH-Bell Canada | 125% | 124% |
| FTTH-Bell Aliant | 125% | 125% |
| FTTH-TELUS | 107% | 105% |
Figure 5 depicts the average download speed by time of day as a percentage of advertised speed achieved by ISPs during peak and off-peak hours. Speed is typically lowest between 7pm and 11pm. MTS experienced the largest variance in speeds with a 6% change between peak and off peak hours. All ISPs delivered on average between 97% and 120% of advertised speeds, although some individual plans fell short of this.
Figure 5: Download Speed as a percentage of Advertised Speed by ISP and time of day
Long Description
This line chart illustrates the average download speed as a percentage of advertised achieved by ISPs as follows:
| Time of Day | Bell Canda | Shaw | Bell Aliant | Rogers | Videotron | MTS | Cogeco | Eastlink | TELUS | Northwestel |
|---|---|---|---|---|---|---|---|---|---|---|
| 00:00 – 02:00 | 118% | 111% | 109% | 110% | 107% | 108% | 105% | 103% | 103% | 100% |
| 02:00 – 04:00 | 120% | 112% | 110% | 110% | 107% | 109% | 105% | 105% | 103% | 101% |
| 04:00 – 06:00 | 119% | 112% | 110% | 110% | 107% | 110% | 106% | 105% | 103% | 99% |
| 06:00 – 08:00 | 119% | 112% | 109% | 110% | 107% | 109% | 105% | 105% | 103% | 101% |
| 08:00 – 10:00 | 119% | 111% | 109% | 110% | 107% | 108% | 105% | 104% | 102% | 100% |
| 10:00 – 12:00 | 119% | 111% | 109% | 109% | 107% | 108% | 105% | 104% | 102% | 98% |
| 12:00 – 14:00 | 119% | 111% | 109% | 109% | 107% | 108% | 105% | 103% | 102% | 100% |
| 14:00 – 16:00 | 118% | 111% | 109% | 109% | 107% | 108% | 104% | 103% | 102% | 100% |
| 16:00 – 18:00 | 118% | 110% | 109% | 109% | 106% | 107% | 104% | 102% | 102% | 98% |
| 18:00 – 20:00 | 118% | 110% | 109% | 109% | 106% | 105% | 103% | 102% | 101% | 99% |
| 20:00 – 22:00 | 117% | 108% | 108% | 107% | 106% | 103% | 100% | 101% | 100% | 97% |
| 22:00 – 00:00 | 118% | 109% | 108% | 108% | 106% | 105% | 102% | 102% | 101% | 98% |
Figures 6 through 8 depict download speed as a percentage of advertised speed for ISPs’ individual products. Two products were below 90% of the advertised rate. Bell Aliant achieved an average of 77% of advertised speed on its 7x0.64Mbps DSL plan and TELUS’ 6x1Mbps DSL plan delivered 85% of the advertised speed. Six products exceeded 120% of the advertised rate: Bell Canada’s 5x1Mbps, 15x15Mbps, 25x25Mbps and 50x50Mbps products, as well as Bell Aliant’s 100x50Mbps and 150x50Mbps products.
Some of the MTS and TELUS service tiers are provisioned using either DSL or FTTH technologies, depending on where the customer resides. For purposes of reporting on service tier performance in the following Figures, these results have been grouped together.
Figure 6: Download Speed as a percentage of Advertised Speed by Product, 5-9Mbps speed bucket
Long Description
This bar chart illustrates download speed as a percentage of advertised speed for ISPs’ individual products in the 5-9Mbps speed bucket as follows:.
| ISP Product | Download Speed Throughput |
|---|---|
| MTS 5X0.512 | 100% |
| Bell Canada 5X1 | 135% |
| Videotron 5X1 | 102% |
| TELUS 6X1 | 85% |
| Cogeco 6X2 | 103% |
| Bell Aliant 7X0.64 | 77% |
| Shaw 7.5X0.5 | 105% |
Figure 7: Download Speed as a percentage of Advertised Speed by Product, 10-15Mbps speed bucket
Long Description
This bar chart illustrates download speed as a percentage of advertised speed for ISPs’ individual products in the 10-15 Mbps speed bucket as follows:
| ISP Product | Download Speed Throughput |
|---|---|
| Shaw 10X0.5 | 102% |
| Videotron 10X1.5 | 102% |
| MTS 10X2 | 111% |
| TELUS 15X1 | 104% |
| Cogeco 15X2 | 101% |
| Bell Canada 15X10 | 104% |
| Bell Canada 15X15 | 124% |
Figure 8: Download Speed as a percentage of Advertised Speed by Product, 16-39Mbps speed bucket
Long Description
This bar chart illustrates download speed as a percentage of advertised speed for ISPs’ individual products in the 16-39 Mbps speed bucket as follows:
| ISP Product | Download Speed Throughput |
|---|---|
| Northwestel 16X0.768 | 101% |
| Eastlink 20X2 | 105% |
| MTS 25X2 | 100% |
| Shaw 25X2.5 | 110% |
| TELUS 25X5 | 104% |
| Bell Canada 25X10 | 106% |
| Bell Canada 25X25 | 125% |
| Rogers 30X5 | 107% |
| Shaw 30X5 | 110% |
| Videotron 30X10 | 109% |
Figure 9: Download Speed as a percentage of Advertised Speed by Product, 40Mbps+ speed bucket
Long Description
This bar chart illustrates download speed as a percentage of advertised speed for ISPs’ individual products in the 40Mbps+ speed bucket as follows:
| ISP Product | Download Speed Throughput |
|---|---|
| Cogeco 40X10 | 101% |
| Northwestel 50X2 | 94% |
| Eastlink 50X5 | 99% |
| Shaw 50X5 | 114% |
| Bell Canada 50X10 | 107% |
| TELUS 50X10 | 105% |
| Bell Canada 50X50 | 124% |
| Cogeco 60X10 | 100% |
| Rogers 60X10 | 107% |
| Videotron 60X10 | 108% |
| Rogers 70X10 | 100% |
| Eastlink 100X10 | 106% |
| Rogers 100X10 | 115% |
| Bell Aliant 100X50 | 125% |
| Eastlink 150X10 | 95% |
| Bell Aliant 150X50 | 126% |
| Rogers 250X20 | 102% |
Table 1 shows the advertised download speed for each ISP and corresponding products included in this study, expressed both in absolute terms and as a percentage of the advertised rates. Some products are classified as both DSL and FTTH as they are provided in either type of access technology depending on the user’s area. As such, the results shown for said products e.g. TELUS’ 6x1Mbps plan or MTS’s 10x2Mbps product are an average across both technologies.
| ISP | Technology | Package | Actual Download Speed (Mbps) | Actual/Advertised Download Speed (%) |
|---|---|---|---|---|
| MTS | DSL | 5x0.512 | 5.02 | 100% |
| Bell Canada | DSL | 5x1 | 6.77 | 135% |
| Videotron | Cable/HFC | 5x1 | 5.09 | 102% |
| TELUS | DSL, FTTH | 6x1 | 5.10 | 85% |
| Cogeco | Cable/HFC | 6x2 | 6.16 | 103% |
| Bell Aliant | DSL | 7x0.64 | 5.36 | 77% |
| Shaw | Cable/HFC | 7.5x0.5 | 7.88 | 105% |
| Shaw | Cable/HFC | 10x0.5 | 10.22 | 102% |
| Videotron | Cable/HFC | 10x1.5 | 10.18 | 102% |
| MTS | DSL, FTTH | 10x2 | 11.14 | 111% |
| TELUS | DSL, FTTH | 15x1 | 15.66 | 104% |
| Cogeco | Cable/HFC | 15x2 | 15.17 | 101% |
| Bell Canada | DSL | 15x10 | 15.58 | 104% |
| Bell Canada | FTTH | 15x15 | 18.61 | 124% |
| Northwestel | Cable/HFC | 16x0.768 | 16.22 | 101% |
| Eastlink | Cable/HFC | 20x2 | 21.03 | 105% |
| MTS | DSL, FTTH | 25x2 | 24.90 | 100% |
| Shaw | Cable/HFC | 25x2.5 | 27.42 | 110% |
| TELUS | DSL, FTTH | 25x5 | 26.00 | 104% |
| Bell Canada | DSL | 25x10 | 26.60 | 106% |
| Bell Canada | FTTH | 25x25 | 31.18 | 125% |
| Rogers | Cable/HFC | 30x5 | 32.16 | 107% |
| Shaw | Cable/HFC | 30x5 | 33.09 | 110% |
| Videotron | Cable/HFC | 30x10 | 32.73 | 109% |
| Cogeco | Cable/HFC | 40x10 | 40.34 | 101% |
| Northwestel | Cable/HFC | 50x2 | 47.24 | 94% |
| Eastlink | Cable/HFC | 50x5 | 49.59 | 99% |
| Shaw | Cable/HFC | 50x5 | 56.86 | 114% |
| Bell Canada | DSL | 50x10 | 53.26 | 107% |
| TELUS | DSL, FTTH | 50x10 | 52.59 | 105% |
| Bell Canada | FTTH | 50x50 | 62.14 | 124% |
| Cogeco | Cable/HFC | 60x10 | 60.28 | 100% |
| Rogers | Cable/HFC | 60x10 | 64.35 | 107% |
| Videotron | Cable/HFC | 60x10 | 64.79 | 108% |
| Rogers | Cable/HFC | 70x10 | 70.04 | 100% |
| Eastlink | Cable/HFC | 100x10 | 106.19 | 106% |
| Rogers | Cable/HFC | 100x10 | 115.19 | 115% |
| Bell Aliant | FTTH | 100x50 | 124.72 | 125% |
| Eastlink | Cable/HFC | 150x10 | 143.12 | 95% |
| Bell Aliant | FTTH | 150x50 | 189.34 | 126% |
| Rogers | Cable/HFC | 250x20 | 254.18 | 102% |
TELUS upload tiers were tested only in the provinces of Alberta and British Columbia. Upload speed tests were not performed in Quebec for TELUS’ 15x1.5Mbps, 25x3Mbps, and 50x12Mbps products, as the upload tiers did not meet the minimum number of whiteboxes threshold required for inclusion.
Upload Throughput
Upload throughput is the measure of how fast data can be transmitted from the home to the Internet. Higher speeds can allow for pictures, music and documents to be uploaded and shared more quickly.
To characterize the user’s maximum access link capacity, measurements were conducted between a nearby test server and panelists’ homes.
Historically, the amount of data that users download has vastly outweighed the amount of data that users upload. This has led technologies to be engineered to be asymmetric; i.e. they offer faster download rates than upload rates. However, as can be seen by comparing the download speeds to upload speeds, this ratio is falling for new services, such as those delivered using FTTH, which was designed with support for symmetric speeds in mind.
As with the download throughput, results in this section are presented as a percentage of the ISP’s advertised product in addition to results for the absolute level of upload speed. This enables comparability between products of vastly different speeds.
Figure 10 depicts upload speed as a percentage of advertised speed, broken down by region and access technology during the peak and off-peak periods. The majority of technologies and regions exceeded their respective advertised speeds, with the exception of DSL in the Central and East regions, where they achieved a respective 86% and 77% of advertised during peak hours. The East region was mostly influenced by Bell Aliant’s 7x0.64Mbps product, whereas Bell Canada’s 15x10 product had the greatest impact in the Central region.
In contrast, DSL technology in the West & North regions exceeded the advertised rates considerably, driven primarily by the overprovisioning of TELUS’ 15x1Mbps product. FTTH services in the West & North regions also showed an overprovisioned level of upload speed. This was also due to the influence of TELUS’ 15x1Mbps FTTH product.
Figure 10: Peak vs Off-Peak Upload Speed as a percentage of Advertised Speed by technology and region
Long Description
This bar chart illustrates upload speed as a percentage of advertised speed, broken down by region and access technology during the peak and off-peak period as follows:
| Technology-Region | Upload Speed Throughput –Off-Peak | Upload Speed Throughput-Peak |
|---|---|---|
| DSL-Central | 86% | 86% |
| DSL-East | 77% | 77% |
| DSL-West & North | 132% | 131% |
| Cable/HFC-Central | 105% | 105% |
| Cable/HFC-East | 106% | 106% |
| Cable/HFC-West & North | 103% | 102% |
| FTTH-Central | 113% | 113% |
| FTTH-East | 110% | 110% |
| FTTH-West & North | 139% | 138% |
Figure 11 depicts upload speeds as a percentage of advertised speeds, broken down by ISP and access technology. Like with download speed, the majority of ISPs met or exceeded advertised rates. With the exception of Northwestel’s Cable/HFC services, all Cable/HFC and FTTH services exceeded the advertised upload rates. FTTH services delivered an average of 120% of advertised speed during the peak period. DSL services delivered an average of 108%, and Cable/HFC services achieved an average of 103%.
The majority of DSL services did not meet their advertised upload rates. Bell Canada DSL delivered an average of 85% of advertised upload rate - impacted particularly by the 15x10Mbps product, which delivered 60% of advertised speed. Bell Aliant also did not meet its advertised upload rates, delivering 81% of advertised speed. The average performance for the technology was lifted by TELUS, which achieved 151%.
TELUS overprovisioned its FTTH services, contributing to the technology's very high overall average, delivering 174% of advertised speed.
Figure 11: Peak vs Off-Peak Upload Speed as a percentage of Advertised Speed by technology and ISP
Long Description
This bar chart illustrates upload speeds as a percentage of advertised speeds, broken down by ISP and access technology as follows:
| Technology-ISP | Upload Speed Throughput –Off-Peak | Upload Speed Throughput-Peak |
|---|---|---|
| DSL-Bell Canada | 85% | 85% |
| DSL-Bell Aliant | 81% | 81% |
| DSL-MTS | 96% | 95% |
| DSL-TELUS | 152% | 151% |
| Cable/HFC-Cogeco | 102% | 101% |
| Cable/HFC-Eastlink | 105% | 105% |
| Cable/HFC-Northwestel | 94% | 94% |
| Cable/HFC-Rogers | 109% | 108% |
| Cable/HFC-Shaw | 104% | 104% |
| Cable/HFC-Videotron | 104% | 103% |
| FTTH-Bell Canada | 113% | 113% |
| FTTH-Bell Aliant | 111% | 111% |
| FTTH-TELUS | 175% | 174% |
Figure 12 shows the average upload speed for each ISP as a percentage of advertised speed by hour of day, across all access technologies. Upload speeds are very stable across all hours of the day. Most ISPs showed less than 1% variation at any given time of the day, with the greatest amount of variation by any ISP being only 2%.
Figure 12: Upload Speed as a percentage of Advertised Speed by ISP and time of day
Long Description
This line chart illustrates the average upload speed for each ISP as a percentage of advertised by hour of day, across all access technologies as follows:
| Time of Day | TELUS | Bell Canada | Rogers | Eastlink | Shaw | Videotron | Cogeco | Bell Aliant | MTS | Northwestel |
|---|---|---|---|---|---|---|---|---|---|---|
| 00:00 – 02:00 | 156% | 97% | 106% | 105% | 104% | 104% | 102% | 101% | 97% | 94% |
| 02:00 – 04:00 | 156% | 97% | 106% | 105% | 104% | 104% | 102% | 101% | 96% | 94% |
| 04:00 – 06:00 | 156% | 97% | 106% | 105% | 105% | 104% | 102% | 101% | 97% | 95% |
| 06:00 – 08:00 | 156% | 97% | 106% | 105% | 104% | 104% | 102% | 101% | 97% | 94% |
| 08:00 – 10:00 | 156% | 97% | 106% | 105% | 104% | 104% | 102% | 101% | 97% | 94% |
| 10:00 – 12:00 | 156% | 97% | 106% | 105% | 104% | 104% | 102% | 101% | 97% | 94% |
| 12:00 – 14:00 | 156% | 97% | 106% | 105% | 104% | 103% | 102% | 101% | 98% | 94% |
| 14:00 – 16:00 | 156% | 97% | 106% | 105% | 104% | 103% | 102% | 101% | 97% | 94% |
| 16:00 – 18:00 | 155% | 97% | 106% | 105% | 104% | 103% | 102% | 101% | 97% | 95% |
| 18:00 – 20:00 | 155% | 97% | 106% | 105% | 104% | 103% | 101% | 101% | 96% | 94% |
| 20:00 – 22:00 | 155% | 97% | 106% | 105% | 104% | 103% | 101% | 101% | 96% | 94% |
| 22:00 – 00:00 | 155% | 97% | 105% | 105% | 104% | 103% | 102% | 101% | 96% | 94% |
Figures 13 through 16 depict upload speed as a percentage of advertised speed for each product. As with download speed, upload speed across most products met or exceeded the advertised rate. Bell Aliant's 7x0.64Mbps DSL plan, TELUS’ 6x1Mbps DSL plan and Bell Canada’s 5x1Mbps, 15x10Mbps and 25x10 DSL plans did not achieve 90% of advertised upload speeds. Both the TELUS and Bell Aliant plans delivered 81% whilst Bell Canada’s plan delivered 86%, 60% and 84% of advertised speeds respectively. Bell Aliant’s plan and TELUS’ plan were the same DSL products that delivered download rates below 90% of their advertised download speed. All other plans delivered 92% of advertised speed or higher. TELUS’ 15x1Mbps FTTH plan delivered vastly above the advertised rates - delivering 311% of advertised speed.
Figure 13: Upload Speed as a percentage of Advertised Speed by Product, 5-9Mbps speed bucket
Long Description
This bar chart illustrates upload speed as a percentage of advertised speed for each product in the 5-9Mbps speed bucket as follows:
| ISP Product | Upload Speed Throughput |
|---|---|
| MTS 5X0.512 | 96% |
| Bell Canada 5X1 | 86% |
| Videotron 5X1 | 99% |
| TELUS 6X1 | 81% |
| Cogeco 6X2 | 102% |
| Bell Aliant 7X0.64 | 81% |
| Shaw 7.5X0.5 | 105% |
Figure 14: Upload Speed as a percentage of Advertised Speed by Product, 10-15Mbps speed bucket
Long Description
This bar chart illustrates upload speed as a percentage of advertised speed for each product in the 10-15Mbps speed bucket as follows:
| ISP Product | Upload Speed Throughput |
|---|---|
| Shaw 10X0.5 | 102% |
| Videotron 10X1.5 | 99% |
| MTS 10X2 | 95% |
| TELUS 15X1 | 311% |
| Cogeco 15X2 | 100% |
| Bell Canada 15X15 | 118% |
| Bell Canada 15X10 | 60% |
Figure 15: Upload Speed as a percentage of Advertised Speed by Product, 16-39Mbps speed bucket
Long Description
This bar chart illustrates upload speed as a percentage of advertised speed for each product in the 16-39Mbps speed bucket as follows:
| ISP Product | Upload Speed Throughput |
|---|---|
| Northwestel 16X0.768 | 92% |
| Eastlink 20X2 | 107% |
| MTS 25X2 | 95% |
| Shaw 25X2.5 | 105% |
| TELUS 25X5 | 120% |
| Bell Canada 25X10 | 84% |
| Bell Canada 25X25 | 109% |
| Rogers 30X5 | 106% |
| Shaw 30X5 | 105% |
| Videotron 30X10 | 105% |
Figure 16: Upload Speed as a percentage of Advertised Speed by Product, 40Mbps+ speed bucket
Long Description
This line bar chart illustrates upload speed as a percentage of advertised speed for each product in the 40Mbps+ speed bucket as follows:
| ISP Product | Upload Speed Throughput |
|---|---|
| Cogeco 40X10 | 101% |
| Northwestel 50X2 | 94% |
| Eastlink 50X5 | 101% |
| Shaw 50X5 | 105% |
| Bell Canada 50X10 | 98% |
| TELUS 50X10 | 122% |
| Bell Canada 50X50 | 113% |
| Cogeco 60X10 | 101% |
| Rogers 60X10 | 108% |
| Videotron 60X10 | 108% |
| Rogers 70X10 | 105% |
| Eastlink 100X10 | 108% |
| Rogers 100X10 | 107% |
| Bell Aliant 100X50 | 111% |
| Eastlink 150X10 | 107% |
| Bell Aliant 150X50 | 111% |
| Rogers 250X20 | 106% |
Table 2 shows the advertised upload speed for each ISP and corresponding products included in this study, expressed both in absolute terms and as a percentage of the advertised rates.
| ISP | Technology | Package | Actual Upload Speed (Mbps) | Actual/Advertised Upload Speed (%) |
|---|---|---|---|---|
| MTS | DSL | 5x0.512 | 0.49 | 96% |
| Bell Canada | DSL | 5x1 | 0.86 | 86% |
| Videotron | Cable/HFC | 5x1 | 0.99 | 99% |
| TELUS | DSL, FTTH | 6x1 | 0.81 | 81% |
| Cogeco | Cable/HFC | 6x2 | 2.03 | 102% |
| Bell Aliant | DSL | 7x0.64 | 0.52 | 81% |
| Shaw | Cable/HFC | 7.5x0.5 | 0.52 | 105% |
| Shaw | Cable/HFC | 10x0.5 | 0.51 | 102% |
| Videotron | Cable/HFC | 10x1.5 | 1.49 | 99% |
| MTS | DSL, FTTH | 10x2 | 1.90 | 95% |
| TELUS | DSL, FTTH | 15x1 | 3.11 | 311% |
| Cogeco | Cable/HFC | 15x2 | 1.99 | 100% |
| Bell Canada | DSL | 15x10 | 5.98 | 60% |
| Bell Canada | FTTH | 15x15 | 17.77 | 118% |
| Northwestel | Cable/HFC | 16x0.768 | 0.71 | 92% |
| Eastlink | Cable/HFC | 20x2 | 2.14 | 107% |
| MTS | DSL, FTTH | 25x2 | 1.89 | 95% |
| Shaw | Cable/HFC | 25x2.5 | 2.63 | 105% |
| TELUS | DSL, FTTH | 25x5 | 5.99 | 120% |
| Bell Canada | DSL | 25x10 | 8.42 | 84% |
| Bell Canada | FTTH | 25x25 | 27.35 | 109% |
| Rogers | Cable/HFC | 30x5 | 5.29 | 106% |
| Shaw | Cable/HFC | 30x5 | 5.24 | 105% |
| Videotron | Cable/HFC | 30x10 | 10.52 | 105% |
| Cogeco | Cable/HFC | 40x10 | 10.10 | 101% |
| Northwestel | Cable/HFC | 50x2 | 1.88 | 94% |
| Eastlink | Cable/HFC | 50x5 | 5.06 | 101% |
| Shaw | Cable/HFC | 50x5 | 5.23 | 105% |
| Bell Canada | DSL | 50x10 | 9.78 | 98% |
| TELUS | DSL, FTTH | 50x10 | 12.24 | 122% |
| Bell Canada | FTTH | 50x50 | 56.71 | 113% |
| Cogeco | Cable/HFC | 60x10 | 10.06 | 101% |
| Rogers | Cable/HFC | 60x10 | 10.82 | 108% |
| Videotron | Cable/HFC | 60x10 | 10.75 | 108% |
| Rogers | Cable/HFC | 70x10 | 10.52 | 105% |
| Eastlink | Cable/HFC | 100x10 | 10.79 | 108% |
| Rogers | Cable/HFC | 100x10 | 10.75 | 107% |
| Bell Aliant | FTTH | 100x50 | 55.50 | 111% |
| Eastlink | Cable/HFC | 150x10 | 10.70 | 107% |
| Bell Aliant | FTTH | 150x50 | 55.43 | 111% |
| Rogers | Cable/HFC | 250x20 | 21.14 | 106% |
TELUS upload tiers were tested only in the provinces of Alberta and British Columbia. Upload speed tests were not performed in Quebec for TELUS’ 15x1.5Mbps, 25x3Mbps, and 50x12Mbps products, as the upload tiers did not meet the minimum number of whiteboxes threshold required for inclusion.
Latency
Latency is a measure of how long it takes a packet to travel between point A and point B. It is a significant factor in Internet performance, as latency is a fundamental property of the infrastructure upon which everything else must build. If you have a high-latency link, then it does not matter how fast your broadband connection is; you will be limited by latency.
The results presented in this section show ‘round-trip’ latency (i.e. how long it takes for a packet to travel from point A to point B and then back to point A). While round-trip latency is the most common latency measurement taken (for example, the ‘ping’ utility captures round-trip latency), the ‘round-trip’ qualifier is very often omitted. For the remainder of this document ‘latency’ should be taken to mean ‘round-trip latency’. Please note that the geographical proximity of the servers to the end user will affect latency results, as latency is proportional to distance.
Latency is almost always expressed in milliseconds. Lower results are better. Latency itself has a lower bound governed by the speed of light, and often there are technological limitations which raise that lower bound. For example, DSL services typically have higher latencies than FTTH services.
Whilst latency is unrelated to line speed, an increase in latency can have a detrimental effect on how long it takes to transfer files and other objects. Moreover, an increase in latency during peak hours is an early indicator of congestion somewhere on the network path, as routers are taking longer to receive data packets and pass them on. It is worth noting that even the highest latencies exhibited here would more than be adequate for any common Internet application at present. For the majority of use cases, the approximately 10ms latency difference between the best and worst service would be indistinguishable.
Figure 17 below shows peak period latency by technology. DSL services yielded the highest latencies, at 21.8ms on average. Cable/HFC services were near behind at 20.3ms. Latency of FTTH services was lowest at 9.2ms. This matches the behaviour of web page loading time, emphasizing the existence of a positive correlation between them. All access technologies also show very little difference between the peak and off-peak latencies, with FTTH displaying the smallest difference.
Figure 17: Peak vs Off-Peak Latency by technology
Long Description
This bar chart illustrates peak period latency by technology as follows:
| Technology | Latency –Off-Peak | Latency-Peak |
|---|---|---|
| DSL | 21.2 milliseconds | 21.8 milliseconds |
| Cable/HFC | 19.6 milliseconds | 20.3 milliseconds |
| FTTH | 9.2 milliseconds | 9.2 milliseconds |
Figure 18 shows latency by access technology and speed bucket. Latency for FTTH services are consistently very low, at 10.2ms or less, regardless of speed bucket and time of day. DSL and Cable/HFC services exhibited higher latencies that varied by speed bucket too. For DSL, latency improved with the faster services, which is most likely caused by the use of newer DSL variants (such as VDSL) coupled with shorter copper loop lengths. However, average latency for the 5-9Mbps DSL bucket was pulled upwards by one ISP who deliver services to remote areas. Latency for cable services increased with higher speed services. This counterintuitive behavior is a function of the sample for Cable/HFC, which included an ISP delivering high speed cable services to some remote areas of the East.
Figure 18: Latency by advertised download speed and technology
Long Description
This bar chart illustrates latency by access technology and speed bucket as follows:
| Technology/Speed Bucket | Latency –Off-Peak | Latency-Peak |
|---|---|---|
| DSL/5-9Mbps | 25.5 milliseconds | 26.9 milliseconds |
| DSL/10-15Mbps | 21.1 milliseconds | 21.3 milliseconds |
| DSL/16-39 Mbps | 17.9 milliseconds | 18.1 milliseconds |
| DSL/40+ Mbps | 17.7 milliseconds | 17.8 milliseconds |
| Cable/HFC/5-9 Mbps | 15.7 milliseconds | 16.3 milliseconds |
| Cable/HFC/10-15 Mbps | 16.4 milliseconds | 17.2 milliseconds |
| Cable/HFC/16-39 Mbps | 19.8 milliseconds | 20.5 milliseconds |
| Cable/HFC/40+ Mbps | 21.2 milliseconds | 21.2 milliseconds |
| FTTH/10-15 Mbps | 10.2 milliseconds | 10.3 milliseconds |
| FTTH/16-39 Mbps | 8.8 milliseconds | 8.8 milliseconds |
| FTTH/40+ Mbps | 9.0 milliseconds | 9.1 milliseconds |
Packet Loss
The rate of packet loss describes how likely it is that a packet sent from point A will not reach point B. Packet loss is closely related to latency and is a fundamental metric in determining how applications perform on a broadband connection. A high rate of packet loss will prevent many applications from working to a satisfactory level. A small increase in packet loss during peak hours is to be expected, as networks are busier and congestion at even one point in a network path may lead to a packet being dropped. Packet loss is generally measured and expressed as a percentage of the overall data packets sent.
Figure 19 below shows that packet loss was low across all access technologies, with the highest level of packet loss seen being 0.17%. This was observed on lower speed DSL during peak hours. FTTH technology delivered significantly lower packet loss than DSL and Cable/HFC, at just 0.04%. It is worth noting that ISPs may not be responsible for packets being lost; packet loss can occur anywhere along a network path, including after the time traffic leaves the ISP’s network.
Figure 19: Peak vs Off-Peak Packet Loss by technology
Long Description
This bar chart illustrates peak period packet loss by technology as follows:
| Technology | Latency –Off-Peak | Latency-Peak |
|---|---|---|
| DSL | 0.13 % | 0.17 % |
| Cable/HFC | 0.1% | 0.13 % |
| FTTH | 0.04 % | 0.04 % |
Figure 20 depicts packet loss by access technology and speed bucket. FTTH services delivered the lowest levels of packet loss across all speed buckets, at 0.07% or lower on average. The fastest FTTH services in the 40Mbps+ bucket delivered 0.02% packet loss. Cable/HFC services exhibited very similar levels of packet loss regardless of speed bucket, with all speed buckets delivering between 0.13% and 0.14% of packet loss during peak hours. There is a consistent 0.02-0.04% increase in packet loss during peak hours. DSL services also delivered very low levels of packet loss, with the exception of services in the 5-9Mbps bucket where an average of 0.39% packet loss was found during peak hours. With the exception of the 5-9Mbps DSL service, the packet loss exhibited here across all technologies and buckets would be imperceptible to modern Internet applications. Users of some 5-9Mbps DSL services may notice some small intermittent delays with this level of packet loss, such as lag when loading web pages.
Figure 20: Packet Loss by advertised download speed and technology
Long Description
This bar chart illustrates packet loss by access technology and speed bucket as follows:
| Technology/Speed Bucket | Packet Loss –Off-Peak | Packet Loss-Peak |
|---|---|---|
| DSL/5-9Mbps | 2.62% | 3.86% |
| DSL/10-15Mbps | 0.70% | 0.73% |
| DSL/16-39 Mbps | 0.75% | 0.73% |
| DSL/40+ Mbps | 0.44% | 0.49% |
| Cable/HFC/5-9 Mbps | 1.1% | 1.43% |
| Cable/HFC/10-15 Mbps | 0.82% | 1.26% |
| Cable/HFC/16-39 Mbps | 1.14% | 1.24% |
| Cable/HFC/40+ Mbps | 0.97% | 1.25% |
| FTTH/10-15 Mbps | 0.69% | 0.68% |
| FTTH/16-39 Mbps | 0.51% | 0.51% |
| FTTH/40+ Mbps | 0.32% | 0.23% |
Web Page Loading Time
The web page loading time test captures how long it takes for all of the elements of a web page to be received by an end user. Unlike other measurements, this test is conducted against real websites on the Internet, rather than dedicated test servers.
Web page loading time is heavily influenced by many factors, including a user’s download speed, latency, the speed of the web server and congestion in other networks. The proximity of the user to the web server is one of the factors in determining web page load times. Content providers (such as Google and Facebook) will typically host their web servers in major metropolitan areas. Customers in remote locations will often have high latency to the web servers, which will impact page load times.
Figure 21 shows web page loading time by access technology during the peak and off-peak time periods. Loading times proved generally low across all access technologies, with DSL exhibiting the highest loading times. This is to be expected as products with lower advertised rates and higher average latencies typically use DSL. All access technologies are also very consistent between the peak and off-peak periods, with loading times proving only slightly higher during peak hours. FTTH in particular shows almost no change between the peak and off-peak periods.
Figure 21: Peak vs Off-Peak Web page Loading Time by technology
Long Description
This bar chart illustrates web page loading time by access technology during the peak and off-peak time periods as follows:
| Technology | Web page loading time –Off-Peak | Web page loading time-Peak |
|---|---|---|
| DSL | 1.3 seconds | 1.3 seconds |
| Cable/HFC | 0.9 seconds | 0.9 seconds |
| FTTH | 0.7 seconds | 0.7 seconds |
Figure 22 below shows average web page loading times by ISP and access technology. ISPs using DSL services generally deliver higher loading times than Cable/HFC or FTTH services. Bell Aliant and MTS show the highest web page loading times. This is due to the fact that Bell Aliant DSL users are concentrated in rural areas such as New Brunswick, Newfoundland and Prince Edward Island. Similarly, MTS’s users are also concentrated in rural areas, where the remoteness leads to higher latencies and lower speed products being available. Bell Canada’s web browsing performance also compares favourably to Bell Aliant’s due to its significantly better throughput and latency results. Differences between the peak and off-peak measurement periods are also larger for ISPs providing DSL services compared to other access technologies.
Web page loading times for ISPs providing Cable/HFC and FTTH services never exceed 1 second with the exception of Northwestel’s Cable/HFC services, which show a loading time of 1.4 seconds during the peak period. This reflects its lower download speed compared to other Cable/HFC and FTTH services.
Figure 22: Peak vs Off-Peak Web page Loading Time by technology and ISP
Long Description
This bar chart illustrates average web page loading times by ISP and access technology …as follows:
| Technology-ISP | Web page loading time –Off-Peak | Web page loading time-Peak |
|---|---|---|
| DSL-Bell Canada | 0.9 seconds | 0.9 seconds |
| DSL-Bell Aliant | 1.9 seconds | 2.1 seconds |
| DSL-MTS | 1.7 seconds | 1.8 seconds |
| DSL-TELUS | 1.2 seconds | 1.2 seconds |
| Cable/HFC-Cogeco | 0.9 seconds | 0.9 seconds |
| Cable/HFC-Eastlink | 0.8 seconds | 0.8 seconds |
| Cable/HFC-Northwestel | 1.3 seconds | 1.4 seconds |
| Cable/HFC-Rogers | 0.8 seconds | 0.8 seconds |
| Cable/HFC-Shaw | 0.9 seconds | 1.0 seconds |
| Cable/HFC-Videotron | 0.9 seconds | 1.0 seconds |
| FTTH-Bell Canada | 0.6 seconds | 0.6 seconds |
| FTTH-Bell Aliant | 0.7 seconds | 0.7 seconds |
| FTTH-TELUS | 0.8 seconds | 0.8 seconds |
Studies in other markets have shown that download speed determines web browsing performance up to approximately 10Mbps, after which latency becomes the dominant factor. This is indicated in figure 23 below, which depicts average web page loading time by the advertised download speed for each type of access technology.
As expected, web page loading time improved as download speed increased, although this improvement tails off rapidly as advertised rates go above 10Mbps. 5-9Mbps DSL services web pages loaded in the longest time of 2.2 seconds. This improved to 1.1 seconds for 10-15Mbps DSL services. The rate of improvement declined after this point, with 40Mbps+ DSL services loading pages in 0.7 seconds. Similar behavior is observed on Cable/HFC services, with a significant improvement between 5-9Mbps services and 10-15M services (1.4 seconds versus 1.0 seconds). FTTH services yielded the lowest page load times, even with their slowest tiers. This is driven by the lower latencies that FTTH services exhibited, as seen earlier in the latency analysis.
Figure 23: Web page loading time by advertised download speed and technology
Long Description
This bar chart illustrates average web page loading time by the advertised download speed for each type of access technology as follows:
| Technology/Speed Bucket | Web page loading time –Off-Peak | Web page loading time -Peak |
|---|---|---|
| DSL/5-9Mbps | 2.01 seconds | 2.14 seconds |
| DSL/10-15Mbps | 1.12 seconds | 1.13 seconds |
| DSL/16-39 Mbps | 0.87 seconds | 0.89seconds |
| DSL/40+ Mbps | 0.70seconds | 0.71 seconds |
| Cable/HFC/5-9 Mbps | 1.38 seconds | 1.41 seconds |
| Cable/HFC/10-15 Mbps | 0.99seconds | 1.02seconds |
| Cable/HFC/16-39 Mbps | 0.88 seconds | 0.92 seconds |
| Cable/HFC/40+ Mbps | 0.79 seconds | 0.83 seconds |
| FTTH/10-15 Mbps | 0.79 seconds | 0.80 seconds |
| FTTH/16-39 Mbps | 0.68 seconds | 0.69 seconds |
| FTTH/40+ Mbps | 0.63 seconds | 0.64 seconds |
DSL services displayed higher web page loading times across all regions, as shown in figure 24 below. In particular, DSL services in the East displayed the highest average web page loading time of 2.1 seconds during the peak period compared to other regions as well as compared to all other technologies in all regions. This reflects the low download speed of DSL in this region. Latency in the East region is also higher than in the Central region. Given that DSL users in the East region are concentrated in rural areas, it is to be expected that latency is higher as packets of information have to travel longer distances to an exchange. This also impacts the performance of web browsing in the region. In contrast, web page loading times of FTTH technology were the lowest across all regions, displaying a maximum loading time of 0.77 seconds during peak hours in the West & North region.
Figure 24: Peak vs Off-Peak Web page Loading Time by technology and region
Long Description
This bar chart illustrates average web page loading time by each type of access technology and region as follows:
| Technology-Region | Web page loading time –Off-Peak | Web page loading time -Peak |
|---|---|---|
| DSL-Central | 1.1 seconds | 1.1 seconds |
| DSL-East | 2.0 seconds | 2.1 seconds |
| DSL-West & North | 1.4 seconds | 1.5 seconds |
| Cable/HFC-Central | 0.8 seconds | 0.9 seconds |
| Cable/HFC-East | 1.1 seconds | 1.1 seconds |
| Cable/HFC-West & North | 1.0 seconds | 1.0 seconds |
| FTTH-Central | 0.6 seconds | 0.6 seconds |
| FTTH-East | 0.7 seconds | 0.8 seconds |
| FTTH-West & North | 0.8 seconds | 0.8 seconds |
Conclusion
This report represents the first step in providing public access to quality data about the performance of Canadian Internet services.
The majority of broadband services met or exceeded their advertised speeds, with some notable exceptions. FTTH services achieved 121% of advertised download speeds on average, with Cable/HFC and DSL services delivering 105% and 103% respectively. However, two DSL products, one each from Bell Aliant and TELUS, delivered below their advertised download speeds, with download speeds during peak hours falling as low as 77% for Bell Aliant.
Upload speeds generally also met or exceeded advertised rates although – as with download speeds – several DSL products were found to provide speeds below those advertised. Both Bell Aliant and TELUS had DSL products achieving 81% of their respective advertised rates, whilst three of Bell Canada’s products delivered between 60% and 86%. Some ISPs overprovisioned the upload speeds of their products. In the most extreme example, one DSL product from TELUS achieved 311% of the advertised upload rate.
Differences between peak and off-peak performance were minimal. On average, download speeds varied by at most 6%, and upload speeds by at most 2%, regardless of technology.
In the latency, packet loss and web browsing metrics, FTTH services delivered the strongest and most consistent results. FTTH services showed almost no difference in results between the peak and off-peak measurement periods. More variability was found in DSL and Cable/HFC services. However, even the highest latencies exhibited during testing would be more than adequate for any common Internet application. The latency and web browsing results measured in Canada compare favorably to those measured in other jurisdictions, including the United States.
Whilst this report has presented numerous findings about the current state of Canadian Internet performance, it is also important to recognize its limitations. This report presents a snapshot of current performance; ongoing monitoring should be conducted to ensure that the market continues to operate as effectively as it does currently, and that new products being deployed are meeting their advertised service levels.
Moreover, this report focuses heavily on network layer measurements, such as throughput, latency and packet loss. Increasingly, the Internet is being used for delivering services such as on-demand video, gaming and real-time communications and consumers have an interest in learning how these services perform on their Internet connections.
In support of the CRTC 2016 three-year plan, the Measuring Broadband Canada platform will continue to collect measurements on the performance of Canadian Internet connections by including more ISP participants, and may in the future expand beyond network layer measurements to new measurements that enhance the utility of the project.
Appendix A
August 2016
Important Notice
Copyright
The material in this document is protected by Copyright. No part of the materials in this document may be reproduced for any purpose whatsoever without the written permission of SamKnows.
Sample Plan
The below is the sample plan created for the Measuring Broadband Canada report. The parties acknowledge that a limited exception to the 25,000 minimum subscriber rule for testing a specific speed package has been made to address competitive inequity between ISPs serving the same area.
| ISP | Download | Upload |
|---|---|---|
| Bell | 5 | 1 |
| Bell | 15 | 10 |
| Bell | 15 | 15 |
| Bell | 25 | 10 |
| Bell | 25 | 25 |
| Bell | 50 | 10 |
| Bell | 50 | 50 |
| Bell Aliant | 7 | 0.64 |
| Bell Aliant | 100 | 30 |
| Bell Aliant | 150 | 30 |
| COGECO | 6 | 2 |
| COGECO | 15 | 2 |
| COGECO | 40 | 10 |
| COGECO | 60 | 10 |
| EASTLINK | 20 | 2 |
| EASTLINK | 100 | 10 |
| EASTLINK | 50 | 5 |
| EASTLINK | 150 | 10 |
| MTS | 5 | 0.512 |
| MTS | 10 | 2 |
| MTS | 25 | 2 |
| NORTHWESTEL | 16 | 0.768 |
| NORTHWESTEL | 50 | 2 |
| ROGERS | 30 | 5 |
| ROGERS | 60 | 10 |
| ROGERS | 70 | 10 |
| ROGERS | 100 | 10 |
| ROGERS | 250 | 20 |
| SHAW | 7.5 | 0.5 |
| SHAW | 10 | 0.5 |
| SHAW | 25 | 2.5 |
| SHAW | 30 | 5 |
| SHAW | 50 | 5 |
| TELUS | 6 | 1 |
| TELUS | 15 | 1 |
| TELUS | 25 | 5 |
| TELUS | 50 | 10 |
| VIDEOTRON G.P. | 5 | 1 |
| VIDEOTRON G.P. | 10 | 1.5 |
| VIDEOTRON G.P. | 30 | 10 |
| VIDEOTRON G.P. | 60 | 10 |
Code of Conduct
2015 CRTC MEASURING BROADBAND CANADA MEASUREMENT PLATFORM
May 21st, 2015
The Canadian Radio-television and Telecommunications Commission (the “CRTC”) has established a Canadian Broadband Measurement Project (the “Project”) in collaboration with participating Canadian ISPs (“ISPs”) and SamKnows Limited (“SamKnows”).
To ensure the effective and unbiased use of the data, and the integrity and validity of study results, this Code of Conduct has been drawn up which each of the ISPs, SamKnows and the CRTC (each a “Participant”) agrees to sign.
The undersigned, as a Participant in the Project, agree with the following principles:
- At all times during the Project, the Participants agree to act in good faith.
- The Participants agree not to tamper with the testing infrastructure or methodology, or take any other actions which influence the results of any test, through positive action or omission, for any individual panelist or participating ISP.
- The Participants acknowledge that it will not be a violation of the principle set out in paragraph 2 above for ISPs to:
- Operate and manage their business, which includes modifying or improving services delivered to any class of subscribers that may or may not include panelists among them, provided that such actions are consistent with normal business practices;
- Address service issues for individual panelists at the request of the panelists or based on information not derived from the trial; and
- Advise their customer service representatives of the identity of subscribers who are panelists in the trial so that those representatives may address service and billing questions.
- The Participants acknowledge that it will not be a violation of the principle set out in paragraph 2 above to monitor the tests and components of the testing architecture provided that no impact to the CRTC data occurs. In particular, the Participants acknowledge that the ISPs may advise SamKnows wherever a technical concern is observed in respect of an individual panelist, so that SamKnows may contact said panelist to investigate and remedy the problem.
- Without limiting the terms of the Non-Disclosure Agreement entered into on or around January 26, 2015, by the ISPs and SamKnows, the ISPs and SamKnows agree not to publish any data generated by the tests, nor make any public statements about or based on such data until such time as the CRTC releases data or makes a public statement regarding any of the results of the tests, except where expressly permitted by the CRTC, all provided that any data published is data previously released by the CRTC and any public statements made are based solely on such previously released data.
- For greater certainty, this Code of Conduct does not apply to any SamKnows measurement panel, or any data generated from such panel, that any ISP operates independently of the CRTC Broadband Canada Measurement Platform.
- The Participants shall ensure that their employees, agents and representatives, as appropriate, act in accordance with this Code of Conduct.
- Date modified: