The time has come for the “2017 edition – Best CMS Platform for Web Performance” study!
For those of you that weren’t there for last year’s blog post on web performance, titled “What is the Best E-Commerce Platform for Web Performance?“, let me catch you up on the purpose of this little study of ours.
Why is web performance a big issue for e-commerce companies?
Well it’s a simple calculation, really. 😉
“Time is money”, as the saying goes. So for an ever growing economy like the e-commerce business, that amounted to $1,915 billion of worldwide revenue in 2016 (and who will keep increasing by up to 20% each year, in the near future), time truly is of the essence.
And in the e-commerce world, who is the worst enemy of speed (and thus revenue)? Bad web performance.
So you can imagine how most companies, aware of the importance of e-commerce and heavily invested in the field, would want to get the better out of their CMS platform, and achieve a high level of performance and user experience. Simply because if your e-store is slow, customer won’t come / stay / buy, and you’ll have lost a lot of money in customer acquisition / marketing / development / …etc.
Why choosing the right CMS platform is such an important decision for web performance?
Using a CMS to build an online store is a sound tactical choice. It allows you to focus on:
- sales, instead of on the technical aspects of running an online store,
- SEO and marketing actions,
- or even on the finer points of you internet catalogue.
However, using a CMS instead of a custom-made platform for an e-shop is not without consequences:
- By default, it often implements many functionalities that might not be useful, which can make the overall system quite heavy and slow at first.
- It can be more or less prone to bugs, slowdowns or even downtimes; depending on how the CMS is built and customized after installation.
So when it comes down to it, to your speed and conversion rate, choosing the right CMS platform for your e-store is crucial.
And that is why, for the second year in a row, we decided to conduct a study on the performance of 7 leading, well-known, or brand new, e-commerce platforms that seem to be all the rage in 2017. 😉
The idea is to establish a ranking, and find out which one of the platforms delivers the most stable and fastest experience during the classic e-commerce sales funnel.
So without further ado, here is our 2017 competitors pick for the Best E-commerce Platform competition:
- Magento (we combined the results for Magento EE and CE, but we’ll be sure to do a more thorough blog post on their respective performances later on),
- Drupal Commerce,
- and Demandware.
You’ll notice on the chart below, representing the worldwide proportion of websites for each of our chosen CMS, that some in our selection have quite small communities. But we do believe that these CMS present high hopes for the future of e-commerce, in terms of technology, so we’ve decided to test them nonetheless.
METHODOLOGY FOR THIS STUDY
How did we select the websites for the study?
Like for our 2016 CMS platforms study, the study was carried out on a panel of 120 websites (20 per technology), with an Alexa ranking between 200 000 and 600 000. The medium values revolve around rank 430.000, which represents websites with approximately 67.000 visitors/month.
Choosing such a range gives more representativity to the study, since these websites are more likely to meet minimum UX and performance agreed-upon standards.
How did we analyze the performance?
The data was collected using our monitoring tool, Quanta: it reproduces the behavior of a visitor going through the classic sales funnel (Home > Category > Subcategory > Product > Cart), and analyzes the speed and availability of each page from the Home Page to the Cart.
Selected sites are analyzed every minute, 24/7, during 14 days. The tool goes through the main five pages of the website (pages following the sales funnel, ndlr), every minute. And does this for every website selected for the study. This represents a total of 600 clicks per minute, and a total amount of 12 millions HTTP requests for the whole study.
The numbers featured throughout the study are the median values for each page of a regular sales funnel, out of the 20 websites selected per CMS. That is to say that for a better reading comfort, we displayed the median results for all websites, per CMS.
For each technology, we created a summary in the form of an illustration, displaying availability rates and page load times, for each step of an ordinary sales funnel.
Disclaimer. You may notice that the further you go into the sales funnel, the more results are likely to decrease. That’s perfectly normal since the results are added up to each step that follows.
Now, let’s cut the chit-chat and dive right in! 😉
WooCommerce is the leading e-commerce platform on the market. Free and easy to use, it works as an extension of WordPress, and thus:
- does not require an external hoster,
- uses the same plugins as the ones for WordPress (ex: for SEO, it’s Yoast SEO),
- has many themes and plugins available to upgrade the stores,
- and is very scalable, making it a good solution both for big and small companies.
But even with all these technical advantages, it’s too early to declare WooCommerce as the ideal solution for performance. That was the case in 2016, and it remains so today.
The performance test shows that WooCommerce is still overly average in terms of availability and average response time (ART).
The average page load time for all monitored stores is 0,63 seconds (s), which is OK, but not great considering that the immediacy threshold, the holy grail of web performance hailed by Google itself, is 100 milliseconds. Plus, out of the 20 tested websites, we noticed a big a gap in the ARTs; proving that WooCommerce is not a reliable platform in its “out of the box”configuration. For example, a cart once loaded in 4,14 s!
However, it’s true that the results are way better than last year’s. The Home Page and the Cart remain big performance bottlenecks, but we’ve seen a 20% increase in speed for these two steps.
For the availability, WooCommerce comes in 3rd in our competition. With its 99,87% availability rate on average (0,35% more than last year), the solution is stable, even if it still represents a 13 minutes downtime every week on average.
In conclusion, despite its great technical advantages and its increased availability and average response time, WooCommerce as it is often set up, can’t be considered a reference in matters of performance. Of course, in some measure, you can overcome its slowness by:
- using a hoster specialized in WordPress and WooCommerce websites,
- using additional modules like WP Rocket,
- compressing big and heavy images,
- or caching the maximum number of elements in your code.
The Canadian solution Shopify is very easy to use if you want to launch an e-commerce platform. It was created in 2006, in order to build a complete website in ten minutes, tops.
Shopify is an all-in one e-commerce platform, which doesn’t require any knowledge in development to set up. You simply have to create your catalogue with the articles, texts and images you need for the store. The structure of the website and themes are already provided when subscribing to the solution.
Shopify is not only simple, but also very performant. Again! The data extracted from our study revealed that on average, web pages loaded in 0,35 s, when the Google Guidelines recommend an average of 0,50 s for loading times! Thanks to its fast response times and results, Shopify delivers a very good overall user experience.
But contrary to last year, Shopify can’t be considered the most stable solution. It actually comes in 6th, with its 99,83% Sales Funnel availability (99,95% last year), on average. Of course, this could be explained by the increase in worldwide availability incidents but still, it’s a little bit disappointing for our 2016 Performance Champion. It represents a 17 minutes downtime each week.
Such performances can be explained by the fact that Shopify is a full-Saas platform; meaning that it manages both the CMS and its hosting. This specificity is definitively an advantage pertaining to loading times and availability, but the drawback is that when Shopify’s servers are down or suffer a slowdown, all the websites hosted are affected in the same way. If you want to know more about this subject, I actually wrote a blog post on the impact of the 2016 Dyn DNS attacks on Shopify websites. You can find it here.
Created in 2008, Magento is an open-source e-commerce solution, with a big and active community, providing advanced user assistance and many dedicated extensions. It is currently the leading e-commerce platform in Europe, with more than 230 000 websites using their technology.
However, to this day, it remains quite difficult to master, due to its architecture and all its functionalities. Skilled developers are often needed to undertake modifications.
During our study, we discovered that, in 2016, Magento has been a rather stable platform since the availability rate reached 99.89%, on average, which is better than 2015’s 99,71%. The availability rate for the Subcategory, Product and Cart steps are way more satisfactory. So it comes in 3rd position in terms of general availability.
Although Magento websites are the most frequented (with an average of 3000 visitors per day), they are not the fastest! The results showed that the pages, composing a classic user’s sales funnel, loaded in 0,60 s on average, which is beyond the “under 500 milliseconds” goal set by Google. And even if the Product’s average response time has been cut by half since last year, it remains far from the racing cars that are the likes of Shopify.
Nonetheless, Magento’s Cart load time is reasonable with a result of 0,58 s; which is fast enough to retain clients until the end of the sales funnel.
In conclusion, we can safely say that Magento is a complex machine, making it slower than the average other solutions.
To improve its performance, try using Varnish, or the built-in Full Page Cache functionality, developed by Magento itself. Those two popular cache systems can really boost you store and help it near the sweet spot of 100 milliseconds, the immediacy threshold.
PrestaShop is free to download and install, and provides native basic functionalities. However, in order to have a fully functional website, it is necessary to download themes and extensions in the PrestaShop market, which are hardly ever free…
The analysis of PrestaShop websites’ performance showed average results. The average speed for every page is 0,65 s, with the Cart being the fastest page at 0,56 s. It presents significantly better results compared to last year’s, particularly for the Category and Subcategory pages.
However, PrestaShop still presented a very good (even if a little bit diminished) availability rate of 99,90%, garnering it the 2nd place in our competition for Best Availability.
Those results could possibly be improved using many optimization and caching tools the solution provides, as well as choosing hosting services fitted for PrestaShop, using a CDN and reverse proxy solutions.
BigCommerce is our second full-SaaS solution. A CMS that we chose to include in this 2017 edition of our study, because of its growing reputation and use by e-commerce companies. Created in 2009 by Eddie Machaalani and Mitchell Harper, following a chance meeting in an online chatroom in 2003, it is tailored for fast-growing, mid-market, companies.
Unlike a Magento or a Drupal Commerce, BigCommerce offers few customization options. There are four different levels of pricing – Standard, Plus, Pro and Enterprise -, that each offer specific features and services. So if you’re looking for a close-to-custom-made platform, BigCommerce is not made for you.
But even if it wears proudly its “Full-SaaS feel” when it comes to customization, BigCommerce is a great choice when it comes to overall web performance.
As you can see on the illustration below, most of the indicators for the Sales Funnel steps, for both availability and ART, are in the green. With its average response time of 0,50 s, BigCommerce is a fast platform. This response time places it in the third position of our competition for Best Average Response Time, and matches exactly the recommendation of Google when it comes to speed.
As is customary for many full-SaaS CMS, it is also quite stable. With an average availability rate of 99,87%, BigCommerce seems to be a valid option for e-commerce websites.
We monitored a panel of Drupal Commerce stores. Created in 2011, Drupal Commerce is an extension of Drupal, adding to the original CMS all the e-commerce features needed by online companies. It is free and open-source.
One of Drupal Commerce’s particularity, is to be really well-fitted for SEO, which allows websites to climb the rankings in Google search engine.
Drupal Commerce was a rather good solution when it came to performance. In fact, the medium for the pages load time is 0,56 s, which represents the fourth best performance out of all the technologies tested, very close to SaaS solutions like Shopify and BigCommerce!
With 99.86% availability rate, Drupal Commerce represents the sixth most available technology among the tested CMS. But even if it seems poor, Drupal Commerce remains a reliable technology to use for an e-commerce store.
However, creating and updating a Drupal Commerce based website is not as simple as using Shopify or PrestaShop. It requires a dedicated team of developers, and thus, to invest some time and money into the setup.
And last but not least, here comes Demandware. It is a cloud-based e-commerce platform created in 2004 by Stephan Schambach, and purchased in 2016 by Salesforce to reinforce its e-commerce services.
Demandware was conceived as a full SaaS platform with a touch of predictive technology, that aims at managing all the aspects of an e-business: hosting, marketing, and customer services. As you can see in the “CMS Proportion 2017” chart in the beginning of this study, Demandware is not a widely used CMS at the moment. But we chose to include it in our study because we do believe that the Salesforce buy will give this CMS nw possibilities of expansion.
That being said, let’s take a look at the results of our web performance test.
If you’ve been with us since the beginning of this study, the results in the graph below should not really be a surprise to you. As a Shopify or a BigCommerce, Demandware presents overall excellent results when it comes to availability and average response time.
See? Green almost everywhere! Demandware presents the best overall results in both availability rate and average response time.
The medium average response time is a tiny 0,35 s, with a really fast Cart that peaks at 0,23 s. Yup. That’s closing fast on the immediacy threshold, and are pretty impressive results.
But Demandware is not only fast, it is also really stable. With its 99,96% availability rate (which represents only 5 minutes of downtime each week), it’s the number 1 reliable CMS platform for this year’s study.
AND THIS YEAR BEST WEB PERFORMANT PLATEFORM IS…
Before unveiling the name of this year Web Performance champion, we thought it was a good idea to look more precisely at the availability rate and average response time results.
As we’ve said before, every e-company is different, and its unique features call for a tailored choice of CMS. Even if web performance as a whole must remain the key factor in choosing a CMS, some businesses will want to focus on availability, or on average response time.
So, here you go. Here are the results for the Best Availability Rate, and Best Average Response Time rankings.
As expected, the full Saas platforms perform well, especially when it comes to the average response times (advantage only possible because they host the websites themselves, instead of employing an external hosting service and communicating with it). And we can also conclude that it is not the most used platforms that presents the most impressive results. Moreover, if you compare these results to last year’s ranking, it cements the idea that CMS platforms, like all technology, evolve throughout the time, and that yesterday’s good results do not necessarily mean tomorrow’s success.
But these rankings are just partial answers to our founding question of “Who is the best web performant CMS platform for 2017?”. So without further ado, here is the definitive podium.
Drum roll please!
We’re pleased to announce that, after taking into account all the datas provided and analyzed, the most web performant e-commerce platform is Demandware.
The study revealed Demandware websites tends to be faster and more stable than other CMS-based websites. And more broadly, that seems to be the case for the other Saas CMS, like Shopify and BigCommerce.
BigCommerce is also a fast and reliable full Saas solution, garnering it the second place on our podium.
And last, but not least, on the 3rd step of our podium, we find Magento, that represents a safe and good option in terms of web performance. Even if it is not the fastest, or the most available, its medium scores in these two areas makes it a well balanced platform.
Of course, if your site runs under Prestashop, or WooCommerce, there is no reason to be alarmed! The results of all these platforms are very close, and the differences between the different rankings on the podium are often minimal. But we do believe that a little healthy competition can nudge the publishers of these solutions to better take into account the issue of web performance. 😉
Moreover, we observed during the study that significant differences could exist in the performance of sites running on the same technology. It’s just that some sites are better tuned than others. This only reinforces my belief (and the message I tried to get through to you!) that a CMS will never be perfect “out of the box”, and that optimizations to cater to the specific needs of an e-commerce website are absolutely necessary.
And this conclude this year Best Web Performant CMS Platform competition!
But here is a little sneak peek at our 2018 edition.
QUANTA’s Head of Operations, Matthieu Rosinski, tested the performance and scalability of Oro‘s new cutting-edge platform, OroCommerce, and the results were very promising! So we can’t wait for next year to come, and hope that we’ll have the opportunity to test the web performance of these websites. 😉
Remember that the most important conclusion to be drawn is that the technology used for a website is not the only key factor to get good performances.
Relying only on a CMS native availability or speed will certainly lead to poor performance, and numerous issues. So, to get the best performances possible, it is crucial to rely on a professional team of developers, who will be able to maximize website optimization, and improve page load time speed. And this, even for full Saas CMS platform that, even if performant in their out-of-the-box configurations, need fine tuning to push further the boundaries of web performance.
Another important thing to do when talking about web performance is to monitor e-commerce websites constantly, because you never know when a slowdown or downtime can become an important loss of revenue. So, companies must always keep in mind that performance is and remains key.