One of the great advantages of decoupled architecture is its ability to weather spikes in web traffic. With a traditional CMS, a sudden burst of traffic could overwhelm the server and paralyze your site, including the administration interface, which may only be used by a small fraction of your users. With a headless architecture, heavy traffic on the front-end can have little to no impact to the back-end.
This was a key selling point of our custom, Next.js-based, Next-Drupal solution for a large client whose website receives very high traffic. While the primary goal was to migrate their site from Drupal 7 to 10, they also took the opportunity to rebuild their site in a more robust and resilient format. We recently completed load testing of the new site to help prove it could handle the projected traffic.
Testing Methodology
Optimal load testing doesn’t mean throwing all the traffic you can at the site, but rather determining and testing for a target traffic volume. In this case, we were able to look at historical analytics data to determine the expected load. We then set a target that exceeded that, allowing for traffic spikes or organic growth.
Instead of immediately opening the floodgates to the traffic target, we staged a series of scaled tests, allowing us to better monitor and make adjustments if needed. We began with a small test that was lower than our expected traffic, and scaled up in steps, ending with a test that exceeded our expected volume.
To execute the tests we used k6 on Grafana Cloud. The open source version of k6 made it easy to develop the load test harness locally, and then execute them at scale in cloud environments. The k6 documentation outlines the three main types of tests:
- Protocol-based testing, specifically HTTP requests that bypass the user interface, and are primarily focused on backend performance.
- Browser-based testing, which verifies front-end performance by simulating user behavior using a headless browser.
- Hybrid testing, which is a combination of protocol-based testing and browser-based testing. This commonly uses protocol-based tests to scale up load, followed by a smaller number of browser-based tests to test front-end performance at scale.
During our tests, we ran into issues with the reliability of the hybrid phase, most likely because k6 browser tests are still considered experimental. In response to this, we ran a series of individual protocol and browser tests.
Generally in our tests, we had an initial ramp up period, an extended period of peak traffic, and then a ramp down period. In addition to scaling the requests per second, we also scaled up and down the number of simultaneous virtual users (VUs). Here’s a simple example from the k6 docs that shows multiple stages ramping up the number of virtual users.
import http from 'k6/http';
import { check, sleep } from 'k6';
export const options = {
stages: [
{ duration: '30s', target: 20 },
{ duration: '1m30s', target: 10 },
{ duration: '20s', target: 0 },
],
};
export default function () {
const res = http.get('https://httpbin.test.k6.io/');
check(res, { 'status was 200': (r) => r.status == 200 });
sleep(1);
}
Let your hosting providers know about your load testing plans in advance, they’ll appreciate it. It should also make it less likely that your tests will be blocked as an attack.
Architectural Decisions
Once we completed these tests, we were able to make some important architectural decisions to support the site’s expected traffic volume.
These decisions included setting the top 50 or so pages to pre-render during Next.js builds, allowing us to keep build times low while ensuring requests to high traffic pages are always super-fast. Other pages will be server-rendered on demand, but we’re also using Next.js Incremental Static Regeneration to cache generated pages and regenerate them when Sruoal content changes. This is supported out of the box with Next-Drupal.
We’ve also ensured that client-side API calls are cached using a stale-while-revalidate pattern. Next-Drupal takes advantage of the Drupal Subrequests module to greatly reduce the number of front-end API calls required. Additionally, we’ve made use of a CDN for image optimization.
Lessons Learned
The results from the tests were generally positive, showing performance gains over the existing site, and minimal impact to the Drupal admin interface during periods of high front-end load.
We were also able to perform some comparable tests against the existing site, confirming the effectiveness of various caching layers and CDNs. While these tests were worthwhile, performance gains could be on the scale of milliseconds, especially in protocol tests for well cached routes.
A solid baseline of back-end performance demonstrated by protocol tests leaves a lot of room for perceived front-end performance improvements, using approaches like client side preloading with the Next.js link component.
Conclusions
Based on these results, we’re confident we have developed an architecture that will handle the site’s expected load, provide baseline performance improvements, and create an ongoing path for front-end performance improvements with Next.js. We expect this to be one of our largest ever Next-Drupal sites, and are very excited for the upcoming launch.
We’re also excited for future performance improvements made possible by React Server Components and the Next.js App Router, which we now support in the Next.js for Drupal 2.0 beta.
If you’re using Next.js for Drupal, we’d love to hear more about your experience. If you have any questions or comments on your experience, we encourage you to contact us.
