code-for-a-living October 7, 2020

Static site generation with single page app functionality? That’s what’s coming Next(.js)

We spoke with Vercel CEO Guillermo Rauch and Next.js development leader Tim Neutkens about what new features this version brings, what’s next for Next.js, and how their approach helps the faster growing cohort of web users stay online.

React has become an immensely popular front-end framework since it was first released in 2013. In our 2020 Developer Survey, it ranked as both the second most popular and the second most loved web framework. But it primarily produces single-page applications (SPAs) and can burden every page with every dependency. Developers looking for simpler multi-page setups have turned to static site generators (SSGs), which take markdown (often with other templating languages) and build a full site structure that can be deployed to a web server. Along comes Next.js.

It’s a hybrid of the two styles, allowing you to prerender pages (or even components on pages) that still include all the client-side computing that React gives a site. With version 9.5, Vercel is making Next even more versatile, allowing partial builds to only update some changed pages, easy incremental adoption of Next.js, and a slew of developer-friendly updates. 

I spoke with Vercel CEO Guillermo Rauch and Next.js development leader Tim Neutkens about what new features this version brings, what’s next for Next.js, and how their approach helps the faster growing cohort of web users stay online. 

If you’re interested in hearing more, the Next.js user conference is happening on October 27th. 

Note: this conversation has been edited for clarity and length. 

Ryan: All right. so, for folks that don’t know, can you give me a little overview of what Next.js is?

Tim: It’s a web framework that allows you to leverage a technology that Facebook open sourced quite a few years ago called React to build websites and applications. React itself is a view layer, so it allows you to write components and have those interact together, but it doesn’t cover all this different stuff that you have to set up in order to go to production. Things like server-side rendering, compilation, transpilation, getting modern code to run in older browsers—polyfilling, optimizing all that. This is really what Next helps with. It gives you a set of defaults that work really well. 

Ryan: Your website talks about it as middle ground between a single site generator and a fully rendered server side, dynamic system. What’s the sweet spot for you on that?

Tim: Next.js allows you to pre-render pages. It creates HTML on a server at build time with static site generation or uses run-time rendering on the server side. Next allows you to do a hybrid of those. Unlike most other frameworks, you are not bound by, oh, I’m going to build my app completely statically generated. Instead, you’re allowed to have some pages be server-side rendered and some pages be statically generated.

In the new release we make it possible to update these statically generated pages without having to run a new build, rebuilding your whole app.

Ryan: Right. That’s a good lead into the new stuff. How do you render everything without building the entire web web platform or website again?

Guillermo: When you use the static site generator, you’re creating a bunch of HTML files at build time. But you can technically do the same also with server side rendering, right? A lot of people server-side render and then they cache, so the technology is similar. The metaphor that I like to use is that static generation is running server-side rendering at build time. You’re doing it a little bit before the website goes live.

Next.js shares the same underlying technology and infrastructure to do static generation and server rendering. The developer can choose to pre-render their most important pages at build time or defer some to build just in time at runtime.

If non-technical folks just want to make a change in the content of a certain page with Next.js, they have the ability to preview and then submit that change.We’re regenerating one page at a time instead of the entire site.

For our customers, like the Washington Post, they have hundreds of thousands of stories. The ability to dynamically change and update pages becomes the difference between a functional site and a non-functional site.

Ryan: For dynamic pages like on the Washington Post, they have a lot of changing content on the sides and bottoms of the pages. Does that change automatically on the fly? Does it get pre-rendered regularly?

Guillermo: One really cool benefit about Next.js is that in addition to having rendered either at build time or at the edge—just in time for server side rendering—you still have the full power of React on the client side.

Next.js allows you to segregate and put computation everywhere. You can put computation on the device when you’re interacting with animations and hover effects and clicks and prefetching and all that. You can put computation on the edge for rendering just in time.

You can put computation at build time. You can do it before or any traffic hits your website. For  example, to dynamically render the weather of San Francisco when someone visits the Washington Post. Obviously, you’re not going to pre-render the weather in San Francisco for every single visitor, especially if they’re visiting like Ryan from New York.

So you have the full power of React and React hooks, React lifecycle, React state to add dynamism on the client.

Ryan: The rendering on the client side—is that new for Next.js?

Guillermo: Client-side has always been the norm. When react first came out, people were using it strictly for single page applications. That’s actually what motivated us to create Next.js. We’re like, you’re going to lose SEO. You’re going to lose performance because now you’re rendering everything on the device. You could have rendered some ahead of time and then shared it with everybody else. 

We’re also worried about emerging markets where JavaScript processing capability on the device is not the same as my iPhone. Rendering all the stuff on the client’s side was very harmful for it. This is the fastest growing cohort of internet users in the world. 

Next.js came out to do server rendering first. But then we realized that a very interesting cohort of users that was emerging in the world, particularly around the Jamstack ecosystem. They wanted to also have fully static pages, full stop. They didn’t want to hit a server. It’s just pure HTML that has been pre-computed. The idea of a hybrid came up because we realized we shouldn’t have to choose between one rendering model and the other. Sometimes it comes down to the specific page. Then in 9.5, we added the ability to—at run time, once you’ve deployed your site—add new static pages or refresh them. We call that incremental static generation.

Ryan: It sounds like you’re allowing a lot of generation points for various pages. What else is 9.5 bringing?

Tim: We tried to focus on solving a lot of common issues that people had. Some are seemingly small changes that have a really high impact, like redirecting trailing slashes automatically, then handling that behavior for if you want to have trailing slashes in your URLs.

Another is allowing you to set a base path. Previously, it was possible to host a Next app on a sub path of a domain. Say you want to adopt Next in only your documentation initially. You could host it on /docs, but it was hard to then have all these different pieces work together where the server has to route /docs, the client side has to handle the /docs prefix and all that. All the links had to be prefixed /docs. You’d have random 404s when you forgot to prefix it. 

Now in 9.5, you get this base path option where if you set it to /docs, it’s going to automatically serve the right assets on /docs and then prefix all the URLs automatically. You don’t have to think about, Hey, I want to change /docs to some other URL because we’re moving the site. You don’t have to think about all these different prefixes or where to set them up.

Guillermo: This is part of a broader initiative. Another feature, which we call rewrites, is all about incremental adoption of Next.js. Since the initial release, we’ve seen sites of all sizes adopt the technology, but now we’ve gotten to the point where we’re seeing a lot of interest in migrating to the stack from either a React application or a homegrown framework that they’re tired of maintaining. 

Most teams don’t want to do the full rewrite overnight. The industry is known to be very risk averse, wary of adopting technology. You can start one page at a time or one subtree of pages at a time. In the example that Tim gave, we decide that for a certain section of my website, we replace it entirely with a new Next.js app. I’m going to host `/docs` and this sub tree of pages.

Another way that you can do that is with the rewrite option that we added where Next.js will handle the pages that you’ve defined in the pages directory. For example, say I’ve defined `about.js`, meaning that I’m registered as the `/about` route. Then you can say for everything else, go to the legacy stack.

This can implement the fig strangler pattern, where you keep adding pages to Next.js. The visitor comes to Next.js first, if this page exists in the Next.js directory, we load it. Otherwise the visitor goes to the legacy stack. You’re giving teams an incremental and collaborative way of adopting modern technology, while de-risking the conversion. It shows the changes side by side and measures the client side and server side performance of the two stacks. In a lot of cases, we see really good results. Eventually, a lot of apps become a single Next.js app.

Ryan: It’s an easy way for them to stick their toe in the water without risks. As you said, React is mostly for single page applications. What did you have to do to make Next.js not a single page application framework?

Tim: When you start building a Next.js application, you start with the pages directory. Every value you create there is going to be a different route.This could be a set in stone route, like `about.js` that maps to `/about` or it could be a dynamic route.

If you want to do dynamic segments, it allows you to do that as well. That might sound like a pretty simple thing, but it allows us to create a multi-page architecture where every page itself is almost a sub-app inside of the React app. That means every single page is going to be a separate bundle; it’s code split by default.

If you’re importing something really large, like a one megabyte NPM library, it’s not going to affect all the other pages in the application. When you build a React app, you have to be fairly aware of what you’re importing and how it’s going to affect the rest of the application. You’re going to ship the code for every single route in the application in the initial page load. Say you open a website that is stock React, but built using a custom router. It’s going to send the dashboard plus all the settings pages and all that. But you don’t want that for the initial marketing page, right? 

Guillermo: Developers have to be very aware of the amount of JavaScript that they ship to the client. They’re sort of paranoid because the more JavaScript they ship, the slower they make some of the most important web vitals, like time to interactive with a page for the average user. When they press the buy button on a store item, is that going to actually work and respond quickly? If I’m clicking on an ad that takes me to Lululemon pants—I use Lululemon’s as an example, because they use Next.js—then I want to only load the code that I need to render those pants. 

In this case, the rendering of the pants has already been done in cache at the edge. So all the structure, the page, the gallery, the carousel, the price, most of that is coming from pre-rendered pages. Some other personalization things could happen at the last mile, but for the most part, a lot of the structure of the page is pre-rendered. When I press buy, that has to be super fast because otherwise, I’m not selling the pants. Next.js did a lot automatically for that team because we gave them this architecture of multiple page entry points. The framework knew how to code split the JavaScript for the client side in little bundles and extract common bundles from those automatically. 

From a rendering pipeline perspective, it’s not just the code splitting, but it’s also that again, we can render code that React would have run in the client. We can execute it at the edge.

Ryan: So you’re able to build a lot of little pre-rendered building blocks and assemble a page bit by bit. What other new features does 9.5 have?

Tim: One of the things that we didn’t cover yet is that we improved our fast refresh core. React fast refresh allows you to preserve states while making edits and does so reliably. What that means is, in React, you have this state that you can keep inside of a component. Say you have a tab bar, where you click on different tabs. State will be preserved inside of the component. What fast refresh allows you to do is when you click on this tab, you make some code changes, and it’s going to update in place, but it’s not going to reset that component to its initial state.

That tab stays open when you’re editing. Fast refresh covers development time improvements. When we update, we make these changes in place, so you’re not refreshing your browser all the time. But what if you’re introducing some kind of error? You make a typo somewhere in your application and suddenly it crashes. 

In 9.5, you get this really nice message saying, this is the exact line where you messed up. We made some extra improvements to help developers. This is an ongoing effort as well. Any errors that we find that are not helpful to engineers and developers, we’re improving those.

Guillermo: This is a hard problem in the JavaScript world with the language being so dynamic. Developers are always adding languages on top of vanilla JS because a lot of people want a new syntax that compiles down and is supported by browsers, or they add TypeScript on top. Then on top of TypeScript, you have all the syntax that is unique to React, namely JSX, which allows you to embed tags that look like HTML inside of your JS file. The language itself has been super dynamic and versatile. Errors being accurate has actually been a rare thing in this world.

Take code location—do you have a stack trace that makes sense? Funny enough, it’s still an ongoing effort together with the React team. Providing better and better stack traces for its template component stack traces is a feature that’s coming, so you can understand exactly where in the rendering pipeline a certain error happens.

What’s really neat—this is where the strength of this ecosystem lies—is this fast refresh technology is extremely fast. When you save, you see your change on your web browser in a matter of a hundred milliseconds or so. This provides a level of flow for the developer and a connectedness between what they’re thinking about and what they’re working on with their ability to continue to iterate.

It’s pretty much unmatched by any other stack. Swift UI is beginning to make some progress in this direction. But React has given us something that we really never had before. People used to do what’s called live reload. When it would detect a change, it would refresh the entire page, equivalent to pressing a F5 or the refresh button. React is now aware of what state is changing and what leaves of the graph of components are changing. Fast refresh technology is making the code update happen in the web browser very granularly. Because it’s so granular, we have done very little work. We’re trending towards this world where we’re talking about even going lower than a hundred milliseconds. We can show your changes in 10, 15, 20 milliseconds. It all comes down to making the patch to the screen as the developer makes changes on their local device as fast as possible and as granular as possible.

Ryan: That’s a lot of new stuff. Anything else worth talking about? 

Guillermo: For sure. We’ve added experimental support for Webpack 5. Tim has been working on this, where not only are we improving the developer experience—how quickly you update something on your local device—but what happens when you push, and the CI/CD layer starts building the site? 

Next.js for the most part has abstracted out the bundling technology. You never had to configure Webpack. Until now, Next.js was using Webpack 4, but we were able to upgrade it for the majority of our users to Webpack 5. It’s an experimental flag for now, but it has tremendous performance benefits that Tim can talk about.

We’re extremely excited that it will change the performance of builds of Next.js.

Tim: Webpack 5 is currently in beta. It’s been in beta for over a year, maybe even two years, in continuous development. The new version has a really large surface area because it aims to solve a lot of inconsistencies in the current stable version of Webpack.

Currently in Webpack, a build is not deterministic. That means if you take the same exact code base and run Webpack twice, it’s not going to output the same exact JavaScript bundles. That’s generally not a problem if you’re going to recompute every time, but it becomes a problem when you want to cache something. But I want them to only recompute the bits that were changed in my code. This is where the deterministic output makes a really big difference. You can’t cache it onto non-deterministic output, because the output would change with every build.

Webpack 5 is completely deterministic in its output, and it adds this caching layer that was previously missing. Effectively, you can get up to 25 times faster builds, depending on your application. On larger apps, it’s going to have a really big impact.

If you have many different pages in your Next.js app, all of those pages have to be compiled. Currently, every time you run a build, it’s going to cache some of the work, but not everything. When Webpack 5 is enabled, you can run builds much quicker in your CI environment and in development.

In development, this is going to help tremendously with developer experience. When you boot up your Next.js instance, it’s going to be much faster to get started. All the work that you did yesterday is still cached.

Your deployment of a Next.js app is going to be significantly faster. It only has to recompute the bits that were changed. Generally, you’re not making changes to every single page in your application. You’re changing only a few pages or one page, even just fixing a typo. In those cases, you want builds to be fast. It’s going to be released very soon, so we’re really happy about it.

Ryan:  Do you want to tease some of the interesting things coming in the future?

Tim:. We have a Next.js user conference planned  on the 27th of October, which currently has more than 50,000 signups already. We’re going to announce a lot of really cool new features, at the conference as well.

Tags: ,
Podcast logo The Stack Overflow Podcast is a weekly conversation about working in software development, learning to code, and the art and culture of computer programming.

Related

newsletter October 16, 2020

The Overflow #43: Simulated keyboards

Welcome to ISSUE #43 of the Overflow! This newsletter is by developers, for developers, written and curated by the Stack Overflow team and Cassidy Williams at Netlify. This week, we’re coding for the long game, learning the difference between cats and not cats, and translating our favorite JRPGs into English. From the blog Play the long game when…
code-for-a-living June 23, 2020

Is it time to give Drupal another look?

For many people discussion of content management systems raises unpleasant specters of the early 2000s. But while CMS platforms may not feel like the shiniest new tech on the block, they still have a lot to offer, and they've evolved in ways that might surprise you. Let's talk about Drupal, a 20 year old open source project that still manages to be on the leading edge of the CMS world.
code-for-a-living April 24, 2020

The Overflow #18: Changing traffic patterns

April 2020 Welcome to ISSUE #18 of The Overflow, a newsletter by developers, for developers, written and curated by the Stack Overflow team and Cassidy Williams at Netlify. This week, stay in the Loop with our feedback frameworks, remembering John Conway’s achievements, and the CEO who moved to the ER. From the blog The Loop:…