After years and years fighting with and against JavaScript build stacks, I eventually gave a try to Next.js and fell in love with it for two simple reasons: it's barely opinionated, and it packages a simple and unique build configuration shared across back-end and front-end. But as it's not Express underneath the API routes, we have to find some workarounds to be able to build a real all-in-one application.
To call it a decent API, we need quite more than just routes handling. Standalone entry points are required for executing scripts and workers; chaining middlewares really helps keeping route security layers declaration succinct; and as most middlewares and router-dependent packages have been written for Express, we also need a way to integrate them seamlessly.
One solution would be using a custom Express server, but we'd go against the framework and lose its main advantage: Automatic Static Optimization. So let's try to use the built-in server, and address the issues one by one to make it all run smooth.
Issue 1: chaining middlewares
This one is a no-brainer. Just use next-connect! It emulates the next()
behavior of Express and gives us back our well-appreciated .use()
, .get()
, .post()
, .all()
etc. methods that takes away the need for the verbose in-route method checking (if (req.method === 'POST') { ... }
) that Next.js suggests on their documentation.
import nc from 'next-connect';
const handler = nc()
.use(someMiddleware())
.get((req, res) => {
res.send('Hello world');
})
.post((req, res) => {
res.json({ hello: 'world' });
});
export default handler;
Also, a very convenient feature is passing other next-connect instances to the .use()
method, and therefore predefine reusable handler middlewares:
// /path/to/handlers.js
import nc from 'next-connect';
import { acl, apiLimiter, bearerAuth } from '/path/to/middlewares';
export const baseHandler = () => nc({
// 404 error handler
onNoMatch: (req, res) => res.status(404).send({
message: `API route not found: ${req.url}`,
}),
// 500 error handler
onError: (err, req, res) => res.status(500).send({
message: `Unexpected error.`,
error: err.toString(),
}),
});
export const secureHandler = baseHandler()
.use(apiLimiter)
.use(bearerAuth)
.use(acl);
// /pages/api/index.js
import nc from 'next-connect';
import { secureHandler } from '/path/to/handlers';
const handler = nc()
.use(secureHandler) // benefits from all above middlewares
.get((req, res) => {
res.send('Hello world');
});
export default handler;
Issue 2: testing routes
Within the test environment, Next.js server is not running, forcing us to find a way to emulate both the request and its resolution. Supertest pairs really well with Express, but needs to run the server in order to pass the request to the handler through all its layers. That being said, it doesn't need to be Express.
So without adding any new dependency, we create a bare HTTP server with the native node http
lib, and manually apply the built-in resolver of Next.js, nicely packaged as a utility function, just like this:
import { createServer } from 'http';
import { apiResolver } from 'next/dist/next-server/server/api-utils';
import request from 'supertest';
export const testClient = (handler) => request(httpCreateServer(
async (req, res) => {
return apiResolver(req, res, undefined, handler);
},
));
In our test files, the only thing we need then is passing the handler to our client, with Supertest running as usual:
import { testClient } from '/path/to/testClient';
import handler from '/pages/api/index.js';
describe('/api', () => {
it('should deny access when not authenticated', async (done) => {
const request = testClient(handler);
const res = await request.get('/api');
expect(res.status).toBe(401);
expect(res.body.ok).toBeFalsy();
done();
});
});
That way we don't have anything to setup repeatedly for each route test. Pretty elegant.
Issue 3: custom entry points
Entry points are scripts that are meant to be run manually - usually background processes like a queue worker, or migration scripts. If set as standalone node processes, they won't inherit from the 'import' syntax built-in inside Next.js, neither the path aliases you may have setup. So basically, you'd have to manually rebuild the build stack of Next.js, polluting your package.json
with babel dependencies, and keep it up to date with Next.js releases. We don't want that.
To make it clean, we have to make these pipe through Next.js build. Adding custom entry points is not documented, though it seems to work with that solution, configuring next.config.js
:
const path = require('path');
module.exports = {
webpack(config, { isServer }) {
if (isServer) {
return {
...config,
entry() {
return config.entry().then((entry) => ({
...entry,
// your custom entry points
worker: path.resolve(process.cwd(), 'src/worker.js'),
run: path.resolve(process.cwd(), 'src/run.js'),
}));
}
};
}
return config;
},
};
Sadly the only thing it does is compiling these new JavaScript files through the internal webpack process and outputs them inside the build directory, as is. Since they're not tied to the server, all the features of Next.js are missing, including the only important one for this case: environment variables.
Next.js relies on dotenv, so it's already set as a dependency that we could reuse. Yet calling dotenv at the top of these entry points, for some reasons, won't propagate the environment variables to the imported modules:
// /.env
FOO='bar';
// /src/worker.js
import dotenv from 'dotenv';
dotenv.config();
import '/path/to/module';
console.log(process.env.FOO); // outputs 'bar';
// /src/path/to/module.js
console.log(process.env.FOO); // outputs 'undefined';
That is very annoying. Thankfully, it can be quickly solved by dotenv-cli, which actually resolves .env
files the same way than Next.js. We only need to prefix our script commands in package.json
:
"worker": "dotenv -c -- node .next/server/worker.js",
Note that it calls the script from the build folder. You need either to have next dev running, or previously have run next build. It's a small price to pay in regard of the benefits of keeping them within the Next.js build stack.
Issue 4: Express-based packages
Next-connect already makes some Express packages compatible out of the box, like express-validator that I'm used to when it comes to checking request parameters. That's because they simply are middleware functions.
Some of these functions rely on Express-specific properties, like express-acl. Usually they throw an exception when hitting that missing property, and digging a little bit the error and the package source will help you find it and fix it with a handler wrapper:
import acl from 'express-acl';
acl.config({
baseUrl: '/api',
filename: 'acl.json',
path: '/path/to/config/folder',
denyCallback: (res) => res.status(403).json({
ok: false,
message: 'You are not authorized to access this resource',
}),
});
export const aclMiddleware = (req, res, next) => {
req.originalUrl = req.url; // Express-specific property required by express-acl
return acl.authorize(req, res, next);
};
So the biggest challenge happens when the package deeply depends on Express because it creates router or app definitions. That's the case of monitoring interfaces like bull-board. When we can't find a standalone alternative, then our only chance is to find a way to emulate the whole Express application. Here's the hack:
import Queue from 'bull';
import { setQueues, BullAdapter, router } from 'bull-board';
import nc from 'next-connect';
setQueues([
new BullAdapter(new Queue('main')),
]);
// tell Express app to prefix all paths
router.use('/api/monitoring', router._router);
// Forward Next.js request to Express app
const handler = nc();
handler.use((req, res, next) => {
// manually execute Express route
return router._router.handle(req, res, next);
});
export default handler;
A few things to note here:
- This file should be located inside
/pages/api
because Next.js only recognize server-side routes under that folder. - For Express to handle all sub-routes declared by the package, we have to create a catch-all on Next.js route. That can be done naming our route file
/pages/api/monitoring/[[...path]].js
as specified in their docs (replace "monitoring" with whichever name you'd prefer). - In this specific case, bull-board exposes an entire Express instance under the confusing name router. That's why we're calling
router._router.handle()
to manually execute the route handler. If by reading the source you find out it's aexpress.Router
instance, call insteadrouter.handle()
directly. - We also need to tell Express that the base URL of its entire app is the route we're calling it from. Let's just define it with
app.use('/base/url', router)
as we would normally do. Just keep in mind the confusion betweenexpress
andexpress.Router
instances. - Finally, Express handles the response part as we're passing it the full Response object. No need for us to send headers on its behalf.
The reasons why I don't use this trick to forward the whole API to an emulated Express app is that I don't know how it'll affect performances, and most importantly, I'd rather respect Next.js natural patterns not to disorient other developers.
Not so bad, isn't it? We end up having a full-featured server with footprint-limited patches over the blind spots. I still wish Next.js could provide all these features in its core, but I'm happy we didn't denature it much either with these workarounds. Given the current state of JavaScript, Next.js may very well be the ultimate full-stack framework.
PS: I didn't go over setting up sessions and user authentication because with these issues now solved, you can virtually make everything work as usual. Though, I'd recommend looking into next-session or NextAuth.js.
Top comments (13)
Great article and nice clean code. I want to follow your approach but do not get it working with
withApiAuthRequired
(nextjs-auth0). Could you point me in the right direction? Keep getting the error 'invalid hook call'.Thanks a lot!
nc().use()
requires the function arg to be compatible with the props it receives. If you can't make it work out of the box, you may need to wrap it:Thanks for your help! Below a suggestion from my side to replace
const handler = nc().use(errorHandler)
withconst handler = router()
.Yeah my original code didn't work, that's a good way to put it. I just wouldn't call it router to avoid confusion, because it's just a handler, the router being managed inside Next core :).
Here is my strategy:
I searched for example how to test Next.js api with supertest and this is really useful.
The thing is
return apiResolver(req, res, undefined, handler);
3rd arg is query params so if you have a route/api/posts/:id
inside Next.js controllerreq.query.id
will not be passed and will be hardcodedundefined
. Do you know how to pass api dynamic route params?You're right, you can extract it from
req
with the help of theqs
(query string) package if it needs specific formatting.Have a look here: nodejs.org/en/knowledge/HTTP/clien... :)
Thank you for great article. Just quick note that this is not working for me now:
The only way for me is to specify
onError
andonNotFound
directly onnc
instance:Otherway the initial error callbacks are invoked.
Thanks! Indeed, my workaround here is to define a function that returns a new instance of a base handler:
Article updated :).
This is incredible. I just put together a tutorial on next-middleware, but this is so much easier. Thanks for sharing.
Yeah, next-connect really makes everything so much easier. It also handles async error handling out of the box. It's perfect :D. But nice article of yours though, you must have learned quite a few things in the making!