How to create a modular RESTfull API with Node.js, Express and ECMAScript 6 Part 1

In this 2 part blog post I will explain how to get started with building a RESTful API with Node.js, Express and ECMAScript 6 using a module approach.

Go to https://github.com/LuukMoret/blog-node-api-express-es6-part1 to see the finished code.

Node.js

Node.js is an asynchronous event driven JavaScript runtime, designed to build scalable network applications. Node.js can handle many connections concurrently in contrast to today’s more common concurrency model where OS threads are employed.

Thread-based networking is relatively inefficient and very difficult to use. Furthermore, users of Node are free from worries of dead-locking the process, since there are no locks. Almost no function in Node directly performs I/O, so the process never blocks. Because nothing blocks, scalable systems are very reasonable to develop in Node.

You can read more about Blocking vs Non-Blocking here.

Express

Express is a minimal and flexible Node.js web application framework that provides a robust set of features for web and mobile applications. There are alternatives like Koa and Hapi but Express is definitely the most widely used web framework for node.js today.

ECMAScript 6

ECMAScript is a subset of JavaScript. JavaScript is basically ECMAScript at it’s core but builds upon it. Languages such as ActionScript, JavaScript, JScript all use ECMAScript as its core.

ECMAScript 6, also known as ECMAScript 2015, is the latest version of the ECMAScript standard. ES6 is a significant update to the language, and the first update to the language since ES5 was standardized in 2009. Read more about these new features here.

There is also a fantastic JavaScript Style Guide from Airbnb. If you want to enforce these rules in your project, consider using the airbnb eslint npm package found here .

With the above technologies explained we can start writing our node api!

Prerequisites

• Node.js 6 or higher should be installed.
• Git should be installed.

Project setup

Start by creating a folder and cd into it:

mkdir mynodeapi && cd mynodeapi

Install the official express generator:

npm install express-generator -g

Run the express generator:

express -e --git

-e defaults to ejs engine instead of jade as (usually) won’t be needing this with an api. We will remove these files later.

--git will add a .gitignore file.

Install the necessary npm packages:

npm install

Create middleware folder and app folder:

mkdir middleware app

Delete routes folder, views and public folders:

rm -rf routes views public

Remove unused dependencies:

npm uninstall --save ejs

Install runtime dependencies

npm install --save compression cors helmet lodash longjohn nconf request request-debug request-promise winston

Short explanation of each dependency:

• compression - adds gzip compression.
• cors - adds cors support.
• helmet - Helmet helps you secure your Express apps by setting various HTTP headers. It’s not a silver bullet, but it can help!
• lodash - Lodash makes JavaScript easier by taking the hassle out of working with arrays, numbers, objects, strings, etc.
• nconf - Hierarchical node.js configuration with files, environment variables, command-line arguments, and atomic object merging.
• longjohn - Long stack traces for node.js with configurable call trace length.
• request-debug - an easy way to monitor HTTP(S) requests performed by the request module, and their responses from external servers.
• request-promise - adds a Bluebird-powered .then(…) method to Request call objects.
• winston - a simple and universal logging library with support for multiple transports.

Install compile time dependencies

npm install --save-dev chai eslint eslint-config-standard eslint-plugin-promise eslint-plugin-standard eslint-watch istanbul mocha nock nodemon sinon sinon-as-promised supertest

Short explanation of each dependency:

• chai - BDD/TDD assertion library for node.js and the browser. Test framework agnostic.
• eslint - a tool for identifying and reporting on patterns found in ECMAScript/JavaScript code.
• eslint-config-standard - module needed for eslint-watch
• eslint-plugin-promise - module needed for eslint-watch
• eslint-plugin-standard - module needed for eslint-watch
• eslint-watch - a simple command line tool that wraps Eslint. Eslint Watch provides file watching and command line improvements to the currently existing Eslint command line interface.
• istanbul - JS code coverage tool that computes statement, line, function and branch coverage with module loader hooks to transparently add coverage when running tests.
• mocha - simple, flexible, test framework
• nock - an HTTP mocking and expectations library for Node.js
• nodemon - nodemon will watch the files in the directory in which nodemon was started, and if any files change, nodemon will automatically estart your node application.
• sinon - standalone and test framework agnostic JavaScript test spies, stubs and mocks.
• sinon - as-promised -sinon with promises
• supertest - the motivation with this module is to provide a high-level abstraction for testing HTTP, while still allowing you to drop down to the lower-level API provided by super-agent.

Configure ESLint

ESLint is a pluggable linting utility for JavaScript. We can use this to enforce code styling rules.

Create the following files in the project directory to configure ESLint:
.eslintignore with the following content:

/public/**/*.js
/coverage/**/*.js

.eslintrc.yml with the following content:

extends: standard
plugins:
  - standard
parserOptions:
  ecmaversion: 6
env:
  node: true
  es6: true
  mocha: true
rules:
  semi:
    - error
    - always
  quotes:
    - error
    - single
  curly: error
  space-before-function-paren: off

You can adjust these options as needed.

Add NPM Tasks

Update the scripts section with the following npm tasks in the package.json file:

• start - start the application and watch for file changes
• test - kicks off build process
• build - run lint, tests and coverage
• lint - run lint
• lint:watch - run lint continuously and watch for file changes
• test:single - run tests
• unit-tests - run unit tests
• unit-tests:watch - run unit-tests continuously and watch for file changes
• integration-tests - run integration tests
• coverage - run coverage for all tests
• coverage-unit-tests - run coverage for unit tests
• coverage-integration-tests - run coverage for integration tests

"start": "npm install && nodemon ./bin/www",
"test": "npm run build",
"build": "npm install && npm run lint && npm run test:single && npm run coverage",
"lint": "esw .",
"lint:watch": "esw . -w",
"test:single": "npm run unit-tests && npm run integration-tests",
"unit-tests": "mocha --check-leaks --harmony app/**/*.spec.js",
"unit-tests:watch": "mocha --check-leaks --harmony app/**/*.spec.js -w",
"integration-tests": "mocha --check-leaks --harmony app/**/*.integration.js",
"coverage": "npm run coverage-unit-tests && npm run coverage-integration-tests",
"coverage-unit-tests": "istanbul cover --root app --dir ./coverage/unit -x **/**/*.integration.js -x **/**/*.spec.js ./node_modules/mocha/bin/_mocha -- --check-leaks --harmony --grep unit app/**/*.js",
"coverage-integration-tests": "istanbul cover --root app --dir ./coverage/integration -x **/**/*.integration.js -x **/**/*.spec.js ./node_modules/mocha/bin/_mocha -- --check-leaks --harmony --grep integration app/**/*.js"

npm start and npm test can be run as is, the other tasks need to include the run keyword.
npm run build for example.

Create additional files

Create config.json to store configurable properties with the following content:

{
  "application": {
    "port": 3100,
    "local": "true"
  },
  "logging": {
    "level": "info"
  }
}

Create middleware/configs.js to bootstrap nconf with the following content:

'use strict';
 
const winston = require('winston'); // https://www.npmjs.com/package/winston
const nconf = require('nconf'); // https://www.npmjs.com/package/nconf
const path = require('path'); // https://www.npmjs.com/package/path
 
exports.register = () => {
  let rootPath = path.join(__dirname, '../');
  let configFile = 'config.json';
 
  // 1. any overrides
  nconf.overrides({});
 
  // 2. `process.env`
  // 3. `process.argv`
  nconf.env()
    .argv();
 
  // 4. Values in `config.json`
  nconf.file(rootPath + configFile);
 
  // 5. Any default values
  nconf.defaults({
    application: {
      port: 3100
    }
  });
 
  // Log current configuration
  winston.info('app - config: logging: ', nconf.get('logging'));
  winston.info('app - config: config file loaded from: ', rootPath + configFile);
  winston.info('app - config: application:', nconf.get('application'));
 
  winston.info('app - config: nconf loaded');
};

Create middleware/handlers.js to configure default http handlers with the following content:

'use strict';
 
const winston = require('winston'); // https://www.npmjs.com/package/winston
 
exports.register = (app) => {
  registerDefaultHandler(app);
  winston.info('app - handlers: default handler loaded');
 
  registerNotFoundHandler(app);
  winston.info('app - handlers: not found handler loaded');
 
  registerErrorHandler(app);
  winston.info('app - handlers: error handler loaded');
};
 
function registerDefaultHandler(app) {
  app.get('/', (req, res) => {
    res.send('');
  });
}
 
function registerNotFoundHandler(app) {
  app.use((req, res, next) => {
    let err = new Error('Not Found');
    err.status = 404;
    next(err);
  });
}
 
/**
 * development error handler, will print stacktrace
 * production error handler, no stack traces leaked to user
 * @param app
 */
function registerErrorHandler(app) {
  if (app.get('env') === 'development') {
    app.use((err, req, res, next) => {
      res.status(err.status || 500)
        .send({message: err.message, error: err});
    });
  }
 
  app.use((err, req, res, next) => {
    res.status(err.status || 500)
      .send(err.message);
  });
}

Create middleware/routes.js, this will contain the routes of each module. Add the following content:

'use strict';
 
exports.register = (app) => {
 
};

Create middleware/utils.js with the following content:

'use strict';
 
const winston = require('winston'); // https://www.npmjs.com/package/winston
const morgan = require('morgan'); // https://www.npmjs.com/package/morgan
const cookieParser = require('cookie-parser'); // https://www.npmjs.com/package/cookie-parser
const bodyParser = require('body-parser'); // https://www.npmjs.com/package/body-parser
const cors = require('cors'); // https://www.npmjs.com/package/cors
const helmet = require('helmet'); // https://www.npmjs.com/package/helmet
const nconf = require('nconf'); // https://www.npmjs.com/package/nconf
const compression = require('compression'); // https://www.npmjs.com/package/compression
const rp = require('request-promise'); // https://www.npmjs.com/package/request-promise
 
exports.register = (app) => {
  let verboseLogging = (nconf.get('logging:level') === 'debug');
  let local = (nconf.get('application:local') === 'true');
  if (verboseLogging || local) {
    winston.info('app - utils: debug logging enabled loaded');
 
    app.use(morgan('dev'));
    try {
      winston.remove(winston.transports.Console);
    } catch (error) {
    }
    winston.add(winston.transports.Console, {'timestamp': true});
    winston.info('app - utils: morgan dev loaded');
 
    require('longjohn'); // https://www.npmjs.com/package/longjohn
    winston.info('app - utils: longjohn loaded');
 
    if (verboseLogging || !local) {
      require('request-debug')(rp); // https://www.npmjs.com/package/request-debug
      winston.info('app - utils: request-debug loaded');
    }
  } else {
    app.use(morgan('combined'));
    winston.info('app - utils: morgan combined loaded');
  }
 
  app.use(compression());
  winston.info('app - utils: gzip compression loaded');
 
  app.use(cookieParser());
  winston.info('app - utils: cookieparser loaded');
 
  app.use(bodyParser.json());
  app.use(bodyParser.urlencoded({extended: false}));
  winston.info('app - utils: bodyparser loaded');
 
  app.use(cors());
  winston.info('app - utils: cors loaded');
 
  app.options('*', cors());
  winston.info('app - utils: cors preflight loaded');
   
  app.use(helmet());
  winston.info('app - utils: helmet loaded');
};

Update the app.js file, which will bootstrap the middleware components. Replace the entire file with the following content:

'use strict';
 
const winston = require('winston');
const nconf = require('nconf');
 
winston.info('app: configs loading');
const configs = require('./middleware/configs');
configs.register();
winston.info('app: configs loaded');
 
winston.info('app loading');
 
winston.info('app: express loading');
const express = require('express');
let app = express();
winston.info('app: express loaded');
 
winston.info('app: utils loading');
const utils = require('./middleware/utils');
utils.register(app);
winston.info('app: utils loaded');
 
winston.info('app: routes loading');
const routes = require('./middleware/routes');
routes.register(app);
winston.info('app: routes loaded');
 
winston.info('app: handlers loading');
const handlers = require('./middleware/handlers');
handlers.register(app);
winston.info('app: handlers loaded');
 
winston.info('app loaded');
 
winston.level = nconf.get('logging:level');
 
module.exports = app;

Update the bin/www file, the entry point of the application. Replace the entire file with the following content:

#!/usr/bin/env node
'use strict';
 
/**
 * Module dependencies.
 */
 
const nconf = require('nconf');
const app = require('../app');
const debug = require('debug')('source:server');
const http = require('http');
const winston = require('winston');
 
winston.info('http loading');
const port = normalizePort(nconf.get('application:port'));
const server = http.createServer(app);
app.set('port', port);
server.listen(port);
server.on('error', onError);
server.on('listening', onListening);
winston.info('http loaded');
 
winston.info('node application started and listening on port', port);
 
/**
 * Normalize a port into a number, string, or false.
 */
function normalizePort(val) {
  let port = parseInt(val, 10);
 
  if (isNaN(port)) {
    // named pipe
    return val;
  }
 
  if (port >= 0) {
    // port number
    return port;
  }
 
  return false;
}
 
/**
 * Event listener for HTTP server "error" event.
 */
function onError(error) {
  if (error.syscall !== 'listen') {
    throw error;
  }
 
  let bind = typeof port === 'string'
    ? 'Pipe ' + port
    : 'Port ' + port;
 
  // handle specific listen errors with friendly messages
  switch (error.code) {
    case 'EACCES':
      console.error(bind + ' requires elevated privileges');
      process.exit(1);
      break;
    case 'EADDRINUSE':
      console.error(bind + ' is already in use');
      process.exit(1);
      break;
    default:
      throw error;
  }
}
 
/**
 * Event listener for HTTP server "listening" event.
 */
function onListening() {
  let addr = server.address();
  let bind = typeof addr === 'string'
    ? 'pipe ' + addr
    : 'port ' + addr.port;
  debug('Listening on ' + bind);
}
 
module.exports = server;

Run npm start to start the application. You should be able to navigate to http://localhost:3100/ now (which will serve an empty page).

That’s it! The project setup is now done! We will discuss writing modules in part 2.