Analysis of Node.js - Koa source code

[mominger]

The upper part is the English version, and the lower part is the Chinese version, with the same content. If there are any wrong, or you have anything hard to understand, pls feel free to let me know.many thx.

koa2 is a quite small and expressive Node.js web framework
koa office websit
koa source code

• 1 How to use
• 2 Directory Structure
• 3 Key source code analysis
  • 3.1 application.js
    • 3.1.1 constructor()
    • 3.1.2 listen()
    • 3.1.3 callback()
    • 3.1.3.1 Analyse koa-compose
    • 3.1.4 handleRequest()
    • 3.1.5 respond()
    • 3.1.6 use()
    • 3.1.7 onerror()
    • 3.1.8 Other
  • 3.2 context.js
    • 3.2.1 onerror()
    • 3.2.2 delegate response and request
      • 3.2.2.1 Analyse delegates
      • 3.2.2.1.1 delegate.method()
      • 3.2.2.1.2 delegate.access()
  • 3.2 request.js
  • 3.3 response.js
  • 3.4 Summarize
• 4 Common middleware analysis
  • 4.1 koa-router
  • 4.1.1 The http methods it supports
  • 4.1.2 Invoke http method
  • 4.1.3 router.routes()
  • 4.1.4 router.allowedMethods()
  • 4.1.4 Summarize
  • 4.2 koa-bodyparser
  • 4.2 koa-static
  • 4.3 koa-cors

1 How to use

  const Koa = require('koa’);

  const app = new Koa();

  //add middleware
  app.use(async ctx => {
    ctx.body = 'Hello World';
  });

  //launch a http server
  app.listen(3000);

2 Directory Structure

The entrance "main": "lib/application.js”
As below is an analysis of the source code under lib

3 Key source code analysis

3.1 application.js

source code

Skeleton

const http = require('http');
const Emitter = require('events');

class Koa extends Emitter {
  ...
  constructor() {
    super();
  }
  
  listen() {}

  toJSON() {}

  inspect() {}
  
  use() {}
  
  callback() {}
  
  handleRequest() {}
  
  createContext() {}

  onerror() {}
}

function respond(ctx) {}

module.exports = Koa;

3.1.1 constructor()

constructor(options) {
  super();
  options = options || {};
  ...

  //define middlewares queue
  this.middleware = [];

  //inherit: ctx.__proto__.__proto__ = context
  this.context = Object.create(context);
  this.request = Object.create(request);
  this.response = Object.create(response);
  ...
}

Initialize 3 items: options, middleware queue, independent context/request/response objects

3.1.2 listen()

listen(...args) {
  ...
  const server = http.createServer(this.callback());
  return server.listen(...args);
}

Launch a http servie by Node

3.1.3 callback()

callback() {
  const fn = compose(this.middleware);
  ...
  const handleRequest = (req, res) => {
    const ctx = this.createContext(req, res);
    return this.handleRequest(ctx, fn);
  };

  return handleRequest;
}

Use koa-compose to process the middleware array and get a recursively called function fn(fnMiddleware)
Initialize a new ctx instance by createContext() for the current request
return an http service callback function handleRequest

3.1.3.1 Analyse koa-compose

source code

function compose (middleware) {
  ...
  return function (context, next) {
    // last called middleware #
    let index = -1
    return dispatch(0)
    function dispatch (i) {
      if (i <= index) return Promise.reject(new Error('next() called multiple times'))
      index = i
      let fn = middleware[i]
      if (i === middleware.length) fn = next
      if (!fn) return Promise.resolve()
      try {
        return Promise.resolve(fn(context, dispatch.bind(null, i + 1)))
      } catch (err) {
        return Promise.reject(err)
      }
    }
  }
}

context is the ctx parameter in the middleware function, next is the next middleware function
dispatch.bind(null, i + 1) means the next middleware function, use bind to point this to null, which is next in the middleware function

3.1.4 handleRequest()

 handleRequest(ctx, fnMiddleware) {
    const res = ctx.res;
    res.statusCode = 404;
    const onerror = err => ctx.onerror(err);
    const handleResponse = () => respond(ctx);
    onFinished(res, onerror);
    return fnMiddleware(ctx).then(handleResponse).catch(onerror);
  }

After the middleware is processed, invoke handleResponse, if has an error, invoke ctx.onerror
onFinished is used to invoke the onerror method after the end of the monitoring res stream, similar to on('finish') or on('error').

3.1.5 respond()

function respond(ctx) {
  ...

  // responses
  if (Buffer.isBuffer(body)) return res.end(body);
  if ('string' === typeof body) return res.end(body);
  if (body instanceof Stream) return body.pipe(res);

  // body: json
  body = JSON.stringify(body);
  if (!res.headersSent) {
    ctx.length = Buffer.byteLength(body);
  }
  res.end(body);
}

According to the data type of response data, pass it to res.end() after processing

3.1.6 use()

 use(fn) {
    ...
    this.middleware.push(fn);
    return this;
  }

Add into the middlewares queue

3.1.7 onerror()

onerror(err) {
  ...

  const msg = err.stack || err.toString();
  console.error(`\n${msg.replace(/^/gm, '  ')}\n`);
}

Print exception messages
Only synchronous exceptions can be caught here. Asynchronous exceptions in middleware need to be caught by process.on("unhandledRejection",()=>{}), or caught with try..catch asynchronous codes handled by async/await
Note: in the callback method, this line if (!this.listenerCount('error')) this.on('error', this.onerror); means that invoke the onerror method only if we do not use the on('error') form to listen exceptions

3.1.8 Other

inspect invoke toJSON method, toJSON prints object information

3.2 context.js

source code
Skeleton

const delegate = require('delegates');

const proto = module.exports = {
    ...
    onerror(err) {}
    ...
    delegate(proto, 'response')
    ...
   
    delegate(proto, 'request')
    ...

}

3.2.1 onerror()

onerror(err) {
    …
    this.app.emit('error', err, this);
    …
    
    this.status = err.status = statusCode;
    res.end(msg);
}

Throw error to trigger to execute application.onerror, and finally call res.end to respond to client

3.2.2 delegate response and request

delegate(proto, 'response')
.method('attachment')
.method('redirect’)
...

delegate(proto, 'request')
.method('acceptsLanguages')
.method('acceptsEncodings')
.method('acceptsCharsets')
...

When calling properties or methods of ctx, delegate to response or request
For example, ctx.attachment() is equivalent to response.attachment(), ctx.acceptsLanguages() is equivalent to request.acceptsLanguages()
proto is the context object

3.2.2.1 Analyse delegates

source code

3.2.2.1.1 delegate.method()

Delegator.prototype.method = function(name){
  var proto = this.proto;
  var target = this.target;
  this.methods.push(name);

  proto[name] = function(){
    return this[target][name].apply(this[target], arguments);
  };

  return this;
};

Make the functions of the target can also be called by proto

3.2.2.1.2 delegate.access()

Delegator.prototype.access = function(name){
  return this.getter(name).setter(name);
};

Delegator.prototype.getter = function(name){
  ...

  proto.__defineGetter__(name, function(){
    return this[target][name];
  });

  return this;
};

Delegator.prototype.setter = function(name){
  ...

  proto.__defineSetter__(name, function(val){
    return this[target][name] = val;
  });

  return this;
};

Hijack proto's get by defineGetter and access target instead
Also can be done it by Object.defineProroty or Proxy, defineGetter, defineSetter are obsolete api

3.2 request.js

source code

Wraped properties of the origin request object and extended some properties or methods
Such as

...
get header() {
   return this.req.headers;
},
...

Recommended to use ctx.headers in the code
Invoking chain: ctx.headers -> request.headers -> req.headers

3.3 response.js

source code

Wraped properties of the origin response object and extended some properties or methods
Such as

...
get status() {
    return this.res.statusCode;
},
...

Recommended to use ctx.status in the code
Invoking chain: ctx.status -> response.status -> res.statusCode

3.4 Summarize

application.js exports the Koa instance class. It inherits EventEmitter, which is convenient to monitor exceptions by the form of on('error')
application.js: listen() launchs an http sever; use() adds the middleware function into the middleware queue; callback() is used as the callback function of the http service, specifically iterating the middlewares queue to generate a recursive function fnMiddleware, and after executing fnMiddleware use handleRequest to handle some general response and exceptions
context.js: proxy for properties and methods of request.js, response.js
request.js: encapsulates the properties of the http request
response.js: encapsulates the properties of the http response

4 Common middleware analysis

It is difficult to build a web system in MVC mode with koa alone, so need some middleware with
As below is an analysis of commonly used middleware

4.1 koa-router

source code
How to use

const Koa = require( 'koa' );
const Router = require( 'koa-router' );
const router = new Router();
const app = new Koa();

router.get( '/', ( ctx, next ) => {
    ...
} ).post('/test',( ctx, next ) => {
    ...
} )


app.use( router.routes() )
    .use( router.allowedMethods() );

4.1.1 The http methods it supports

Actually it supports dozens of http methods in the methods package

4.1.2 Invoke http method

Finally, return the 'this' object, so it supports chained calls

register()

Invoke router.[http method] such as router.get(path, callback), mainly to create a Layer Object and then add it into the stack queue

Layer
source code

4.1.3 router.routes()

match()

routes()

layer.stack is an array of route middleware, so need to execute memo.concat(layer.stack)

4.1.4 router.allowedMethods()

allowedMethods() does some common finishing works after an error in the response, i.e. no status code or 404

4.1.4 Summarize

Wrap the route into a Layer object and register it in the stack queue of Layer object by router.method. router.routes() will return a dispatch function that invoked the chain of koa middlers. When the user requests a url from client, the dispatch function will be executed, it will iterate the paths of all Layer objects in the stack to compare ctx.path. Then get the matching Layer.stack (middlewares of each route) and execute them in order.
Since all routes will be iterated, if there are many routes, such as thousands or more, you may need to consider optimization it

4.2 koa-bodyparser

source code

parse.json()

json: Determine if the Content-Type is json such as application/json, convert the data of the request stream into a string, after parsing it with JSON.parse, then assign it to the ctx.body

form: After determining that it is "application/x-www-form-urlencoded" type, convert a string like 'a=1&b=1' into a json object by qs package, and then assign it to ctx.body
text/plain or text/xml: assigned to ctx.body string value
default ctx.body = {}
If you need to support multipart/form-data (upload files), you need to use koa-multer additionally, or use koa-body instead of koa-bodayparser

4.2 koa-static

source code
How to use

  const static = require('koa-static')

  app.use(static('public')

koa-send
source code

koa-static parsed the path parameter to the absolute path by path.resolve and passes it to koa-send. koa-send creates a readable stream by file.createReadStream and assigns it to ctx.body. Finally, After processing by the line of code in koa's respond method 'if (body instanceof Stream) return body.pipe(res) ' , the response is sent to the client
Sometimes assert(condition, error) instead of if is a good choice, such as the line: assert(root, 'root directory is required to serve files')

4.3 koa-cors

source code

Not OPTIONS request

OPTIONS request

Set some CORS-related response headers. It is recommended to set the cache time max-age for OPTIONS requests.

The following is the Chinese version, the same content as above

koa2是一个非常精简的Node.js web框架.
koa官网
koa源码

• 1 如何使用
• 2 目录结构
• 3 关键源码分析
  • 3.1 application.js
    • 3.1.1 constructor()
    • 3.1.2 listen()
    • 3.1.3 callback()
    • 3.1.3.1 分析koa-compose
    • 3.1.4 handleRequest()
    • 3.1.5 respond()
    • 3.1.6 use()
    • 3.1.7 onerror()
    • 3.1.8 其他
  • 3.2 context.js
    • 3.2.1 onerror()
    • 3.2.2 delegate response and request
      • 3.2.2.1 分析delegates
      • 3.2.2.1.1 delegate.method()
      • 3.2.2.1.2 delegate.access()
  • 3.2 request.js
  • 3.3 response.js
  • 3.4 总结
• 4 常见中间件分析
  • 4.1 koa-router
  • 4.1.1 它支持的method
  • 4.1.2 Http method 调用
  • 4.1.3 router.routes()
  • 4.1.4 router.allowedMethods()
  • 4.1.4 总结
  • 4.2 koa-bodyparser
  • 4.2 koa-static
  • 4.3 koa-cors

1 如何使用

  const Koa = require('koa’);

  const app = new Koa();

  //add middleware
  app.use(async ctx => {
    ctx.body = 'Hello World';
  });

  //launch a http server
  app.listen(3000);

2 目录结构

入口 "main": "lib/application.js”
下面对lib下的源码进行分析

3 关键源码分析

3.1 application.js

源码

骨架

const http = require('http');
const Emitter = require('events');

class Koa extends Emitter {
  ...
  constructor() {
    super();
  }
  
  listen() {}

  toJSON() {}

  inspect() {}
  
  use() {}
  
  callback() {}
  
  handleRequest() {}
  
  createContext() {}

  onerror() {}
}

function respond(ctx) {}

module.exports = Koa;

3.1.1 constructor()

constructor(options) {
  super();
  options = options || {};
  ...

  //定义中间件队列
  this.middleware = [];

  //继承: ctx.__proto__.__proto__ = context
  this.context = Object.create(context);
  this.request = Object.create(request);
  this.response = Object.create(response);
  ...
}

初始化3项: 配置项，中间件队列，独立的context、request、response对象

3.1.2 listen()

listen(...args) {
  ...
  const server = http.createServer(this.callback());
  return server.listen(...args);
}

node 起一个http服务

3.1.3 callback()

callback() {
  const fn = compose(this.middleware);
  ...
  const handleRequest = (req, res) => {
    const ctx = this.createContext(req, res);
    return this.handleRequest(ctx, fn);
  };

  return handleRequest;
}

使用compose 处理中间件数组，获得一个递归调用的函数fn(fnMiddleware)
为当前请求通过 createContext 初始化一个全新的 ctx 实例
返回一个 http 服务回调函数 handleRequest

3.1.3.1 分析koa-compose

源码

function compose (middleware) {
  ...
  return function (context, next) {
    // last called middleware #
    let index = -1
    return dispatch(0)
    function dispatch (i) {
      if (i <= index) return Promise.reject(new Error('next() called multiple times'))
      index = i
      let fn = middleware[i]
      if (i === middleware.length) fn = next
      if (!fn) return Promise.resolve()
      try {
        return Promise.resolve(fn(context, dispatch.bind(null, i + 1)))
      } catch (err) {
        return Promise.reject(err)
      }
    }
  }
}

context 就是中间件函数里的ctx, next就是下一个中间件函数
dispatch.bind(null, i + 1) 下一个中间件函数，用bind把this指向null，也就是中间件函数里的next

3.1.4 handleRequest()

 handleRequest(ctx, fnMiddleware) {
    const res = ctx.res;
    res.statusCode = 404;
    const onerror = err => ctx.onerror(err);
    const handleResponse = () => respond(ctx);
    onFinished(res, onerror);
    return fnMiddleware(ctx).then(handleResponse).catch(onerror);
  }

中间件处理完后，执行handleResponse，有错误会执行ctx.onerror
onFinished 是监听res流结束后，类似on('finish') 或 on('error')后，执行onerror方法

3.1.5 respond()

function respond(ctx) {
  ...

  // responses
  if (Buffer.isBuffer(body)) return res.end(body);
  if ('string' === typeof body) return res.end(body);
  if (body instanceof Stream) return body.pipe(res);

  // body: json
  body = JSON.stringify(body);
  if (!res.headersSent) {
    ctx.length = Buffer.byteLength(body);
  }
  res.end(body);
}

根据响应数据的类型，进行处理后再给 res.end()

3.1.6 use()

 use(fn) {
    ...
    this.middleware.push(fn);
    return this;
  }

加入 middlewares 队列

3.1.7 onerror()

onerror(err) {
  ...

  const msg = err.stack || err.toString();
  console.error(`\n${msg.replace(/^/gm, '  ')}\n`);
}

打印错误信息
这里只能捕获同步异常，中间件内的异步错误需要 process.on("unhandledRejection”,()=>{})捕获，或通过 async/awiat处理异步，用try..catch捕获。
注意在callback方法里: if (!this.listenerCount('error')) this.on('error', this.onerror); 即我们没有用on('error')形式去监听错误才会走onerror方法。

3.1.8 其他

inspect调用toJSON方法， toJSON打印对象信息

3.2 context.js

源码
骨架

const delegate = require('delegates');

const proto = module.exports = {
    ...
    onerror(err) {}
    ...
    delegate(proto, 'response')
    ...
   
    delegate(proto, 'request')
    ...

}

3.2.1 onerror()

onerror(err) {
    …
    this.app.emit('error', err, this);
    …
    
    this.status = err.status = statusCode;
    res.end(msg);
}

抛出error给application.onerror, 并调用res.end 响应

3.2.2 delegate response and request

delegate(proto, 'response')
.method('attachment')
.method('redirect’)
...

delegate(proto, 'request')
.method('acceptsLanguages')
.method('acceptsEncodings')
.method('acceptsCharsets')
...

调用ctx的属性或方法时，委托给response或request
如ctx.attachment() 相当于调用 response.attachment(), ctx.acceptsLanguages() 相当于调用 request.acceptsLanguages()
proto就是context对象

3.2.2.1 分析delegates

源码

3.2.2.1.1 delegate.method()

Delegator.prototype.method = function(name){
  var proto = this.proto;
  var target = this.target;
  this.methods.push(name);

  proto[name] = function(){
    return this[target][name].apply(this[target], arguments);
  };

  return this;
};

将target上的函数也能让proto去调用

3.2.2.1.2 delegate.access()

Delegator.prototype.access = function(name){
  return this.getter(name).setter(name);
};

Delegator.prototype.getter = function(name){
  ...

  proto.__defineGetter__(name, function(){
    return this[target][name];
  });

  return this;
};

Delegator.prototype.setter = function(name){
  ...

  proto.__defineSetter__(name, function(val){
    return this[target][name] = val;
  });

  return this;
};

通过__defineGetter__劫持proto的 get，转而去访问 target
通过 Object.defineProroty或Proxy 可以达到同样的效果，defineGetter、__defineSetter__是老的api

3.2 request.js

源码

包裹了原生req的属性，进行了扩展
如

...
get header() {
   return this.req.headers;
},
...

推荐在代码里会使用ctx.headers
调用链: ctx.headers -> request.headers -> req.headers

3.3 response.js

源码

包裹了原生res的属性，进行了扩展
如

...
get status() {
    return this.res.statusCode;
},
...

推荐在代码里会使用ctx.status
调用链: ctx.status -> response.status -> res.statusCode

3.4 总结

application.js 导出 Koa 实例类。它继承了EventEmitter,方便以on('error')形式监听错误
application.js: listen()起一个http sever;use()添加中间到middleware队列里;callback()作为http服务的回调函数，具体是迭代middlewares队列生成一个递归函数fnMiddleware，并用handleRequest在执行完fnMiddleware后处理一些通用的响应处理和错误处理
context.js: 代理 request.js、response.js 上的属性和方法
request.js: 封装了原生的req的属性
response.js: 封装了原生的res的属性

4 常见中间件分析

光有koa难以构建一个MVC模式的web系统，需要借助一些中间件
以下就是对常用的中间件进行分析

4.1 koa-router

源码
用法

const Koa = require( 'koa' );
const Router = require( 'koa-router' );
const router = new Router();
const app = new Koa();

router.get( '/', ( ctx, next ) => {
    ...
} ).post('/test',( ctx, next ) => {
    ...
} )


app.use( router.routes() )
    .use( router.allowedMethods() );

4.1.1 它支持的method

实际它支持 methods 包里的几十种 http method

4.1.2 Http method 调用

最后返回了this，支持链式调用

register()

调用router.[http method]如router.get(path, callback)，主要是创建了一个Layer,并加入了stack队列

Layer
源码

4.1.3 router.routes()

match()

routes()

由于layer.stack是一个route middleware数组，所以需memo.concat(layer.stack)

4.1.4 router.allowedMethods()

在响应出错后，即没有状态码或404时，allowedMethods()进行一些常见的收尾工作

4.1.4 总结

通过router.method将route 包裹成Layer对象注册到 stack队列里。router.routes() 会返回一个符合koa middler形式的dispatch函数。当用户请求的时候，会执行dispatch函数，它会迭代stack里的所有Layer对象的path去比对ctx.path.然后拿到符合的Layer.stack(每一个route的middlewares)，按顺序执行。
由于会迭代所有路由，如果路由很多，比如几千上万个，可能需要考虑优化

4.2 koa-bodyparser

源码

parse.json()

json: 判定Content-Type 是 json 如 application/json, 将请求流的数据转换成字符串，然后用JSON.parse解析后，赋给ctx.body json对象
form: 判定是"application/x-www-form-urlencoded" 类型后，通过qs将像 'a=1&b=1' 的字符串转换成json对象，再赋给ctx.body
text/plain 或 text/xml: 赋给ctx.body 字符串
默认 ctx.body = {}
如果需要支持 multipart/form-data (上传文件)，需额外使用 koa-multer，或用koa-body替代koa-bodayparser

4.2 koa-static

源码

用法

  const static = require('koa-static')

  app.use(static('public')

koa-send
源码

koa-static 通过path.resolve 解析出绝对路径，传递给koa-send. koa-send 通过file.createReadStream创建可读流赋值给ctx.body. 最后经过 koa的 repond方法里的 这一行代码 if (body instanceof Stream) return body.pipe(res) 处理后，响应给客户端
有时用assert(condition,error)代替if是一个不错的选择，例如这行代码 assert(root, 'root directory is required to serve files')

4.3 koa-cors

源码

非 OPTIONS 请求

OPTIONS 请求

设置一些CORS相关的response headers. OPTIONS请求 建议设置缓存时间max-age。