Utilize Github's Webhook to Fulfill Automatic Deployment

Github’s webhook functionality can fulfill automatic deployment conveniently. This article records the process of development and deployment via Node.js, when the master branch is pushed, the project will be automatically deployed, the complete code is on GitHub

Add Webhook

  1. On the homepage of the corresponding project of Github, click menu Setting in top right corner, click menu Webhooks on left side, click button Add webhook of top right corner

  2. Set Payload URL as address which will receive event, suggested Payload URL should be applicaiton/json, Secret is optional, and can be any string, choose Just the push event. for Which events would you like to trigger this webhook?, check Active, click button Add webhook below

Develop Request Handling

Receive Request

Utilize Node.js to setup a http server, receive POST request and handle the submitted data

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
const { createServer } = require('http');
const port = process.env.GITHUB_WEBHOOK_PORT || '3000';

const server = createServer((req, res) => {
  if('POST' === req.method){
    let body = '';
    req.on('data', chunk => {
      body += chunk.toString();
    });
    req.on('end', () => {
    });
  }
})

server.listen(port, () => {
  console.log(`Listening on ${port}`);
});

if the default port 3000 need to be changed, you can firstly run the following command to add environment variable (NUMBER can be any port)

export GITHUB_WEBHOOK_PORT=NUMBER

Parse Body

In the end event handler of req, parse string body to object 

1
2
3
4
5
6
req.on('end', () => {
  try{
    body = JSON.parse(decodeURIComponent(body).replace(/^payload=/, ''));
  }catch(e){
    console.log(e)
  }

If Content type is set to applicaiton/json, just body = JSON.parse(body) is sufficient, the above code add compatibility of situation when Content type is set to application/x-www-form-urlencoded

Pull Updates

According to push payload for body, extract project and branch information, if it’s master branch, command to enter corresponding project and pull the branch will be executed

1
2
3
4
5
6
if('object' === typeof body){
  if('refs/heads/master' === body.ref){
    const { exec } = require('child_process');
    const command = `cd ../${body.repository.name} && git pull origin master`;
    exec(command, (error, stdout, stderr) => {
    });

Note that the directory where project locates, and the directory where this application locates, are in the same parent directory, or else the entry path in the command should be adjusted

Verify Secret

The above step have fulfilled automatically pull updates, but there’s security issue, because not only Github can send this kind of request, so it’s better to set Secret and proceed security verification

1
2
3
4
5
6
7
8
9
10
11
12
13
const secret = process.env.GITHUB_WEBHOOK_SECRET || '';
...
    req.on('end', () => {
      if('' !== secret){
        const { createHmac } = require('crypto');
        let signature = createHmac('sha1', secret).update(body).digest('hex');
        if(req.headers['x-hub-signature'] !== `sha1=${signature}`){
          console.log('Signature Error');
          res.statusCode = 403;
          res.end();
          return;
        }
      }

Before application is run, firstly run the following command to add secret variable (STRING can be any string)

export GITHUB_WEBHOOK_SECRET=STRING

  • After Secret is set, Github will add header x-hub-signature as sha1=SIGNATURE when request is sent, wherein SIGNATURE is the HMAC hex digest of body, with key Secret, and algorithm sha1
  • Through verification of Secret, We can make sure that only who know Secret, can send correct request with header x-hub-signature, or else it will be rejected
  • The above code add compatibility for situation when Secret is not set, namely if variable GITHUB_WEBHOOK_SECRET is not added, the handling logic will be the same as origin, without any verification

Build via Local Hook

If the project need to be built after pull updates, the building command can be added in the end of variable command, such as && npm run build, but building command of different project may not be the same, furthermore building command of some project can be complicated, git’s local hook can be set to handle these kind of situation

cd /PATH/TO/PROJECT/.git/hooks
nano post-merge

1
2
#!/bin/sh
SHELL_SCRIPT

chmod +x post-merge

  • Here /PATH/TO/PROJECT/ is the directory location of the project, SHELL_SCRIPT can be any Shell script
  • Since git pull is combination of git fetch and git merge, the pull updates will trigger post-merge hook
  • New added file has not execute permission by default, so we need to add x bit via chmod

Deploy Application Online

Persistence and automation need to be fulfill to deploy application online, namely the project should be always running, and if the server is reboot, the project should run automatically

Automatically Create Variable

Script for variable creation in /etc/profile.d/ will run automatically when server reboot, so a setting script is added in it

nono /etc/profile.d/github-webhook.sh

1
2
export GITHUB_WEBHOOK_PORT=NUMBER
export GITHUB_WEBHOOK_SECRET=STRING

Run the following command to make the variable creation to take effect at once

source /etc/profile

Run Application via pm2

pm2 can guarantee sustained running of Node application, and functionality of monitoring, hot patching and so on can be fulfilled via configuration

npm install pm2 -g
pm2 start app.js –name github-webhook

Automatically Run After Reboot

pm2 has build-in support to config the original application to auto-runn when start up, which can be fulfilled by the following command

pm2 startup
pm2 save

pm2 startup will create and enable service which will automatically run when start up, pm2 save will save the current pm2 running application, as restore content after reboot

Summarize

In this automatically deployment process which base on Github webhook, the following technologies have been used:

  • http，child_process and crypto module of Node.js
  • post-merge Shell hook of Git
  • Automatically variable setting via profile and pm2 toolkit