Dec 4th, 2023: [EN] DevOps: Github Action that uses nodejs+jest for API integration tests run against Elasticsearch

[walterra (Walter Rafelsberger)]

In this post, we'll explore the building blocks to set up a GitHub Action that utilizes Node.js and Jest to run API integration tests against an Elasticsearch instance. Automating these kinds of tests can significantly improve your development process and will allow you to develop new features or do refactorings with more confidence and reduce the risk of regressions.

Package setup

Depending on the type of library or application you’re developing, you should add the @elastic/elasticsearch package to your dependencies, and the packages jest and frisby to your devDependencies. For example, from the root of your project, run:

yarn add @elastic/elasticsearch
yarn add jest frisby --dev

There are many ways to configure jest. For this example we're going for something simple and add the config inline to our package.json like:

{
    ... // rest of your package.json
    "jest": {
        "testMatch": [
          "**/__tests__/**/*.test.js"
        ]
    }
}

This will tell jest to look for test files only in a __tests__ directory and filenames like my_test.test.js. Note we're setting this up to run API integration test via jest only. If you also want to run plain unit tests your setup might become more complex and you might want to split up how they are run.

To run the tests, add the following script to your package.json:

  "scripts": {
    ....
    "test": "jest --runInBand --detectOpenHandles --forceExit",
  },

The options for the script are a bit of workarounds to avoid problems when running jest later on through the Github Action.

Local development

The tests we're going to write expect that you have an Elasticsearch instance running without security at http://localhost:9200 (That's what the tests then also will expect to be available within the Github Action's environment). Let me repeat this: This particular setup without security is intended to be used only for your local development setup to write these tests. Do not run Elasticsearch like this in any kind of production environment. Using docker, you can set this up locally with:

# Download the docker image
docker pull docker.elastic.co/elasticsearch/elasticsearch:8.10.4

# Run the container
docker run --name es01 --net elastic -p 9200:9200 -it -m 1GB -e "discovery.type=single-node" -e "xpack.security.enabled=false" docker.elastic.co/elasticsearch/elasticsearch:8.10.4

Writing tests

Let's create a test file in __tests__/my_test.test.ts like this:

const frisby = require('frisby');

// Define the Elasticsearch server URL
const elasticsearchBaseUrl = 'http://localhost:9200'; // Replace with your Elasticsearch server URL

// Example data to index
const documentData = {
  title: 'Sample Document',
  content: 'This is the content of a sample document.',
};

// Define an Elasticsearch index name
const indexName = 'myindex'; // Replace with your index name

it('should create and index a document', () => {
  return frisby
    .post(`${elasticsearchBaseUrl}/${indexName}/_doc/1?refresh=true`, documentData, { json: true })
    .expect('status', 201)
    .expect('json', {
      result: 'created',
      _index: indexName,
      _id: '1',
    });
});

it('should search for a document', () => {
  return frisby
    .get(`${elasticsearchBaseUrl}/${indexName}/_search?q=content:sample`)
    .expect('status', 200)
    .expect('json', {
      hits: {
        total: {
          value: 1,
        },
      },
    });
});

it('should delete a document', () => {
  return frisby
    .delete(`${elasticsearchBaseUrl}/${indexName}/_doc/1`)
    .expect('status', 200)
    .expect('json', {
      result: 'deleted',
      _index: indexName,
      _id: '1',
    });
});

To get you started, this test file isn't really testing your application, but runs some tests against Elasticsearch by using it's API directly. For this post that's how far we take this code, because the focus is more on the overall setup. In a more real-world example, you'd import your own code into these tests which would use Elasticsearch APIs under the hood. If you set up your local Elasticsearch istance correctly, running yarn run test should run the above test successfully.

Github Action

Now let's head on and create the Github Action. Create a file named .github/workflows/ci.yml and add the following:

name: node-es-transformer CI

on:
  push:
    branches:
      - main # Adjust this to match your branch name

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - name: Configure sysctl limits
        run: |
          sudo swapoff -a
          sudo sysctl -w vm.swappiness=1
          sudo sysctl -w fs.file-max=262144
          sudo sysctl -w vm.max_map_count=262144

      - name: Runs Elasticsearch
        uses: elastic/elastic-github-actions/elasticsearch@master
        with:
          stack-version: 8.10.0
          security-enabled: false

      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Node.js and run tests
        uses: actions/setup-node@v3
        with:
          node-version: '18.x'
      - run: yarn
      - run: yarn build --if-present
      - run: yarn test

This action sets up Elasticsearch with some options to make it work within the Github Action environment and then goes on to run your tests against this instance. So once you push this to your Github repository you should see your tests running under the "Actions" tab.

To dive into this a bit more have a look at the repository of the node-es-transformer utility, it uses this setup to run its tests.