Skip to content

Testing: Integration Testing Server APIs

Jump to bottom
Jonathan Niles edited this page Jul 18, 2016 · 7 revisions

Integration Testing

Integration testing has many advantages:

  1. It safeguards against regressions in API behavior
  2. It provides information on application performance
  3. It documents APIs used throughout the application

bhima uses mocha and chai-http for integration testing. The tests are found in server/test/api/.

Running Integration Tests

To run integration tests properly, the database must be rebuilt, the server started, and then a call to mocha must be made with the path to the API tests. There is a bash shell script to automate this procedure.

# rebuild the database, build and start the server, and run mocha `/server/test/api/`
./sh/integration-tests.sh

Integration testing takes less than a few minutes. Performance varies by machine, but the bottleneck is typically building the test database.

Integration tests are designed to ensure that all APIs have uniform behavior, and should test every possible HTTP endpoint. Complex endpoints (such as a /search?paramA=valueA&paramB=valueB) should have a variety of tests that demonstrate the endpoint can handle complex queries and fail gracefully. Because integration tests are lightweight, more tests are generally better than fewer tests.

Writing Integration Tests

For easy testing, the following variables are globally available (setup.js binds them to the global object).

  1. chai (configured with chai-as-promised and chai-datetime).
  2. agent - the HTTP agent for making API requests

In order to create uniform behavior between APIs, bhima has helper utilities to facilitate writing new tests. As always, see the source for full documentation.

const helpers = require('./helpers');

// you can now use helpers.api.* to check for proper API responses

The agent provided globally is already logged in! You can go ahead an begin making requests to the server without any additional setup.

Testing CRUD APIs

The most frequently created APIs are CRUD APIs that perform specific database actions via HTTP verbs and reply with specific HTTP status codes to indicate the result of the action. Since these tests are so common, there are some utilities to help evaluate the status of a request.

/**
 * asserts that the server sent a JSON-encoded response with status code 201.  The
 * response should contain either an ID (integer) or UUID (36 character length string).
 */
helpers.api.created(res);


/**
 * asserts that the server sent a JSON-encoded response with status code 204.  The
 * response should have an empty body.
 */
helpers.api.deleted(res);


/**
 * asserts that the server sent a JSON-encoded response with status code {code}.  The
 * response should contain the properties "code" (translatable error code) and "reason"
 * (human readable error message). 
 */
helpers.api.errored(res, code);


/**
 * asserts that the server sent a JSON-encoded response with status code 200.  The
 * response should be an array of length {length}.
 */
helpers.api.listed(res, length);

What should be tested?

Every HTTP endpoint should have at least one test. For more complex endpoints, or endpoints that may return different results based on server state, there should be multiple tests.

API Test Quick Checklist

Since that majority of bhima's integration tests concern APIs, below is a quick checklist of routes and conditions to check when creating API tests.

  • GET /resources should return an empty list of no resources present in the database.
  • GET /resources/unknown should return a 404 NOT FOUND error (unknown id)
  • GET /resources/:id should return a 200 OK HTTP status header for an existing id
  • GET /resources?param=value should transform output based on the query string. For example, ?detailed=1 should be tested by making sure all the properties of the resource are returned.
  • POST /resources should succeed for a valid object, with a specified id/uuid.
  • POST /resources should succeed for a valid object, without a specified id/uuid.
  • POST /resources should fail (400 BAD REQUEST) for an invalid object (missing keys, failed unique constraints, etc).
  • PUT /resources/:id should succeed for a valid update, and return the full changed object.
  • PUT /resources/unknown should fail (404 NOT FOUND) for an unknown id.
  • PUT /resources/:id should fail (400 BAD REQUEST) for an invalid update.
  • DELETE /resources/:id should succeed for a resource that can be deleted.
  • DELETE /resources/unknown should fail (404 NOT FOUND) for an unknown id.
  • DELETE /resources/:id should fail (400 BAD REQUEST) if the resource has a constraint preventing it from being deleted. For example, a reference constraint.

Tips & Tricks

  1. Use before(helpers.login(agent)) instead of beforeEach(helpers.login(agent)) in your tests. This will reduce the number of HTTP login requests form one request per test suite to one request per test function.

  2. Prefer using the helpers.api.created(), helpers.api.listed(), etc, functions as described here. They are much easier to read, and reduce the number of lines of code Steven has to review. It also will reduce the number of bugs in your code - it is easier to make a typo in six lines of code than in one line. Finally, it it ensure that your API behaves the same way as all other developers.

  3. Ensure your JSLint does not throw errors for mocha's describe(), it(), beforeEach(), etc, functions. We have configured the .jshintrc file in the repository to not throw errors for these functions. You no longer need to start your tests with /* global describe, it, beforeEach */. If you still have errors, your JSLint is not using bhima's lint rules, and it will be impossible to know if your code is up to bhima's standards.

  4. There really isn't a need to write "METHOD : VERB, PATH : /route" in front of every integration test. Simply writing "VERB /route" is much easier to read in the console, when a test does break. Conforming to these standards allows much easier debugging in the future.

  5. Tests should never expect to receive an InternalServerError (500). If that error is returned, it means the server broke. You should always expect to see a 4XX error code from the server, if an error is expected. Here is the full list of HTTP codes that we expect to have returned for API endpoints.

Clone this wiki locally