Skip to content

XQA is a suite of Docker based microservices that improves the scalability of BaseX.

License

Notifications You must be signed in to change notification settings

jameshnsears/xqa-documentation

Repository files navigation

XQA - XML Quality Assurance CircleCI

1. Introduction

  • XQA is a suite of Docker based microservices that improves the scalability of BaseX - an XML database engine.
  • Instead of loading each XML file into a single BaseX engine, XQA distributes them - via an AMQP message broker - across multiple BaseX engines.
  • XQA ships with a basic web frontend - as well as a REST API - which lets you run XQuery, and analytics, against the distributed data.

2. High Level Design

High Level Design

3. Key Features

  • Easy to deploy / extend:
    • each microservice runs in its own container.
    • add extra BaseX engines easily - they self register against the AMQP message broker.
    • can run in either a docker-compose or k8s environment.
  • Proven scalability & performance improvements - graphs in xqa-perf show:
    • ingest timing statistics.
    • XML file distribution.
  • Quality Radiatior for CI; static analysis and coverage metrics.
  • Transparency:
    • container console logging.
    • JSON instrumentation sent to a central PstgreSQL instance.

4. Open Source Technologies

  • ActiveMQ
  • Angular
  • Docker - containers can be built from GitHub or pull'd from hub.docker.com.
  • Java 10/11.
  • PostgreSQL 11
  • Python 3.6
  • Ubuntu 18.04

5. Microservices

microservices

CI GitHub
Build Status xqa-commons-qpid-jms a Maven Central shared library.
Build Status xqa-db PostgreSQL.
Build Status xqa-db-amqp AMQP interface to PostgresSQL.
Build Status xqa-ingest XML file loader.
Build Status xqa-ingest-balancer distributes XML across XQA BaseX engine(s).
Build Status xqa-message-broker ActiveMQ.
Build Status xqa-perf end to end integration tests, with Matplotlib graphs.
Build Status xqa-query-balancer XQA REST API.
Build Status xqa-query-ui UI for querying XQA.
Build Status xqa-shard BaseX engine with AMQP interface.
xqa-test-data a collection of XML files used by XQA.

6. Limitatons

XQA is a proof of concept project. It scratched an itch and achieved what it set out to prove. Refer to the GitHub issue board for outstanding issues.

About

XQA is a suite of Docker based microservices that improves the scalability of BaseX.

Topics

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages