Skip to content

Latest commit

 

History

History
40 lines (23 loc) · 1.42 KB

developers.md

File metadata and controls

40 lines (23 loc) · 1.42 KB
Raw

If you are new to the catalogue pipeline the GitBook documentation is a good place to start.

Developing

Information for developers working on the catalogue-pipeline.

Things you might want to do

Reindexing

If the internal_model has been changed you will want to update the information stored by the pipeline to match that model.

A reindex operation runs the source data from the adapters through the pipeline causing it to be re-transformed / matched & merged as appropriate.

If you want to perform a reindex, follow the instructions in REINDEXING.md.

Lambdas

Some information for working with Python lambdas.

Scripts

Generally small things you might want to do irregularly involving the API & data are in /scripts

Problems you might have

  • Stack overflow from scalac (in IntelliJ) when building projects:

    Go to Settings > Build, Execution, Deployment > Compiler and change Build process heap size (Mbytes) to something large, eg 2048.

  • Pulling docker containers from ECR

    You'll need to log into ECR before local docker can pull from there:

    aws ecr get-login-password --region eu-west-1 --profile platform-dev | \
docker login --username AWS --password-stdin 760097843905.dkr.ecr.eu-west-1.amazonaws.com