HAPI FHIR Bulk Data Export

This project aims at simplifying the Bulk Data Export process in HAPI FHIR, and which is documented here

Maven dependency

You can get the dependency through maven, by adding the following to your pom.xml:

<distributionManagement>
    <repository>
        <id>nexus</id>
        <url>https://nexus3.trifork.com/repository/maven-ehealth-releases-public/</url>
    </repository>
</distributionManagement>

<dependency>
    <groupId>com.trifork.ehealth</groupId>
    <artifactId>hapi-fhir-export-client</artifactId>
    <version>3.0.2</version>
</dependency>

Configuring the client

To create a client, we need a FhirContext which can contain custom configured parsers. We also need a HttpClient, which comes configured with all the necessary security/interceptors and other properties specific to the application.

    FhirContext fhirContext = FhirContext.forR4();
    httpClient = HttpClient.newHttpClient();
    BDExportClient exportClient = new BDExportClient(fhirContext, httpClient);

Setting up the request

To be able to initiate a bulk data export, we have put the options together in BDExportRequest

Example of a request, following the examples given in the Experimental Query Parameters:

    ...
        
    IBDExportRequest request = 
        new BDExportRequest(URI.create("http://myserver.example.org/fhir/$export"))
            .setOutputFormat(Constants.CT_FHIR_NDJSON)
            .addType(ResourceType.MedicationRequest)
            .addType(ResourceType.Condition)
            .setSince(new InstantType(new GregorianCalendar(2023, Calendar.OCTOBER, 27)))
            .addTypeFilter(
                new BDExportTypeFilter(
                    ResourceType.MedicationRequest,
                    "status",
                    new TokenParam("active")
                )
            )
            .addTypeFilter(
                new BDExportTypeFilter(
                    ResourceType.MedicationRequest,
                    "status",
                    new TokenParam("completed")
                ).and(
                    "date",
                    new DateParam(
                        ParamPrefixEnum.GREATERTHAN,
                        Date.from(Instant.now().minus(2, ChronoUnit.DAYS))
                    )
                )
        );
                
    ...

When a BDExportRequest has been constructed, we can go ahead and initiate an export.

Initiating an export

Assuming the client has been configured, and a request has been made, a request can then be initiated, which returns a IBDExportFuture, where the result will eventually be available.

    ...

        IBDExportFuture future = exportClient.startExport(request);
        
    ...

Resuming an export

If the application crashes, while it awaits the future, or another thread should pick up the responsibility for fetching the export result, it is possible to resume an ongoing export, given the polling location of the export

Example:

    ...

        IBDExportFuture future = exportClient.startExport(request);
        URI pollingUri = future.getPollingUri();
        
        ...

        IBDExportFuture futureForTheSameExport = exportClient.resumeExport(pollingUri)
    ...

Output

When the export eventually finishes, the result will be available in a BDExportResponse, and will contain links to FHIR Binary resources, that contain the output:

    ...

        IBDExportFuture future = exportClient.startExport(request);
        BDExportResponse response = future.get();
        
        Optional<BDExportResultResponse> resultOpt = response.getResult();
        if (resultOpt.isPresent()) {
            BDExportResultResponse result = resultOpt.get();
            List<BDExportResultResponse.OutputItem> output = result.getOutput();
    
            // Fetch and read binary outputs
    
            List<BDExportResultResponse.OutputItem> error = result.getError();
    
            // Fetch and read errors during processing.
        } else if (response.getError().isPresent()) {
            // Handle errors..
        }
        
    ...

Parsing the output

There are many options of handling the output. A BDExportResultResponse only provides a link to the actual output to the Bulk Data Export, usually through a FHIR Binary resource. You could either fetch those binary resources manually, or use a convenient converter in this client library:

    ...
        IGenericClient hapiFhirClient = ...;
        BDExportConverter converter = new BDExportConverter(hapiFhirClient); 
        BDExportResponse response = future.get();
        Optional<BDExportResultResponse> resultOpt = response.getResult();
        if (resultOpt.isPresent()) {
            BDExportResourceResult result = converter.convert(response.getResult().get());
    
            for (BDExportResourceResult.ResourceItem resourceItem : result.getOutput()) {
            Binary resource = resourceItem.getResource();
            byte[] content = resource.getData();
    
            // Handle content...
        }


        for (BDExportResourceResult.ResourceItem resourceItem : result.getError()) {
            Binary resource = resourceItem.getResource();
            byte[] content = resource.getData();
    
            // Handle errors, e.g. logging them.
            }
        } else if (response.getError().isPresent()) {
            // Handle errors..
        }
    ...