encodeResourceToString not encoding contained resources

[michael-christensen]

Describe the bug
Contained resources are not encoded when using encodeResourceToString to encode a resource. Investigating the issue with a debugger I found that the issue seems to stem from line 280 in JsonParser.java:

List containedResources = getContainedResources().getContainedResources();

The double invocation of "getContainedResources" will always return a zero length list.

To Reproduce
Make a call like this:

FhirContext.forR4().newJsonParser().setPrettyPrint(true).encodeResourceToString(aFhirResource));

Where "aFhirResource" is a resource with contained resources. The resulting string will not have any contained resources.

Expected behavior
Contained resources are encoded to string.

Environment (please complete the following information):

• HAPI FHIR Version: 3.6.0

[jamesagnew]

Hi Michael,

Can you please provide a complete code sample and/or unit test showing this issue?

[michael-christensen]

Hi James,

Something like this Unit test should do the trick

import static org.junit.Assert.assertTrue;
import ca.uhn.fhir.context.FhirContext;
import ca.uhn.fhir.parser.JsonParser;
import ca.uhn.fhir.parser.json.GsonStructure;
import ca.uhn.fhir.parser.json.JsonLikeStructure;
import ca.uhn.fhir.parser.json.JsonLikeWriter;
import java.io.IOException;
import java.io.StringWriter;
import java.io.Writer;
import org.hl7.fhir.instance.model.api.IBaseResource;
import org.junit.Test;

public class testEncodingContained {
    private static final org.slf4j.Logger ourLog = org.slf4j.LoggerFactory.getLogger(testEncodingContained.class);

    private static final String RESOURCE_WITH_CONTAINED =
            "{\"resourceType\":\"Observation\",\"hasMember\":[{\"reference\":\"childObs1\"}],\"contained\":[{\"resourceType\":\"Observation\",\"id\":\"childObs1\"}]}";

    @Test
    public void testDoEncodeResourceToJsonLikeWriter() throws IOException {

        Writer stringWriter = new StringWriter();
        JsonParser jsonParser = (JsonParser) FhirContext.forR4().newJsonParser();
        IBaseResource resource = jsonParser.parseResource(RESOURCE_WITH_CONTAINED);
        JsonLikeStructure jsonStructure = new GsonStructure();
        JsonLikeWriter jsonLikeWriter = jsonStructure.getJsonLikeWriter(stringWriter);
        jsonParser.doEncodeResourceToJsonLikeWriter(resource, jsonLikeWriter);
        String resourceAsString = jsonLikeWriter.getWriter().toString();
        ourLog.debug("Resource before encoding: " + RESOURCE_WITH_CONTAINED);
        ourLog.debug("Resource after encoding: " + resourceAsString);
        assertTrue(resourceAsString.equals(RESOURCE_WITH_CONTAINED));
    }
}

[patrick-werner]

this issue explains: #1184

[michael-christensen]

PS: log outout from the above test is:

Resource before encoding: {"resourceType":"Observation","hasMember":[{"reference":"childObs1"}],"contained":[{"resourceType":"Observation","id":"childObs1"}]} Resource after encoding: {"resourceType":"Observation","hasMember":[{"reference":"childObs1"}]}

[michael-christensen]

Oops, just investigated a bit further. Turns out it all comes down to me forgetting a "#" character in the reference to the contained resource. This means the BaseParser.containResourcesForEncoding will not pick up the reference and thus will never call ContainedResources.addContained. This in turn results in ContainedResources.myResourceList remaining empty.

Not sure if this is by design: That an unreferenced contained resource doesn't get encoded?

[jamesagnew]

That is actually a deliberate decision- If a contained resource has nothing linking to it, it is not valid per FHIR's rules so HAPI FHIR omits it during parsing/serialization.

[michael-christensen]

My issue was as far as i can judge that the parser did not omit the contained resources. I am using the strict parser and there were no complaints over these references and contained resources. Furthermore, inspecting the parsed resource (in the debugger) it did have all the contained resources. So nothing was thrown away by the parser. That made me think all was ok with the input and i went looking for the error elsewhere. Only during subsequent serialization did the contained resources disappear. And that in a totally quiet way. I may be digressing but:
a) Shouldn't the strict parser have complained?
b) If the serializer ignores / throws away stuff shouldn't it be verbose about it? Maybe there's a need for a "strict serializer" that can throw errors under such circumstances?

[patrick-werner]

That is actually a deliberate decision- If a contained resource has nothing linking to it, it is not valid per FHIR's rules so HAPI FHIR omits it during parsing/serialization.

If it is not valid, it should at least log an error in lenient mode. In strict mode i also would have expected an DataFormatException to be thrown.
Just dropping parts of a resource without giving some feedback to the implementer sounds very dangerous to me.
@jamesagnew

[tadgh]

I've also been bitten by this just now... a server we are calling has IDs like this:

ABC-123-008/PLAS:QNHL79902:1

Since the / seems to be considered as an ID split point, the result section of the diagnostic report, during encoding, the references refer to the full ID, but the contained resources is cached in the map with just the PLAS:QNHL79902:1 section. So that contained element suddenly counts as not being referred to, so suddenly our contained Observations are just being silently dropped.

[lawley]

I too was bitten by this last week.

@tadgh note that / is not a valid character in an id - see https://www.hl7.org/fhir/datatypes.html#id

[tadgh]

Yeah i know, just took me a while to catch it, since I am not the steward of this data and can't control its shape. All I can really do here is send a pointed email asking for a fix.

[steveswinsburg]

Just hit us also, we did some digging and found it was indeed the ID of the resource missing the # prefix, which is very easy to do when adding resources to a ListResource (as you wouldn't adjust the IDs, since you don't when adding to a Bundle).

Dropping resources silently is a very dangerous approach.

[jamesagnew]

I didn't think we still did. What version of HAPI FHIR are you using? If not 6.8.0 can you please upgrade and try again? If it's still happening, can you please provide a minimal test case?

[Wpyh]

@jamesagnew I've tried a simple test to see how the resource without # prefix is handled in 6.8.0. It seems to me it's a still a bug. Please find the attached for a test case.
hapifhirResourceIdTest.zip

[brianalind]

I've run into another flavor of this problem. For adaptive forms a Questionnaire resource is contained inside a QuestionnaireResponse. The reference to the Questionnaire is the "questionnaire" field, i.e. "#questionnaireId", which is a canonical and not a reference type. So HAPI discards the contained resource when encoding to JSON. A way to bypass this behavior sure would be nice.