Binary data not recognized by YAML parser

[tmoschou]

Fails to recognise explicit binary type tag !!binary (tag:yaml.org,2002:binary)

Also fails to decode to spec as defined in http://yaml.org/type/binary.html.

Binary data is serialized using the base64 format as defined by RFC2045 (MIME), with the following notes:

• The content is not restricted to lines of 76 characters or less.

• Characters other than the base64 alphabet, line breaks and white space are considered an error.

Sample test case the demonstrates the issues:

From "Example 2.23. Various Explicit Tags" from spec (1.1 and 1.2) - http://www.yaml.org/spec/1.2/spec.html

---
picture: !!binary |
 R0lGODlhDAAMAIQAAP//9/X
 17unp5WZmZgAAAOfn515eXv
 Pz7Y6OjuDg4J+fn5OTk6enp
 56enmleECcgggoBADs=

@Test
public void testBinary() throws IOException {
    final ObjectMapper mapper = new ObjectMapper(new YAMLFactory());
    try (final InputStream inputStream = getClass().getResourceAsStream("yaml1.3-example2.23.yaml")) {
        final JsonNode bean = mapper.readTree(inputStream);
        final JsonNode picture = bean.get("picture");
        assertEquals(JsonNodeType.BINARY, picture.getNodeType()); // fails
        final byte[] gif = picture.binaryValue(); // also fails 
        assertEquals(65, gif.length);
        final byte[] actualFileHeader = Arrays.copyOfRange(gif, 0, 6);
        final byte[] expectedFileHeader = new byte[]{'G', 'I', 'F', '8', '9', 'a'};
        assertArrayEquals(expectedFileHeader, actualFileHeader);
    }
}

[cowtowncoder]

Thank you for reporting this, providing examples.
I'll have to see whether SnakeYAML (actual low-level decoder/encoder) provides enough information to expose it.

Note that Jackson databind does actually handle base64 encoding/decoding when using typed binding (to POJOS with byte[] properties); but JsonNode can not use such metadata.
But the default Base64Variant is one that does not force linefeeds (there are 3 or 4 different commonly used variants, not just one). Perhaps it would make sense to figure out a way to use different defaults for YAML codec; that should be doable.

[arulrajnet]

As per this https://bitbucket.org/asomov/snakeyaml/src/tip/src/test/java/org/yaml/snakeyaml/types/BinaryTagTest.java?fileviewer=file-view-default test class snakeYAML is already supporting this. Jackson YAMLParser lack of this support.

[cowtowncoder]

@arulrajnet Thank you for the link. This will be useful if and when someone has time to work on this.

[cowtowncoder]

Unfortunately test case uses higher level method of snakeYAML (load) which is not what Jackson parser uses (since it builds an in-memory structure instead of incremental). Tag is visible and easy to recognized, and Jackson has its own base64 codec which should work fine so that's not a big deal... but I'll see if snakeyaml might have lower level access to their codec.

[cowtowncoder]

Turns out there is simple class Base64Coder that is used internally so I can just use that I think.
While it may not be as streamlined as jackson codec it would provide closest compatibility wrt yaml, most likely, so it seems like the best way to go for now.

[earonesty]

wow, yaml typing is evil and idiosyncratic.

[cowtowncoder]

@earonesty Somehow it feels on-brand for YAML tho :-)

[earonesty]

There is no specification for the binary tag, no definition of encoding. Base64 is a convention tho, so using it is a safe choice.

[cowtowncoder]

As per tests for #90, Jackson YAML format does actually support decoding from !!binary tagged values, assuming Base64 encoding, without requiring target type of byte[].
So deserializing into JsonNode (via ObjectMapper.readTree()) does produce BinaryNodes; and deserializing as java.lang.Object should produce byte[] values, I think.

So even if YAML spec did not specify encoding, it should work in practice.