failing test testsuite-json.test 256 UGM3 with jq 1.7

[MarDiehl]

when building the AUR package, I encounter a failing test (testsuite-json.test 256 UGM3). Log file is attached.

There are also a few compiler warning:

Making all in src
  CC    tool/fy_tool-fy-tool.o
  CC    lib/fy-parse.o
  CC    lib/fy-types.o
  CC    lib/fy-diag.o
lib/fy-diag.c: In function 'fy_diag_error_atom_display':
lib/fy-diag.c:755:51: warning: 'cols' may be used uninitialized [-Wmaybe-uninitialized]
  755 |                         while (col8 < (line_shift + cols) && (c = fy_utf8_get(s, (e - s), &w)) >= 0 && rbs < rbe) {
      |                                       ~~~~~~~~~~~~^~~~~~~
lib/fy-diag.c:583:19: note: 'cols' was declared here
  583 |         int pass, cols, min_col, max_col, max_line_count, max_line_col8, max_width;
      |                   ^~~~
lib/fy-diag.c:746:29: warning: 'rowbufsz' may be used uninitialized [-Wmaybe-uninitialized]
  746 |                         rbe = rowbuf + rowbufsz;
      |                         ~~~~^~~~~~~~~~~~~~~~~~~
lib/fy-diag.c:594:16: note: 'rowbufsz' was declared here
  594 |         size_t rowbufsz;
      |                ^~~~~~~~
  CC    lib/fy-dump.o
  CC    lib/fy-atom.o
  CC    lib/fy-token.o
  CC    lib/fy-input.o
  CC    lib/fy-docstate.o
  CC    lib/fy-doc.o
  CC    lib/fy-docbuilder.o
  CC    lib/fy-emit.o
  CC    lib/fy-event.o
  CC    lib/fy-accel.o
  CC    lib/fy-walk.o
  CC    lib/fy-path.o
  CC    lib/fy-composer.o
lib/fy-composer.c:348:1: warning: conflicting types for 'fy_composer_process_event' due to enum/integer mismatch; have 'enum fy_composer_return(struct fy_composer *, struct fy_event *)' [-Wenum-int-mismatch]
  348 | fy_composer_process_event(struct fy_composer *fyc, struct fy_event *fye)
      | ^~~~~~~~~~~~~~~~~~~~~~~~~
In file included from lib/fy-parse.h:40,
                 from lib/fy-composer.c:21:
lib/fy-composer.h:51:5: note: previous declaration of 'fy_composer_process_event' with type 'int(struct fy_composer *, struct fy_event *)'
   51 | int fy_composer_process_event(struct fy_composer *fyc, struct fy_event *fye);
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~
  CC    xxhash/xxhash.o
  CC    util/fy-ctype.o
  CC    util/fy-utf8.o
  CC    util/fy-utils.o
  CC    util/fy-blob.o
  CC    thread/fy-thread.o
  CC    blake3/blake3_host_state.o
  CC    blake3/blake3_backend.o
  CC    blake3/blake3_be_cpusimd.o
  CC    blake3/fy-blake3.o
  CC    blake3/blake3_portable.o
  CC    blake3/blake3.o
  LINK  libb3portable.la
  CC    blake3/blake3_sse2.o
  AS    blake3/blake3_sse2_x86-64_unix.o
  CC    blake3/blake3.o
  LINK  libb3sse2.la
  CC    blake3/blake3_sse41.o
  AS    blake3/blake3_sse41_x86-64_unix.o
  CC    blake3/blake3.o
  LINK  libb3sse41.la
  CC    blake3/blake3_avx2.o
  AS    blake3/blake3_avx2_x86-64_unix.o
  CC    blake3/blake3.o
  LINK  libb3avx2.la
  CC    blake3/blake3_avx512.o
  AS    blake3/blake3_avx512_x86-64_unix.o
  CC    blake3/blake3.o
  LINK  libb3avx512.la
  LINK  libfyaml.la
  LINK  fy-tool
  CC    internal/libfyaml_parser-libfyaml-parser.o
  LINK  libfyaml-parser
  CC    internal/fy_thread-fy-thread.o
  LINK  fy-thread
  CC    internal/fy_b3sum-fy-b3sum.o
  LINK  fy-b3sum

test-suite.log

[MarDiehl]

the error is

--- /tmp/tmp.3VOEwfva1W 2023-09-26 10:45:04.296399978 +0200                                         
+++ /tmp/tmp.nIOU0yBdZ3 2023-09-26 10:45:04.299733412 +0200                                         
@@ -15,13 +15,13 @@                                                                                 
   "product": [                                                                                     
     {                                                                                              
       "description": "Basketball",                                                                 
-      "price": 450.00,                                                                             
+      "price": 450,                                                                                
       "quantity": 4,                                                                               
       "sku": "BL394D"                                                                              
     },                                                                                             
     {                                                                                              
       "description": "Super Hoop",                                                                 
-      "price": 2392.00,                                                                            
+      "price": 2392,                                                                               
       "quantity": 1,                                                                               
       "sku": "BL4438H"                                                                             
     }                                                                                              
not ok 256 UGM3 - Spec Example 2.27. Invoice (JSON)                                                 
FAIL: testsuite-json.test 256 UGM3 - Spec Example 2.27. Invoice (JSON)   

so somehow floats with trailing zeros are changed to integers.

I only see this error if jq (https://jqlang.github.io/jq/) is installed. The version that fails is 1.7 from https://archlinux.org/packages/extra/x86_64/jq/

[MarDiehl]

The issue seems to be related to the new jq release (https://github.com/jqlang/jq/releases/tag/jq-1.7). the test fails for libfyaml 0.8 and 0.9

[pantoniou]

Yes, I would guess this would be a problem. I need to think about how to control jq output.

[rwirth]

The expected JSON file contains integers, the yaml file doesn't:

  "product": [
    {
      "sku": "BL394D",
      "quantity": 4,
      "description": "Basketball",
      "price": 450
    },
    {
      "sku": "BL4438H",
      "quantity": 1,
      "description": "Super Hoop",
      "price": 2392
    }
  ],

product:
    - sku         : BL394D
      quantity    : 4
      description : Basketball
      price       : 450.00
    - sku         : BL4438H
      quantity    : 1
      description : Super Hoop
      price       : 2392.00

This looks like a problem with the test itself. Libfyaml behaves correctly by not changing the type -- at least that's what I'd expect. Maybe jq converted floats to ints during canonicalization, which has been changed in newer versions? That would mask that the difference in outputs was always present. I'm running into the same failure with jq 1.6, btw.

E: It's probably this PR 1752. From the changes to the manual:

jq will also try to maintain the original decimal precision of the provided
number literal. See below for examples. [...]

         - program: 'map([., . == 1]) | tojson'
           input: '[1, 1.000, 1.0, 100e-2]'
           output: ['"[[1,true],[1.000,true],[1.0,true],[1.00,true]]"']

[pantoniou]

The real problem is that libfyaml does not do (yet on master) object conversions due to only supporting the built-in schema (i.e. all scalars are strings).

Arguably the test will have to be skipped while this is the case, and I will revisit when schema support lands in master.