Skip to content

Conversation

Leonidas-from-XIV
Copy link
Contributor

@Leonidas-from-XIV Leonidas-from-XIV commented May 29, 2025

Yojson is an optimized parsing and printing library for the JSON format

CHANGES:

2025-05-29

Changed

Fixed

Removed

@mseri
Copy link
Member

mseri commented May 30, 2025

The removal of variant support is breaking some packages:


#=== ERROR while compiling azure-cosmos-db.0.1.2 ==============================#
# context              2.3.0 | linux/x86_64 | ocaml-base-compiler.4.14.2 | file:///home/opam/opam-repository
# path                 ~/.opam/4.14/.opam-switch/build/azure-cosmos-db.0.1.2
# command              ~/.opam/opam-init/hooks/sandbox.sh build dune build -p azure-cosmos-db -j 31
# exit-code            1
# env-file             ~/.opam/log/azure-cosmos-db-7-3c6273.env
# output-file          ~/.opam/log/azure-cosmos-db-7-3c6273.out
### output ###
# (cd _build/default && /home/opam/.opam/4.14/bin/ocamlc.opt -w -40 -g -bin-annot -I src/.cosmos.objs/byte -I /home/opam/.opam/4.14/lib/angstrom -I /home/opam/.opam/4.14/lib/astring -I /home/opam/.opam/4.14/lib/atdgen -I /home/opam/.opam/4.14/lib/atdgen-runtime -I /home/opam/.opam/4.14/lib/base64 -I /home/opam/.opam/4.14/lib/bigstringaf -I /home/opam/.opam/4.14/lib/biniou -I /home/opam/.opam/4.14/lib/bytes -I /home/opam/.opam/4.14/lib/camlp-streams -I /home/opam/.opam/4.14/lib/cohttp -I /home/opam/.opam/4.14/lib/cohttp-lwt -I /home/opam/.opam/4.14/lib/cohttp-lwt-unix -I /home/opam/.opam/4.14/lib/conduit -I /home/opam/.opam/4.14/lib/conduit-lwt -I /home/opam/.opam/4.14/lib/conduit-lwt-unix -I /home/opam/.opam/4.14/lib/cryptokit -I /home/opam/.opam/4.14/lib/domain-name -I /home/opam/.opam/4.14/lib/easy-format -I /home/opam/.opam/4.14/lib/findlib -I /home/opam/.opam/4.14/lib/fmt -I /home/opam/.opam/4.14/lib/ipaddr -I /home/opam/.opam/4.14/lib/ipaddr-sexp -I /home/opam/.opam/4.14/lib/ipaddr/unix -I /home/opam/.opam/4.14/lib/logs -I /home/opam/.opam/4.14/lib/logs/fmt -I /home/opam/.opam/4.14/lib/logs/lwt -I /home/opam/.opam/4.14/lib/lwt -I /home/opam/.opam/4.14/lib/lwt/unix -I /home/opam/.opam/4.14/lib/lwt_log -I /home/opam/.opam/4.14/lib/lwt_log/core -I /home/opam/.opam/4.14/lib/lwt_ssl -I /home/opam/.opam/4.14/lib/macaddr -I /home/opam/.opam/4.14/lib/magic-mime -I /home/opam/.opam/4.14/lib/netstring -I /home/opam/.opam/4.14/lib/netsys -I /home/opam/.opam/4.14/lib/ocaml/threads -I /home/opam/.opam/4.14/lib/ocplib-endian -I /home/opam/.opam/4.14/lib/ocplib-endian/bigstring -I /home/opam/.opam/4.14/lib/ocsigenserver -I /home/opam/.opam/4.14/lib/ocsigenserver/baselib -I /home/opam/.opam/4.14/lib/ocsigenserver/baselib/base -I /home/opam/.opam/4.14/lib/ocsigenserver/cookies -I /home/opam/.opam/4.14/lib/ocsigenserver/http -I /home/opam/.opam/4.14/lib/ocsigenserver/polytables -I /home/opam/.opam/4.14/lib/ppx_sexp_conv/runtime-lib -I /home/opam/.opam/4.14/lib/re -I /home/opam/.opam/4.14/lib/seq -I /home/opam/.opam/4.14/lib/sexplib0 -I /home/opam/.opam/4.14/lib/ssl -I /home/opam/.opam/4.14/lib/stringext -I /home/opam/.opam/4.14/lib/uri -I /home/opam/.opam/4.14/lib/uri-sexp -I /home/opam/.opam/4.14/lib/uri/services -I /home/opam/.opam/4.14/lib/xml-light -I /home/opam/.opam/4.14/lib/yojson -I /home/opam/.opam/4.14/lib/zarith -intf-suffix .ml -no-alias-deps -open Cosmos -o src/.cosmos.objs/byte/cosmos__Json_converter_j.cmo -c -impl src/json_converter_j.ml)
# File "src/json_converter_j.ml", line 2586, characters 10-39:
# 2586 |     match Yojson.Safe.start_any_variant p lb with
#                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
# Error: Unbound value Yojson.Safe.start_any_variant
# (cd _build/default && /home/opam/.opam/4.14/bin/ocamlopt.opt -w -40 -g -I src/.cosmos.objs/byte -I src/.cosmos.objs/native -I /home/opam/.opam/4.14/lib/angstrom -I /home/opam/.opam/4.14/lib/astring -I /home/opam/.opam/4.14/lib/atdgen -I /home/opam/.opam/4.14/lib/atdgen-runtime -I /home/opam/.opam/4.14/lib/base64 -I /home/opam/.opam/4.14/lib/bigstringaf -I /home/opam/.opam/4.14/lib/biniou -I /home/opam/.opam/4.14/lib/bytes -I /home/opam/.opam/4.14/lib/camlp-streams -I /home/opam/.opam/4.14/lib/cohttp -I /home/opam/.opam/4.14/lib/cohttp-lwt -I /home/opam/.opam/4.14/lib/cohttp-lwt-unix -I /home/opam/.opam/4.14/lib/conduit -I /home/opam/.opam/4.14/lib/conduit-lwt -I /home/opam/.opam/4.14/lib/conduit-lwt-unix -I /home/opam/.opam/4.14/lib/cryptokit -I /home/opam/.opam/4.14/lib/domain-name -I /home/opam/.opam/4.14/lib/easy-format -I /home/opam/.opam/4.14/lib/findlib -I /home/opam/.opam/4.14/lib/fmt -I /home/opam/.opam/4.14/lib/ipaddr -I /home/opam/.opam/4.14/lib/ipaddr-sexp -I /home/opam/.opam/4.14/lib/ipaddr/unix -I /home/opam/.opam/4.14/lib/logs -I /home/opam/.opam/4.14/lib/logs/fmt -I /home/opam/.opam/4.14/lib/logs/lwt -I /home/opam/.opam/4.14/lib/lwt -I /home/opam/.opam/4.14/lib/lwt/unix -I /home/opam/.opam/4.14/lib/lwt_log -I /home/opam/.opam/4.14/lib/lwt_log/core -I /home/opam/.opam/4.14/lib/lwt_ssl -I /home/opam/.opam/4.14/lib/macaddr -I /home/opam/.opam/4.14/lib/magic-mime -I /home/opam/.opam/4.14/lib/netstring -I /home/opam/.opam/4.14/lib/netsys -I /home/opam/.opam/4.14/lib/ocaml/threads -I /home/opam/.opam/4.14/lib/ocplib-endian -I /home/opam/.opam/4.14/lib/ocplib-endian/bigstring -I /home/opam/.opam/4.14/lib/ocsigenserver -I /home/opam/.opam/4.14/lib/ocsigenserver/baselib -I /home/opam/.opam/4.14/lib/ocsigenserver/baselib/base -I /home/opam/.opam/4.14/lib/ocsigenserver/cookies -I /home/opam/.opam/4.14/lib/ocsigenserver/http -I /home/opam/.opam/4.14/lib/ocsigenserver/polytables -I /home/opam/.opam/4.14/lib/ppx_sexp_conv/runtime-lib -I /home/opam/.opam/4.14/lib/re -I /home/opam/.opam/4.14/lib/seq -I /home/opam/.opam/4.14/lib/sexplib0 -I /home/opam/.opam/4.14/lib/ssl -I /home/opam/.opam/4.14/lib/stringext -I /home/opam/.opam/4.14/lib/uri -I /home/opam/.opam/4.14/lib/uri-sexp -I /home/opam/.opam/4.14/lib/uri/services -I /home/opam/.opam/4.14/lib/xml-light -I /home/opam/.opam/4.14/lib/yojson -I /home/opam/.opam/4.14/lib/zarith -intf-suffix .ml -no-alias-deps -open Cosmos -o src/.cosmos.objs/native/cosmos__Json_converter_j.cmx -c -impl src/json_converter_j.ml)
# File "src/json_converter_j.ml", line 2586, characters 10-39:
# 2586 |     match Yojson.Safe.start_any_variant p lb with
#                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
# Error: Unbound value Yojson.Safe.start_any_variant

@Leonidas-from-XIV could you have a look at the revdeps and send a separate PR to add the upper bounds? You can do it with opam admin add-constraint --packages=... "yojson<3.0.0" and then selectively add the broken packages to the commit

@Leonidas-from-XIV
Copy link
Contributor Author

A lot of these issues come from atdgen which generates them, so for now I've marked it as incompatible which should transitively solve the issue on packages like azure-cosmos-db (and if we fix it it would make these package compatible again, so that's why I'd rather attach the constraint on atdgen). Similar thing goes for ocf which causes a lot of packages to fail.

The other packages like Goblint, kind2, LSP actively deal with the variants code so they have to be fixed manually. I'll look into submitting some PRs (but it is easier once Yojson 3 is merged, as these PRs will require bumping the Yojson dependency to {>= "3"}).

Weirdly enough the opam admin command did nothing for me. It didn't fail, but it also didn't apply any changes.

@mseri
Copy link
Member

mseri commented May 30, 2025

Thanks a lot. It would be much faster to review and test the PR to submit the bounds changes as a separate PR. Otherwise we have to run a massive amount of tests and revdeps each time we make a change

@Leonidas-from-XIV
Copy link
Contributor Author

I can submit the change as separate PR. The reason why I didn't is that in a separate PR I don't have Yojson 3 so I don't see how applying the constraints interacts with the presence of Yojson 3 (e.g. "does constraining atdgen solve the issues or do packages actually ship with the generated code thus requiring the constraints applied to themselves"). Thus bar of merging that hypothetical PR, rebasing this PR upon the merged PR, fixing potentially newly revealed issues and creating new PRs it seems like a fairly complicated process that involves opam-repository maintainers to keep reviewing and merging additional PRs until this one is ready to be merged.

If you think that is preferable I can definitely cherry-pick the commits and move them to a separate branch.

@Leonidas-from-XIV
Copy link
Contributor Author

Technically only the ppx_deriving_yojson tests are incompatible and the PPX works fine otherwise, so I could probably write the constraint in a way where the tests only run on Yojson < 3 but not sure if OPAM filters allow for such queries. In any case, the fix is submitted upstream and doesn't require bumping the minimum supported version.

CHANGES:

*2025-05-39*

### Changed

- Floats are now always output to JSON in a standard-conformant way or not at
  all (raising an exception). This makes the `std` variants of functions
  identical to the non-`std` variants and the `std` arguments have no effect.
  Users are encouraged to switch to the non-`std` affixed variants, the others
  will be deprecated in the future.  (ocaml-community/yojson#184, @Leonidas-from-XIV)
- Bumped the minimum required version of OCaml for the main package to 4.08
  since the CI dropped the support. This however allows removing the dependency
  on the `seq` library, so the depencency cone becomes slightly smaller. (ocaml-community/yojson#194,
  @Leonidas-from-XIV)

### Fixed

- Fixed handling of escape sequences in JSON5. Known escapes like \b will be
  properly unescaped and undefined escape sequences will unescape to the
  character itself as per spec (ocaml-community/yojson#187, @david-maison-TrustInSoft)
- Fixed tests failing on Windows due to disagreements with the length of an
  input channel and the text mode conversion (ocaml-community/yojson#192, @Leonidas-from-XIV)

### Removed

- Removed support for Tuple and Variant in JSON. It was a non-standard
  extension that was rarely used, so this simplifies the Yojson types and the
  parser more standard-conforming (ocaml-community/yojson#105, ocaml-community/yojson#158, ocaml-community/yojson#185 @Leonidas-from-XIV)
It's actually mostly the tests causing issues, the tool works mostly
fine.
@mseri
Copy link
Member

mseri commented Jun 2, 2025

Thanks! Outside the many network issues, which are a CI problem, the failures now look unrelated

@mseri mseri merged commit 0e1a25f into ocaml:master Jun 2, 2025
0 of 2 checks passed
@Leonidas-from-XIV Leonidas-from-XIV deleted the release-yojson-3.0.0 branch June 3, 2025 08:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants