Invalid array length on "large xml" file

[nunomarks]

There seems to be a limit on the size of the XML that the tool can validate. The following error in thrown for a "large XML file":

RangeError [Error]: Invalid array length

I was able to reproduce the issue on an XML file that surpasses 134_217_724 bytes, or in hex, 0x7FFFFFC. Seems like a weird limit though, but pretty sure this is not an xmllint issue, as I'm able to validate the file via command line.

Is it possible to take a look at this issue?

[noppa]

I could take a look, but I'd need to be able to reproduce the error first.

Can you post the output of

npm version

Are you running on Mac, Windows or Linux, and if on Mac, is it Intel or M1 or M2?

This doesn't look like a mermory limit issue per se, but you could try raising the max memory limit

const {validateXML, memoryPages} = require('xmllint-wasm');

validateXML({
    ...
    maxMemoryPages: xmllint.memoryPages.GiB,

[nunomarks]

npm version

{
  npm: '10.2.4',
  node: '20.11.1',
  acorn: '8.11.2',
  ada: '2.7.4',
  ares: '1.20.1',
  base64: '0.5.1',
  brotli: '1.0.9',
  cjs_module_lexer: '1.2.2',
  cldr: '43.1',
  icu: '73.2',
  llhttp: '8.1.1',
  modules: '115',
  napi: '9',
  nghttp2: '1.58.0',
  nghttp3: '0.7.0',
  ngtcp2: '0.8.1',
  openssl: '3.0.13+quic',
  simdutf: '4.0.4',
  tz: '2023c',
  undici: '5.28.3',
  unicode: '15.0',
  uv: '1.46.0',
  uvwasi: '0.0.19',
  v8: '11.3.244.8-node.17',
  zlib: '1.2.13.1-motley-5daffc7'
}

Are you running on Mac, Windows or Linux, and if on Mac, is it Intel or M1 or M2?

Mac Apple M1 Pro

This doesn't look like a memory limit issue per se, but you could try raising the max memory limit

I did that and increased to the maximum memory possible before creating this issue (and it didn't work), so you are right, it's not a memory limit issue.

[noppa]

Thanks, I don't have a Mac available but I'll try to reproduce with the other environment specs

[nunomarks]

Thanks, I don't have a Mac available but I'll try to reproduce with the other environment specs

Happy to provide you anything else you might need to help you reproduce it. I attached the XML I used to test.

very-large.xml.zip

[noppa]

Thanks for the test XML, I was able to reproduce the issue.

I have a possible (not yet very well tested) fix, here: v5.0.0-alpha (PR #21). Could you test with this prerelease version, e.g. by installing it with

npm i xmllint-wasm@https://github.com/noppa/xmllint-wasm/releases/download/v5.0.0-alpha/xmllint-wasm.tgz

[nunomarks]

The solution from #21 works. I have tested with several large XMLs (all UTF-8 encoded) and everything is flawless 👌

[noppa]

Sweet. I'm a bit busy for a few days but will try to test and craft a proper release soonish. Meanwhile, the github alpha release should work fine.

[noppa]

This is also now in npm, with version 5.0.0-rc.0. I'll also test this in prod a bit before I dare to tag that as the latest release.

[Susccy]

Hello what's the state of this fix? Has it been released yet? Because we're encountering the same issue with large XML files at the moment using xmllint-wasm v4.0.2

[noppa]

It should be fixed in version 5.0.0-rc.0, which you can install with npm i xmllint-wasm@5.0.0-rc.0.

I still haven't done proper testing for it in production yet, so that's why I haven't done a proper 5.0 release yet. Thanks for the reminder. Meanwhile, upgrading to that rc version and using it should work fine.