Regression between atlas in Nim 2.0.2 and Nim 2.0.8 : old hash commit with previous nimble formatting generate no nim.cfg file or empty nim.cfg file

[Clonkk]

I upgraded from 2.0.2 to 2.0.8 and notice a severe regression : atlas use no longer generate a correct nim.cfg files.

[Araq]

If it's incorrect, what does it look like?

[Clonkk]

Sorry I was short on details as I opened the issue from my phone to not forget about it.
Here are the complete details.

All these have been tested on OSX ARM CPU (M1 MacBook Pro).

As a test, I run the following command on both version

atlas init
atlas env 2.0.X
cd nim-2.0.X
# this step is necessary because by default ``atlas env`` do not build atlas for the version, not sure if this is intended or not
./build_all.sh 
cd ..
source nim-2.0.X/activate.sh
atlas use scinim
atlas use zmq

This is the output for atlas compiled using : build_all.sh on Nim 2.0.2 (hash c4c44d10df8a14204a75c34e499def200589cb7c)

~/Workspace/Test/AtlasEmpty_2_0_2 % ll
total 12K
drwxr-xr-x 21 regis.caillaud 672 jul 10 10:08 ./
drwxr-xr-x  6 regis.caillaud 192 jul 10 10:07 ../
drwxr-xr-x 18 regis.caillaud 576 jul 10 10:07 Arraymancer/
-rw-r--r--  1 regis.caillaud  76 jul 10 10:08 AtlasEmpty_2_0_2.nimble
-rw-r--r--  1 regis.caillaud  26 jul 10 10:07 atlas.workspace
drwxr-xr-x  9 regis.caillaud 288 jul 10 10:07 nim-clblast/
drwxr-xr-x 13 regis.caillaud 416 jul 10 10:08 nim-zmq/
-rw-r--r--  1 regis.caillaud 396 jul 10 10:08 nim.cfg
drwxr-xr-x 12 regis.caillaud 384 jul 10 10:07 nimblas/
drwxr-xr-x 10 regis.caillaud 320 jul 10 10:08 nimcl/
drwxr-xr-x 13 regis.caillaud 416 jul 10 10:07 nimcuda/
drwxr-xr-x 15 regis.caillaud 480 jul 10 10:07 nimlapack/
drwxr-xr-x 11 regis.caillaud 352 jul 10 10:08 nimpy/
drwxr-xr-x  8 regis.caillaud 256 jul 10 10:08 opencl/
drwxr-xr-x 13 regis.caillaud 416 jul 10 10:07 packages/
drwxr-xr-x 11 regis.caillaud 352 jul 10 10:08 polynumeric/
drwxr-xr-x 13 regis.caillaud 416 jul 10 10:08 scinim/
drwxr-xr-x 10 regis.caillaud 320 jul 10 10:07 stb_image-Nim/
drwxr-xr-x 10 regis.caillaud 320 jul 10 10:08 threading/
drwxr-xr-x  7 regis.caillaud 224 jul 10 10:07 untar/
drwxr-xr-x  9 regis.caillaud 288 jul 10 10:07 zip/
~/Workspace/Test/AtlasEmpty_2_0_2 % cat atlas.workspace
deps=""
resolver="MaxVer"

and

$ cat nim.cfg
############# begin Atlas config section ##########
--noNimblePath
--path:"Arraymancer/src"
--path:"nimpy"
--path:"scinim"
--path:"nim-zmq"
--path:"nimblas"
--path:"nimlapack"
--path:"nimcuda"
--path:"nimcl"
--path:"nim-clblast/src"
--path:"stb_image-Nim"
--path:"zip"
--path:"untar/src"
--path:"threading"
--path:"polynumeric"
--path:"opencl"
############# end Atlas config section   ##########

By default, all the package are on the latest default branch.

=======================================================================================

This is the output for atlas compiled using : build_all.sh on Nim 2.0.8 (hash 5935c3bfa9fec6505394867b23510eb5cbab3dbf)

This is the output:

~/Workspace/Test/AtlasEmpty_2_0_8 % cat atlas.workspace
{
  "deps": "",
  "overrides": "",
  "plugins": "",
  "resolver": "SemVer",
  "graph": null
}%

Right away, the resolver change from MaxVer to SemVer. Because there are issue of old version taken by atlas I also tried the following steps with MaxVer instead of SemVer (no luck : same result).

$ atlas use scinim

[...downloading a bunch of packages... but finish with an error]

[Info] (untar) Cloning url: https://github.com/dom96/untar
Clonage dans '/Users/regis.caillaud/Workspace/Test/AtlasEmpty_2_0_8/untar'...
remote: Enumerating objects: 101, done.
remote: Counting objects: 100% (8/8), done.
remote: Compressing objects: 100% (8/8), done.
remote: Total 101 (delta 2), reused 2 (delta 0), pack-reused 93
Réception d'objets: 100% (101/101), 20.08 Kio | 6.69 Mio/s, fait.
Résolution des deltas: 100% (31/31), fait.
[Info] (opencl) Cloning url: https://github.com/nim-lang/opencl
Clonage dans '/Users/regis.caillaud/Workspace/Test/AtlasEmpty_2_0_8/opencl'...
remote: Enumerating objects: 32, done.
remote: Total 32 (delta 0), reused 0 (delta 0), pack-reused 32
Réception d'objets: 100% (32/32), 20.67 Kio | 20.67 Mio/s, fait.
Résolution des deltas: 100% (8/8), fait.
fatal.nim(53)            sysFatal
Error: unhandled exception: index out of bounds, the container is empty [IndexDefect]

This error seems to come from the nimble file :

~/Workspace/Test/AtlasEmpty_2_0_8 % cat AtlasEmpty_2_0_8.nimble
requires "https://github.com/SciNim/scinim"

=> The above error does not go away as long as the generated nimble is present and stops from generating a nim.cfg at all.

$ atlas use zmq
[Info] (/Users/regis.caillaud/Workspace/Test/AtlasEmpty_2_0_8) is the current workspace
[Info] (/Users/regis.caillaud/Workspace/Test/AtlasEmpty_2_0_8/AtlasEmpty_2_0_8.nimble) updated
[Info] (AtlasEmpty_2_0_8) installing dependencies for AtlasEmpty_2_0_8.nimble
[Info] (nim-zmq) Cloning url: https://github.com/nim-lang/nim-zmq
Clonage dans '/Users/regis.caillaud/Workspace/Test/AtlasEmpty_2_0_8/nim-zmq'...
remote: Enumerating objects: 731, done.
remote: Counting objects: 100% (258/258), done.
remote: Compressing objects: 100% (146/146), done.
remote: Total 731 (delta 128), reused 174 (delta 109), pack-reused 473
Réception d'objets: 100% (731/731), 249.73 Kio | 12.49 Mio/s, fait.
Résolution des deltas: 100% (401/401), fait.
/Users/regis.caillaud/Workspace/Test/AtlasEmpty_2_0_8/nim-zmq/zmq.nimble(14, 1) Error: invalid indentation
/Users/regis.caillaud/Workspace/Test/AtlasEmpty_2_0_8/nim-zmq/zmq.nimble(14, 17) Error: invalid indentation
/Users/regis.caillaud/Workspace/Test/AtlasEmpty_2_0_8/nim-zmq/zmq.nimble(14, 17) Error: expression expected, but found ','

This error comes from atlas taking an old version (not sure why), even with resolver=MaxVer.

Even when fixing that manually (deleting nimble file), the generated nim.cfg at the root of the workspace is empty :

############# begin Atlas config section ##########
--noNimblePath
############# end Atlas config section   ##########

[Clonkk]

Update : this might be package related, since doing

atlas use timelog # just for an example of a very simple package

works and generate a correct nim.cfg.

So to sum-up I see 2 issues :

• resolver taking too old version - IMO unless a git hash / explicit version is provided (assuming I can't figure out how to atlas use a specific version #114 is eventually a feature) taking latest default branch should be the default ?
• An error occurs when doing atlas use scinim that prevents nim.cfg from being generated (IMO shouldn't happen since as long as the package exists atlas should generate the correct config file)

[Clonkk]

This could be related to atlas fetching older version with a nimble file that has an old syntax.

For example, atlas use opencl checkout on commit 90576e57a1672f3ef977bf3db953d0e3943df4d9

cat open.nimble
=> old nimble syntax :

[Package]
name: "opencl"
version: "1.0"
author: "Andreas Rumpf"
description: "Low-level wrapper for OpenCL"
license: "MIT"

srcDir: "src"

[Deps]
requires: "nim > 0.9.2"

So far this seems the common denominator of all package that cause a problem. Did something change between 2.0.2 and 2.0.8 around the handling on nimble file in atlas ?

[Araq]

Nothing changed but we can improve Atlas so that it doesn't even try to parse .nimble files starting with [.

[Clonkk]

Nothing changed but we can improve Atlas so that it doesn't even try to parse .nimble files starting with [.

It would be a first step; something also changed in how version / commit hash are resolved and on 2.0.8 I get older commit (even on MaxVer which is odd) that don't compile.