bug: vivid: home.sessionVariables and yaml vs scientific notation

[arunoruto]

Are you following the right branch?

• My Nixpkgs and Home Manager versions are in sync

Is there an existing issue for this?

• I have searched the existing issues

Issue description

This issue serves more as a discussion point for two things we have noticed in nix-community/stylix#1938

1. @trueNAHO has noticed that we could use pkgs.runCommand and home.sessionVariables instead of configuring each shell by itself, meaning the config files of the shells won't be cluttered additionally and we don't need the vivid package in the environment. The only problem I have with this, is that the user (potentially?) needs to restart the system for a change to take effect. I am not sure how much of a problem that would be? We could introduce an additional option programs.vivid.enableSessionVariable and configure it through that too! Here is a POC which works on my system: vivid: init stylix#1938 (comment)
2. The testbed in stylix had a particular hex code for a color, namely 1e2030. Since it is treated as a string by Nix and pkgs.remarshal's json2yaml, it is not written with single quotes. But by YAML standards, it is interpreted as a number in scientific notation, leading to vivid failing to load the theme correctly. This is kinda a niche edge case but not so easy to debug. A potential solution for this would be to save the contents as a JSON file, but keeping the .yml file ending. It is kinda weird, but it works.. Maybe an additional option like programs.vivid.saveAsJson could be provided for now (and maybe turn it on by default?). I haven't found any better solution for it now and I notified the dev via an issue, but by the looks of how often he responds, this will not be addressed/fixed in the foreseeable future. Meaning, we could exploit this "bug" and use JSON for now :)

Maintainer CC

@aguirre-matteo

System information

- system: `"x86_64-linux"`
 - host os: `Linux 6.12.50, NixOS, 25.05 (Warbler), 25.05.20251006.20c4598`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.28.5`
 - nixpkgs: `/nix/store/d7s7pkgp0shkac0qwf2f9w956fi6rh1i-source`

[aguirre-matteo]

I just took a look at the discussion. IMO running vivid generate at evaluation time seems like a great idea, so the shell init gets a bit faster. Though, I'm not really getting why you're so worried about making vivid a runtime dependency, @trueNAHO. When it comes to the YAML issue, I locally tested adding single quotes both at the end and the beginning of the hex code, and it led the module to put triple single quotes around the hex code:

{
  colors = {
    primary = "'00aaff'"; # <-----
    secondary = "ff00aa";
  };
  core = {
    directory = {
      foreground = "primary";
    };
    "executable-file" = {
      foreground = "secondary";
      "font-style" = "bold";
    };
  };
}

--- actual
+++ expected
@@ -1,5 +1,5 @@
 colors:
-  primary: '''00aaff'''
+  primary: 00aaff

We could then replace those triple quotes to single ones so the file falls under YAML's standards. This to me seems a bit over-engineered, and we still would have to find a way to know which attributes from cfg.themes.<name> (that are defined as attribute sets, ofc) should be enclosed by the quotes. AFAICS, only the attributes under the colors: section would have to be enclosed. This, to me, seems preferable in the long run rather than exploiting a bug, since otherwise we would have to address this issue in the future. However, we should be aware that the problem lies in the YAML generator not quoting those strings, so this problem could be affecting other programs than vivid. That is out of the scope of this issue, but still I feel like it should be said.

I don't think adding new options would be necessary as long as we keep backwards compatibility. By the way, @arunoruto, if you're willing to maintain the module in the long term, feel free to add yourself as a maintainer.

[arunoruto]

I just took a look at the discussion. IMO running vivid generate at evaluation time seems like a great idea, so the shell init gets a bit faster.
I don't think adding new options would be necessary as long as we keep backwards compatibility.

We can just make the the home.sessionVariables.LS_COLORS the default option and turn off the shell integrations by default. If a user wants to have fast switching of the theme at every rebuild, they can turn on the respective shell integration. We can communicate this through the description I guess.

By the way, @arunoruto, if you're willing to maintain the module in the long term, feel free to add yourself as a maintainer.

Sure! Will do so at the next PR! I haven't contributed much to home-manager so I am not familiar with common practices yet (took me also some time in nixpkgs). Thats why I am asking for feedback!

When it comes to the YAML issue, I locally tested adding single quotes both at the end and the beginning of the hex code, and it led the module to put triple single quotes around the hex code:

{
colors = {
primary = "'00aaff'"; # <-----
secondary = "ff00aa";
};
core = {
directory = {
foreground = "primary";
};
"executable-file" = {
foreground = "secondary";
"font-style" = "bold";
};
};
}

--- actual
+++ expected
@@ -1,5 +1,5 @@
 colors:
-  primary: '''00aaff'''
+  primary: 00aaff

We could then replace those triple quotes to single ones so the file falls under YAML's standards. This to me seems a bit over-engineered, and we still would have to find a way to know which attributes from cfg.themes.<name> (that are defined as attribute sets, ofc) should be enclosed by the quotes. AFAICS, only the attributes under the colors: section would have to be enclosed. This, to me, seems preferable in the long run rather than exploiting a bug, since otherwise we would have to address this issue in the future.

I guess if we are going with running the command at evaluation time, we can tweak the contents of the file before it is written. I will try to make a POC tomorrow when I have time, shouldn't be much of a problem using lib.attrsets.mapAttrs (name: value: "'${value}'") colors on the colors section and then sed to replace "' and '" with single quotes respectively.

However, we should be aware that the problem lies in the YAML generator not quoting those strings, so this problem could be affecting other programs than vivid. That is out of the scope of this issue, but still I feel like it should be said.

That is true! I guess having an option to put all strings in quotes would be preferable to avoid such mistakes, but I couldn't such an option in remarshal's docs.

[trueNAHO]

1. @trueNAHO has noticed that we could use pkgs.runCommand and home.sessionVariables instead of configuring each shell by itself, meaning the config files of the shells won't be cluttered additionally and we don't need the vivid package in the environment. The only problem I have with this, is that the user (potentially?) needs to restart the system for a change to take effect. I am not sure how much of a problem that would be? We could introduce an additional option programs.vivid.enableSessionVariable and configure it through that too!

For reference, this refers to the following comment from the original discussion:

The only problem I have with this approach is that I need to reboot my system so the sessionVariables take effect. I am not sure how much this will affect UX.

-- nix-community/stylix#1938 (comment)

Since this is the expected behavior for all home.sessionVariables values, this seems totally fine to me. Technically, this only requires logging out and back in for the environment variables to be reloaded.

2. [...] A potential solution for this would be to save the contents as a JSON file, but keeping the .yml file ending. It is kinda weird, but it works..

If the JSON format works, then let's use that.

I always use builtins.toJSON in my personal projects because JSON is more reliably parsed than YAML, and because it has a builtins.fromJSON, unlike YAML. Although for "real" public projects, YAML should ideally not be simplified to JSON to avoid confusion. However, since YAML seems to cause problems, let's just call the JSON file .yml.

I stumbled upon a "bug" in vivid, where the filetype needs to be .y[a]ml, but the contents can be in JSON format too! By using builtins.toJSON to parse the attrset to a string and writing it with pkgs.writeText to a file, it magically works! I opened up an issue at vivid to see if this is intended behavior or just a fluke.

-- nix-community/stylix#1938 (comment)

Since JSON is a YAML subset, JSON can be parsed as YAML. This is the expected and intended behavior.

I notified the dev via an issue

I suppose you mean: sharkdp/vivid#166.

I'm not really getting why you're so worried about making vivid a runtime dependency, @trueNAHO.

Initially, I only considered improving the shell startup time by running vivid generate at evaluation time. Based on #6045 (comment), removing the optional runtime dependency seems like the final stage of running vivid generate at evaluation time. So we might as well drive this concept to its logical conclusion, so that it is definitely done.

let
  [...]
  # mod-config = lib.attrsets.overrideExisting config {
  mod-config = {
    home.shell = {
      enableBashIntegration = false;
      enableZshIntegration = false;
      enableFishIntegration = false;
    };
  };
  [...]
in
{
  [...]
  options.programs.vivid = {
    [...]
    enableBashIntegration = mkBashIntegrationOption { config = mod-config; };
    enableZshIntegration = mkZshIntegrationOption { config = mod-config; };
    enableFishIntegration = mkFishIntegrationOption { config = mod-config; };
    [...]
  };

  config =
    [...]
    mkMerge [
      (lib.mkIf (cfg.enable || (cfg.enableSessionVariables && cfg.themes != { })) {
        home.sessionVariables.LS_COLORS = "$(cat ${
          [...]
        })";
      })
      (mkIf cfg.enable {
        [...]
        programs.bash.initExtra = mkIf cfg.enableBashIntegration ''
        [...]
        programs.zsh.initContent = mkIf cfg.enableZshIntegration ''
        [...]
        programs.fish.interactiveShellInit = mkIf cfg.enableFishIntegration ''
        [...]
      })
    ];
}

I disabled the shell integrations, to make sure they are not setting the variable.

-- nix-community/stylix#1938 (comment)

Why not just remove the shell integrations? They are only used to set LS_COLORS, which is supersed by home.sessionVariables.LS_COLORS now.

[aguirre-matteo]

Why not just remove the shell integrations? They are only used to set LS_COLORS, which is supersed by home.sessionVariables.LS_COLORS now.

Yes, once we tweak the module to use home.sessionVariables then the shell integration options will be removed.

Since JSON is a YAML subset, JSON can be parsed as YAML. This is the expected and intended behavior.

Sorry, didn't knew that. In that case I guess there's no too much problem in just calling the file .yml, yet we should probably add a comment in the module's code so anyone that reviews the code in the future knows the reasons behind the design choices.