TypeError in virtualenv 20.33.0: expected string or bytes-like object, got 'list'

[tomeroszlak]

Issue

We're experiencing a breaking issue with virtualenv==20.33.0 when used by Poetry to create ephemeral environments. The following error occurs:

TypeError: expected string or bytes-like object, got 'list'

The traceback shows the error originates from:

  File ".venv/lib/python3.11/site-packages/virtualenv/activation/via_template.py", line 33, in quote
    return shlex.quote(string)

It appears that a list is being passed into shlex.quote() during activation template generation, instead of a string.

Expected behavior: virtualenv should sanitize or validate the inputs to shlex.quote() to ensure they are strings.

This does not happen in version 20.32.0 — pinning to that version resolves the issue. Only 20.33.0 is affected.

Environment

Provide at least:

• OS: Ubuntu 20.04 (Dockerized Jenkins agent running as root)

• pip list of the host python where virtualenv is installed:

  Package         Version
  --------------- -------
  pip             24.0
  poetry          1.8.2
  setuptools      69.2.0
  virtualenv      20.33.0

Output of the virtual environment creation

Make sure to run the creation with -vvv --with-traceback:

[virtualenv:virtualenv.run.session] ============================== target debug ==============================
[virtualenv:virtualenv.run.session] add activators for Bash, CShell, Fish, Nushell, PowerShell, Python

  Stack trace:

   ...
   File ".venv/lib/python3.11/site-packages/virtualenv/activation/via_template.py", line 33, in quote
     return shlex.quote(string)

TypeError: expected string or bytes-like object, got 'list'

Notes

• Python version: 3.11.9 (via pyenv)
• Fails during installation of a package via Poetry
• Works when downgrading to virtualenv 20.32.0

[eacherkan-via]

Following @tomeroszlak's report, here's the relevant output of poetry install -vvv:

   8  .venv/lib/python3.11/site-packages/poetry/utils/env/env_manager.py:551 in build_venv
       549│         args.append(str(path))
       550│ 
     → 551│         cli_result = virtualenv.cli_run(args, setup_logging=False)
       552│ 
       553│         # Exclude the venv folder from from macOS Time Machine backups

   7  .venv/lib/python3.11/site-packages/virtualenv/run/__init__.py:33 in cli_run
        31│     of_session = session_via_cli(args, options, setup_logging, env)
        32│     with of_session:
     →  33│         of_session.run()
        34│     return of_session
        35│ 

   6  .venv/lib/python3.11/site-packages/virtualenv/run/session.py:48 in run
        46│         self._create()
        47│         self._seed()
     →  48│         self._activate()
        49│         self.creator.pyenv_cfg.write()
        50│ 

   5  .venv/lib/python3.11/site-packages/virtualenv/run/session.py:67 in _activate
        65│             LOGGER.info("add activators for %s", active)
        66│             for activator in self.activators:
     →  67│                 activator.generate(self.creator)
        68│ 
        69│     def __enter__(self):

   4  .venv/lib/python3.11/site-packages/virtualenv/activation/via_template.py:38 in generate
        36│         dest_folder = creator.bin_dir
        37│         replacements = self.replacements(creator, dest_folder)
     →  38│         generated = self._generate(replacements, self.templates(), dest_folder, creator)
        39│         if self.flag_prompt is not None:
        40│             creator.pyenv_cfg["prompt"] = self.flag_prompt

   3  .venv/lib/python3.11/site-packages/virtualenv/activation/via_template.py:57 in _generate
        55│         generated = []
        56│         for template in templates:
     →  57│             text = self.instantiate_template(replacements, template, creator)
        58│             dest = to_folder / self.as_name(template)
        59│             # remove the file if it already exists - this prevents permission

   2  .venv/lib/python3.11/site-packages/virtualenv/activation/via_template.py:79 in instantiate_template
        77│         for key, value in replacements.items():
        78│             value_uni = self._repr_unicode(creator, value)
     →  79│             text = text.replace(key, self.quote(value_uni))
        80│         return text
        81│ 

   1  .venv/lib/python3.11/site-packages/virtualenv/activation/via_template.py:33 in quote
        31│         :return: quoted string that works in the activation script
        32│         """
     →  33│         return shlex.quote(string)
        34│ 
        35│     def generate(self, creator):

  TypeError

  expected string or bytes-like object, got 'list'

  at ~/.pyenv/versions/3.11.9/lib/python3.11/shlex.py:329 in quote
      325│ def quote(s):
      326│     """Return a shell-escaped version of the string *s*."""
      327│     if not s:
      328│         return "''"
    → 329│     if _find_unsafe(s) is None:
      330│         return s
      331│ 
      332│     # use single quotes, and put single quotes into double quotes
      333│     # the string $'b is then quoted as '$'"'"'b'

[esafak]

Is the environment variableTCL_LIBRARY defined where this happens?

[tomeroszlak]

@esafak No, TCL_LIBRARY is not defined

[esafak]

Can you also show me how you created the virtualenv with poetry? I cloned a simple repo then ran

poetry env use python
poetry shell
poetry install

but I don't see virtualenv in it:

> pip list
Package                 Version Editable project location
----------------------- ------- ---------------------------------------
iniconfig               2.1.0
loguru                  0.7.3
packaging               25.0
pip                     25.1.1
pluggy                  1.5.0
poetry_tutorial_project 0.1.0   /Users/emre/code/python-poetry-tutorial
psutil                  7.0.0
pytest                  8.3.5

[daviddmd]

I can reproduce this issue with the latest version of virtualenv (20.33.0):

user@hostname:~$ uvx virtualenv==20.33.0 /home/user/venv
TypeError: expected string or bytes-like object, got 'list'

user@hostname:~$ python3 -m venv /home/user/venv
user@hostname:~$ source home/user/venv/bin/activate
(venv) user@hostname:~$ pip install virtualenv==20.33.0
(venv) user@hostname:~$ which virtualenv
/home/user/venv/bin/virtualenv
(venv) user@hostname:~$ virtualenv /home/user/test_venv
TypeError: expected string or bytes-like object, got 'list'

The same occurs with a virtual environment created by a system-wide virtualenv. I've tested this on Python 3.12.3 and Python 3.13.5 on Linux.

[RobKellyAnsys]

Also get this issue since upgrading to 20.33.0 when trying to install black via pre-commit on a github runner with a poetry venv. Downgrading solved the issue.

[mjhuber]

i'm also experiencing this issue on 20.33.0, downgrading to 20.32.0 solved the issue for me

[Mattervispa]

For me it is reproducible if you try to create a second virtual environment within the same OS env (e.g. in a single bash script).

We use a single Gitlab stage to build the RPM and run the audit. To avoid conflicts, we use two different directories. This works up to 20.32, but 20.33 breaks it.

Minimal steps to reproduce:

create virtualenv
activate
deactivate
try to create 2nd virtualenv
TypeError: expected string or bytes-like object

Note: you MUST activate / deactivate to reproduce the issue

details (Everything is running in a Redhat 8 docker):

./run-build.sh
setup venv
real path of <venv> is: /opt/company/py_app
find interpreter for spec PythonSpec(path=/usr/bin/python3.9)
proposed PythonInfo(spec=CPython3.9.20.final.0-64, exe=/usr/bin/python3.9, platform=linux, version='3.9.20 (main, Oct 23 2024, 13:02:27) \n[GCC 8.5.0 20210514 (Red Hat 8.5.0-22)]', encoding_fs_io=utf-8-utf-8)
create virtual environment via CPython3Posix(dest=/opt/company/py_app, clear=False, no_vcs_ignore=False, global=False)
add seed packages via FromAppData(download=False, pip=bundle, setuptools=bundle, via=copy, app_data_dir=/home/developer/.local/share/virtualenv)
add activators for Bash, CShell, Fish, Nushell, PowerShell, Python
created virtual environment CPython3.9.20.final.0-64 in 297ms
  creator CPython3Posix(dest=/opt/company/py_app, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, via=copy, app_data_dir=/home/developer/.local/share/virtualenv)
    added seed packages: pip==25.1.1, setuptools==80.9.0
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
run audit
setup venv
real path of <audit venv> is: /opt/company/audit-py_app
find interpreter for spec PythonSpec(path=/usr/bin/python3.9)
proposed PythonInfo(spec=CPython3.9.20.final.0-64, exe=/usr/bin/python3.9, platform=linux, version='3.9.20 (main, Oct 23 2024, 13:02:27) \n[GCC 8.5.0 20210514 (Red Hat 8.5.0-22)]', encoding_fs_io=utf-8-utf-8)
create virtual environment via CPython3Posix(dest=/opt/company/audit-py_app, clear=False, no_vcs_ignore=False, global=False)
add seed packages via FromAppData(download=False, pip=bundle, setuptools=bundle, via=copy, app_data_dir=/home/developer/.local/share/virtualenv)
add activators for Bash, CShell, Fish, Nushell, PowerShell, Python
TypeError: expected string or bytes-like object```

**TypeError: expected string or bytes-like object**

for env creation we use: 

```shell
python3.9 -m virtualenv --verbose --python="/usr/bin/python3.9" "${venv_dir}"

[esafak]

1. Spot the bug:

   virtualenv/src/virtualenv/discovery/py_info.py

   Line 126 in 08e5dfa

   self.tcl_lib, self.tk_lib = self._get_tcl_tk_libs() if "TCL_LIBRARY" in os.environ else None, None

2. Can people who ran into this error after creating multiple virtualenvs tell me what shell you were using? And can you check if TCL_LIBRARY became set before the run that failed?