Python is no longer easy to use for new users

[rbricheno]

PEP668 while likely to reduce the chances of people trashing their system Python also sets up a pretty big hurdle for new Python users to overcome.

The impact of this could be significantly reduced by (for example):

• creating a default per-user venv for each user on the system, and using it be default in their profile
• configuring thonny to use that venv

@gadgetoid propsed something like this for the Pimoroni software, but I suggest that it would be much better to address it at the OS level otherwise everyone will end up having to (learn how to) do this themselves.

There may of course be a better way to address it. Just documenting it doesn't really cut it, venvs are one of the most confusing things you have to deal with in Python. You might say that's a skill issue, but this is supposed to be a computer that the unskilled can learn on.

[Gadgetoid]

My "solution" is ... sort of okay ... for glossing over command-line use of Python within a venv. Unfortunately it falls flat when it comes to making packages installed therein available to Thonny and other IDEs. In these cases the user must always negotiate some tertiary configuration or jump some additional hoops- so no matter how many times we say: "It's easy, just type:"

python3 -m venv --system-site-packages ~/my_venv_dir
source ~/my_venv_dir/bin/activate

We are largely just kidding ourselves that virtual environments are that simple.

Case in point: If venvs are so hip and cool and easy why does Pi OS Bookworm cheat its way around them by letting users loose on an apt-managed, system Python environment via Thonny? Of course anyone who remains a beginner will never really care why basic "blink an LED" Python Just Works, but how are they expected to advance beyond this manicured garden when it's surrounded by a brick wall?

And it's not just that "venvs are hard." It's the dichotomy of "system python" vs "virtual environments". Beginners trying to push the envelope are not just going to wonder why they can't install "Insert Cool Pypi Python Package Here", but wonder even harder why, when they install it to a venv, it stubbornly refuses to import into their Thonny project.

I don't think RPi should push back against this recommendation. But I do think they should lead the charge in papering over this rift. With Ubuntu now being screamingly awesome on Pi 5, what is Pi OS for if not solving these kinds of problems?

The alternative is everyone doing their own thing and causing absolute bedlam.

(ﾉŎꇴŎ)ﾉ✲ﾟ｡⋆

python3 -m venv --system-site-packages --prompt bedlam ~/absolute_bedlam
source ~/absolute_bedlam/bin/activate
pip install thonny
thonny &

[ghollingworth]

The problem with creating a default environment is that it'll also stop the system python3-xxxx applications from working. Also the default environment then won't be valid if you do "sudo pip install" and if it is (because you use sudo -E pip install) then it will end up writing root owned files into the user's .venv directory which will also break the user's view.

There is no solution to this problem which makes it easier without also breaking it in some subtle way which is not the same as any other linux platform.

There are other options than python3 -m venv such as virtualenvwrapper

$ sudo apt install python3-virtualenv
$ sudo apt install python3-virtualenvwrapper

$ cat << EOL >> .bashrc
export WORKON_HOME=~/.virtualenvs
source /usr/share/virtualenvwrapper/virtualenvwrapper.sh
EOL

Now start a new terminal and do:

$ mkvirtualenv myproject
$ workon myproject

And you're done, I was thinking of installing the bashrc changes by default so we could add the last two lines to tutorials...

[Gadgetoid]

The wrapper would be a good call, since venv is a little cryptic and requires you to remember the explicit full path of your “bin/activate” in each case.

Doesn’t much help Thonny users running into the brick wall that divides system and venv Python though- I’d have been tempted to make the Thonny shortcut a shell script that boots it out of a venv named “thonny” and recommend “workon thonny” to users wishing to side load Pypi packages. I’m sure there are just as many different flaws with this approach.

Maybe some GUI view into available projects (which I think Ben suggested on the forums) with a “open a terminal” or “open Thonny” button would be a less destructive approach that would work in concert with venvwrapper. That’s something I could try prototyping, since I’m not about to come in here and rock the boat without offering some proof of concept 🤣 (though I am officially off today for baby’s first birthday)

Bonus points for somehow being able to use the single system wide Thonny and launch it into different venvs, rather than installing it into each.

[ghollingworth]

I was under the impression that Thonny had always used it's own venv to avoid exactly this problem...

Gordon

[aallan]

I was under the impression that Thonny had always used it's own venv to avoid exactly this problem...

That was my understanding as well? I think you can use the system Python but the default Python is inside a venv.

Yes. Can confirm, just spun up Thonny and the default Python distribution is "Thonny's Python" and it lives in /Applications/Thonny.app/Contents/Frameworks/Python.framework/Versions/3.10/bin/python3.10 on my Mac.

[Gadgetoid]

If I switch Thonny into normal mode and try to install a package via it's own UI, I get:

(Yes I've no earthly idea which screenshot utility to use right now 😆 )

[aallan]

If I switch Thonny into normal mode and try to install a package via it's own UI, I get:

If you look on your screenshot you'll see "Local Python 3 /usr/bin/python" in the bottom right of the Thonny interface, you're trying to install into the system Python.

Poking around, unlike macOS and Windows, on the Pi Thonny defaults to the local system environment, rather than a virtual enviroment. However you can spin up a venv from the settings.

Click on the bottom right interpreter line and choose "Configure the interpreter".

Then select "New virtual environment". After spinning one up it'll be available as part of the drop down in the lower right

(Yes I've no earthly idea which screenshot utility to use right now 😆 )

That'd be grim.

[aallan]

@ghollingworth I think Thonny not defaulting to a venv in a post-Bookworm world probably qualifies as "a bug" at this point. I'm also almost entirely convinced it used to default to a venv, so I'm not sure what happened here?

In any case I'll throw this into the documentation.

[Gadgetoid]

I think Thonny not defaulting to a venv in a post-Bookworm world probably qualifies as "a bug"

YES!

Also, come on. Telling me how to use Thonny? Was there any point at which you were typing that and thinking "well if I have to explain this to a guy with over ten years experience shipping Python libraries to our customers, maybe there's something wrong?" 🤦

[aallan]

Also, come on. Telling me how to use Thonny? Was there any point at which you were typing that and thinking "well if I have to explain this to a guy with over ten years experience shipping Python libraries to our customers, maybe there's something wrong?" 🤦

It was more "for the record" than for you. This thread will turn up in Google. 🤷‍♂️

[aallan]

Just added a section to the documentation, and have pushed that to the deployed site. Should be live soon.

[pelwell]

This thread will turn up in Google. 🤷‍♂️

This.

[Gadgetoid]

None of this really fixes the wider issues of the rift between the Thonny environment and all other interactions with the Pi, however.

If Thonny is entirely self-contained than how do users trying to transition away from Thonny's walled-garden:

• Run their script from the command-line
• Run their script on startup
• Install Python libraries into their Thonny environment from command-line (via helper scripts or from git repositories)
• Edit their Thonny scripts in another IDE

And that's saying nothing about those who don't want to use Thonny at all.

It looks like VSCode looks in ~/.virtualenvs and $WORKON_HOME for Virtualenvwrapper.

I'd strongly suggest the following packages are preinstalled-

• python3-virtualenvwrapper
• virtualendwrapper

I've confirmed that installing VSCode allows it to find venvs here, and using "Select Interpreter" will let you choose one. The whole "Pi 5 runs vscode super well" thing is a feather in its cap, so it seems prudent to lean into what with whatever solution we choose.

Unfortunately I don't believe Thonny knows or cares about ~/.virtualenvs and $WORKON_HOME, but if a giant like VSCode is supporting ~/.virtualenvs then it would seem prudent to encourage Thonny to do the same so they are listed when you use "Configure Interpreter"

How close is Pi's relationship to the Thonny devs? Can this kind of change be eked out of them with any kind of urgency, handled downstream?

Finally, if we're agreeing about a standard path for virtual environments and trying to encourage that via distro choices it would make a basic "Manage Python Environments" GUI a lot easier to throw together and a lot more practical.

This thread will turn up in Google

We're having a conversation about an issue that affects multiple thousands customers ability to install software, and greatly affects how I approach developing and shipping our software in future, and you're trying to SEO optimise it? After giving me an "it works like this on my mac" response? 😵‍💫

This.

I honestly have no words.