Managing Virtual Environments on Python
With the exception of small scripts and programs that only use built-in libraries, pretty much the first step before starting to work in a new Python project is the setup of a virtualenv. As a reminder, virtual environments provide you:
- Dependency isolation: A requires libX v2 and B requires v3? If you get into that situation without venvs, you already lost. See you in DLL hell ;)
- Stable development environment: With requirements.txt (or Pipfile.lock for Pipenv) libraries stay put until you update them explicitly. Useful to avoid finding surprises when you or a colleague download a project you began 1 year ago and where many dependencies got a major version update.
- Non-root package installation: All the Linux distros I’ve used require root privileges to install packages globally. You can do
pip install --user
, but you still have the two previous problems.
There are already enough tutorials about using the virtualenv
command, and I personally no longer use it directly. We’ll cover 3 4 common use cases, in a kind of continuation of the previous post.
- Global Python install & virtualenvwrapper
- Pyenv & pyenv-virtualenvwrapper
- Pyenv & Pipenv (highly recommended)
- Global Python install & Pipenv (a good compromise)
Global Python install and virtualenvwrapper
You can install virtualenvwrapper before or after installing your desired Python version somewhere in your system. This assumes Ubuntu 16.04LTS, like the previous post.
The installation consists of 5 steps. You do this only once:
pip3 install virtualenvwrapper
echo 'export WORKON_HOME=~/.virtualenvs' >> ~/.bashrc
echo 'source /usr/local/bin/virtualenvwrapper.sh' >> ~/.bashrc
sed -i 's/h python/h python3/1' /usr/local/bin/virtualenvwrapper.sh
source ~/.bashrc
By default virtualenvwrapper.sh tries to execute python
to load its module, but as we used pip3
to install it, it will not find it. We change it using sed
so it uses python3
instead.
Using virtualenvwrapper is simple, and you can find documentation at: virtualenvwrapper quickstart and in many other tutorials.
Creating a new virtual environment using mkvirtualenv
will use Python 3.5 by default (that’s to what python3
is linked to). If we want to use Python 3.6, we can pass it as an argument to mkvirtualenv
:
$ mkvirtualenv -p python3.6 myEnv
mkvirtualenv -p python3.6 myEnv
... other output ...
(myEnv) $ python --version
Python 3.6.4
If you try to use a non-installed version, it will fail:
$ mkvirtualenv -p python3.7 myOtherEnv
The path python3.7 (from --python=python3.7) does not exist
Pyenv and pyenv-virtualenvwrapper
For installing pyenv check the previous post.
For ease of use, pyenv provides a plugin called [pyenv-virtualwrapper][]. It requires at least one version of Python installed and set.
$ pyenv install 3.6.4
$ pyenv global 3.6.4
You could use pyenv shell
or pyenv local
too in the previous example.
Installing the plugin it’s just a matter of cloning the Github repository:
$ git clone https://github.com/pyenv/pyenv-virtualenvwrapper.git $(pyenv root)/plugins/pyenv-virtualenvwrapper
Running pyenv virtualenvwrapper
will setup virtualenvwrapper in your current shell. If virtualenvwrapper is not installed, then it will install it in the currently set Python; if it is already installed, then it will just make the virtualenvwrapper commands available, which are the same as usual.
Any new virtual environment created will use the currently set Python by default. You can use a different version with the -p
flag, but it requires you to set multiple versions at once, as it’s described [here]
(https://github.com/pyenv/pyenv/blob/master/COMMANDS.md#pyenv-shell-advanced)
.
$ pyenv shell 3.6.4 2.7.6
$ pyenv virtualenvwrapper
$ mkvirtualenv -p python2.7 myOldie
...output...
(myOldie) $ mkvirtualenv myNewie
...output...
In the creation of myNewie we have myOldie active, but it makes no difference. It’s just that the previous command activates that virtual environment after creating it.
Pyenv and Pipenv
Pipenv is a Pyenv aware tool to manage both the creation of virtualenvs and the installation of packages using pip at the same time.
Just by doing pipenv install
, it automatically creates a new virtualenv if one doesn’t exist, generates requirements.txt-like files and installs your desired Python version in the virtualenv for you (even if you don’t have it installed), if you use Pyenv.
- To install: just run
pip install pipenv
with a Python set on the current shell. - To install a package:
pipenv install thepkg
. As said before, this will create a new virtualenv for the “project” if one doesn’t exist. It will use the currently set Python. - To install a package, initializing the virtualenv with a specific version of python:
pipenv --python=2.7.14 thepkg
Then you can activate the virtual environment by running pipenv shell
.
Additionally, you can set the PIPENV_VENV_IN_PROJECT
environment variable to 1
to install the virtual environments together with the projects (same directory as Pipfile and Pipfile.lock). If you come from .NET like I do, surely you’ll like it too as then it becomes like NuGet.
More documentation can be found at the Pipenv page.
Global Python install and Pipenv
Finally, if you like Pipenv, and you already have a Python installation that you want to use, note that’s possible to do so by running: pipenv --python=python3.6 thepkg
.
Closing notes
I hope this post gives you an idea of what are some of the options at your disposal when it comes to creating virtual environments. Missing however is using virtualenv
directly, and it may be a good option to do so if you want to reduce your dependencies as much as reasonably possible.
Personally, I prefer using Pipenv together with Pyenv, as they complement each other very nicely, and if you want to test your code in several Python versions, you are bound to run into Pyenv anyways.
In a future post, we’ll cover how to deploy a Flask webapp into an Ubuntu VPS, following what we have done until now.