Frequently we will need to import a package that is not part of the core Python installation. For example, my Advent of Code solutions often make use of installable packages like:
To install packages like these, we need to use a Package Manager. A package manager provides the ability to install and uninstall packages. But in addition, a package manager resolves dependencies. This means that if you want to install package
x, but package
x depends on package
y, then the package manager will automatically install both
y, in the right order.
Python comes pre-installed with its own package manager, called pip. To use pip, simply type
pip <command>, e.g.
# install matplotlib pip install matplotlib # uninstall matplotlib pip uninstall matplotlib # upgrade matplotlib pip install --upgrade matplotlib # upgrade pip itself pip install --upgrade pip
Although you can just use
pip <command> as shown above, it’s generally recommended to instead use this syntax:
py -m pip <command>
This syntax ensures that pip installs the package to the currently active Python runtime or environment.
Virtual environments are isolated Python contexts in which we can install Python packages. In short, virtual environments isolate dependencies of Python projects.
Well, you might be working on project
a that requires packages
x, and package
x depends on package
y. And then you might start working on project
b, which needs package
z. But package
z won’t work if you have package
y installed. So your two projects have incompatible package requirements.
The answer? Virtual environments! You simply create a dedicate virtual environment for each project.
Here are some reasons why virtual environments are a good thing:
venvcommand. For example, to create a virtual environment called
py -m venv .my-proj-venv
.my-proj-envto your project’s
.gitignore, since we don’t want it included in version control.
In case it’s useful, here’s the
.gitignore file I tend to use on Python projects:
# Env .*-env/ .env # Build / distribution / packaging .Python build/ dist/ sdist/ var/ target/ pip-log.txt pip-delete-this-directory.txt # Byte-compiled / optimized / DLL files __pycache__ .mypy_cache/ # Build docs docs/_build/ docs/vendor/ # Unit test / coverage reports htmlcov/ .tox/ .coverage .coverage.* .cache nosetests.xml coverage.xml *.cover # Jupyter stuff .ipynb_checkpoints/ .ipython/ .cache/ # secrets *.key .gitsecret/keys/random_seed # Random .vscode *.log desktop.ini message.txt snippets/ output/
You need to activate the virtual environment, every time you want to use it. I.e. every time you’re working in an associated Python project.
Once activated, any
pip installs you perform will be done within the virtual environment.
pip install numpy
Fortunately, if you’re using VS Code, then VS Code will automatically detect your virtual environment the next time you open your project folder. From then onwards, it will automatically activate the virtual environment for you.
Here are my general thoughts on how to work with virtual environments:
requirements.txtfile that captures all your currently installed packages for a given project. And whenever you install / update / remove any packages from your virtual environment, refresh this file. To create / refresh your
requirements.txt, just run this:
py -m pip freeze > path/to/your/requirements.txt
requirements.txtshould be under version control.
py -m pip install -r path/to/your/requirements.txt