This guide discusses how to install packages using pip and
a virtual environment manager: either venv for Python 3 or virtualenv
for Python 2. These are the lowest-level tools for managing Python
packages and are recommended if higher-level tools do not suit your needs.
Note
This doc uses the term package to refer to a
Distribution Package which is different from an Import
Package that which is used to import modules in your Python source code.
Installing pip¶
pip is the reference Python package manager. It’s used to install and
update packages. You’ll need to make sure you have the latest version of pip
installed.
Unix/macOS
Debian and most other distributions include a python-pip package; if you
want to use the Linux distribution-provided versions of pip, see
Installing pip/setuptools/wheel with Linux Package Managers.
You can also install pip yourself to ensure you have the latest version. It’s
recommended to use the system pip to bootstrap a user installation of pip:
python3 -m pip install --user --upgrade pip python3 -m pip --version
Afterwards, you should have the latest version of pip installed in your
user site:
pip 21.1.3 from $HOME/.local/lib/python3.9/site-packages (python 3.9)
Windows
The Python installers for Windows include pip. You can make sure that pip is
up-to-date by running:
py -m pip install --upgrade pip py -m pip --version
Afterwards, you should have the latest version of pip:
pip 21.1.3 from c:\python39\lib\site-packages (Python 3.9.4)
Installing virtualenv¶
Note
If you are using Python 3.3 or newer, the venv
module is
the preferred way to create and manage virtual environments.
venv is included in the Python standard library and requires no additional installation.
If you are using venv, you may skip this section.
virtualenv is used to manage Python packages for different projects.
Using virtualenv allows you to avoid installing Python packages globally
which could break system tools or other projects. You can install virtualenv
using pip.
Unix/macOS
python3 -m pip install --user virtualenv
Windows
py -m pip install --user virtualenv
Creating a virtual environment¶
venv (for Python 3) and virtualenv (for Python 2) allow
you to manage separate package installations for
different projects. They essentially allow you to create a “virtual” isolated
Python installation and install packages into that virtual installation. When
you switch projects, you can simply create a new virtual environment and not
have to worry about breaking the packages installed in the other environments.
It is always recommended to use a virtual environment while developing Python
applications.
To create a virtual environment, go to your project’s directory and run
venv. If you are using Python 2, replace venv
with virtualenv
in the below commands.
The second argument is the location to create the virtual environment. Generally, you
can just create this in your project and call it env
.
venv will create a virtual Python installation in the env
folder.
Note
You should exclude your virtual environment directory from your version
control system using .gitignore
or similar.
Activating a virtual environment¶
Before you can start installing or using packages in your virtual environment you’ll
need to activate it. Activating a virtual environment will put the
virtual environment-specific
python
and pip
executables into your shell’s PATH
.
You can confirm you’re in the virtual environment by checking the location of your
Python interpreter:
It should be in the env
directory:
Unix/macOS
Windows
...\env\Scripts\python.exe
As long as your virtual environment is activated pip will install packages into that
specific environment and you’ll be able to import and use packages in your
Python application.
Leaving the virtual environment¶
If you want to switch projects or otherwise leave your virtual environment, simply run:
If you want to re-enter the virtual environment just follow the same instructions above
about activating a virtual environment. There’s no need to re-create the virtual environment.
Installing packages¶
Now that you’re in your virtual environment you can install packages. Let’s install the
Requests library from the Python Package Index (PyPI):
Unix/macOS
python3 -m pip install requests
Windows
py -m pip install requests
pip should download requests and all of its dependencies and install them:
Collecting requests Using cached requests-2.18.4-py2.py3-none-any.whl Collecting chardet<3.1.0,>=3.0.2 (from requests) Using cached chardet-3.0.4-py2.py3-none-any.whl Collecting urllib3<1.23,>=1.21.1 (from requests) Using cached urllib3-1.22-py2.py3-none-any.whl Collecting certifi>=2017.4.17 (from requests) Using cached certifi-2017.7.27.1-py2.py3-none-any.whl Collecting idna<2.7,>=2.5 (from requests) Using cached idna-2.6-py2.py3-none-any.whl Installing collected packages: chardet, urllib3, certifi, idna, requests Successfully installed certifi-2017.7.27.1 chardet-3.0.4 idna-2.6 requests-2.18.4 urllib3-1.22
Installing specific versions¶
pip allows you to specify which version of a package to install using
version specifiers. For example, to install
a specific version of requests
:
Unix/macOS
python3 -m pip install 'requests==2.18.4'
Windows
py -m pip install "requests==2.18.4"
To install the latest 2.x
release of requests:
Unix/macOS
python3 -m pip install 'requests>=2.0.0,<3.0.0'
Windows
py -m pip install "requests>=2.0.0,<3.0.0"
To install pre-release versions of packages, use the --pre
flag:
Unix/macOS
python3 -m pip install --pre requests
Windows
py -m pip install --pre requests
Installing from source¶
pip can install a package directly from source, for example:
Unix/macOS
cd google-auth python3 -m pip install .
Windows
cd google-auth
py -m pip install .
Additionally, pip can install packages from source in
development mode,
meaning that changes to the source directory will immediately affect the
installed package without needing to re-install:
Unix/macOS
python3 -m pip install --editable .
Windows
py -m pip install --editable .
Installing from version control systems¶
pip can install packages directly from their version control system. For
example, you can install directly from a git repository:
google-auth @ git+https://github.com/GoogleCloudPlatform/google-auth-library-python.git
For more information on supported version control systems and syntax, see pip’s
documentation on VCS Support.
Installing from local archives¶
If you have a local copy of a Distribution Package’s archive (a zip,
wheel, or tar file) you can install it directly with pip:
Unix/macOS
python3 -m pip install requests-2.18.4.tar.gz
Windows
py -m pip install requests-2.18.4.tar.gz
If you have a directory containing archives of multiple packages, you can tell
pip to look for packages there and not to use the
Python Package Index (PyPI) at all:
Unix/macOS
python3 -m pip install --no-index --find-links=/local/dir/ requests
Windows
py -m pip install --no-index --find-links=/local/dir/ requests
This is useful if you are installing packages on a system with limited
connectivity or if you want to strictly control the origin of distribution
packages.
Using other package indexes¶
If you want to download packages from a different index than the
Python Package Index (PyPI), you can use the --index-url
flag:
Unix/macOS
python3 -m pip install --index-url http://index.example.com/simple/ SomeProject
Windows
py -m pip install --index-url http://index.example.com/simple/ SomeProject
If you want to allow packages from both the Python Package Index (PyPI)
and a separate index, you can use the --extra-index-url
flag instead:
Unix/macOS
python3 -m pip install --extra-index-url http://index.example.com/simple/ SomeProject
Windows
py -m pip install --extra-index-url http://index.example.com/simple/ SomeProject
Upgrading packages¶
pip can upgrade packages in-place using the --upgrade
flag. For example, to
install the latest version of requests
and all of its dependencies:
Unix/macOS
python3 -m pip install --upgrade requests
Windows
py -m pip install --upgrade requests
Using requirements files¶
Instead of installing packages individually, pip allows you to declare all
dependencies in a Requirements File. For
example you could create a requirements.txt
file containing:
requests==2.18.4 google-auth==1.1.0
And tell pip to install all of the packages in this file using the -r
flag:
Unix/macOS
python3 -m pip install -r requirements.txt
Windows
py -m pip install -r requirements.txt
Freezing dependencies¶
Pip can export a list of all installed packages and their versions using the
freeze
command:
Which will output a list of package specifiers such as:
cachetools==2.0.1 certifi==2017.7.27.1 chardet==3.0.4 google-auth==1.1.1 idna==2.6 pyasn1==0.3.6 pyasn1-modules==0.1.4 requests==2.18.4 rsa==3.4.2 six==1.11.0 urllib3==1.22
This is useful for creating Requirements Files that can re-create
the exact versions of all packages installed in an environment.
Running pip#
pip is a command line program. When you install pip, a pip
command is added
to your system, which can be run from the command prompt as follows:
Unix/macOS
python -m pip <pip arguments>
python -m pip
executes pip using the Python interpreter you
specified as python. So /usr/bin/python3.7 -m pip
means
you are executing pip for your interpreter located at /usr/bin/python3.7
.
Windows
py -m pip <pip arguments>
py -m pip
executes pip using the latest Python interpreter you
have installed. For more details, read the Python Windows launcher docs.
Installing Packages#
pip supports installing from PyPI, version control, local projects, and
directly from distribution files.
The most common scenario is to install from PyPI using Requirement Specifiers
Unix/macOS
python -m pip install SomePackage # latest version python -m pip install SomePackage==1.0.4 # specific version python -m pip install 'SomePackage>=1.0.4' # minimum version
Windows
py -m pip install SomePackage # latest version py -m pip install SomePackage==1.0.4 # specific version py -m pip install 'SomePackage>=1.0.4' # minimum version
For more information and examples, see the pip install reference.
Basic Authentication Credentials
This is now covered in Authentication.
netrc Support
This is now covered in Authentication.
Keyring Support
This is now covered in Authentication.
Using a Proxy Server#
When installing packages from PyPI, pip requires internet access, which
in many corporate environments requires an outbound HTTP proxy server.
pip can be configured to connect through a proxy server in various ways:
-
using the
--proxy
command-line option to specify a proxy in the form
scheme://[user:passwd@]proxy.server:port
-
using
proxy
in a Configuration Files -
by setting the standard environment-variables
http_proxy
,https_proxy
andno_proxy
. -
using the environment variable
PIP_USER_AGENT_USER_DATA
to include
a JSON-encoded string in the user-agent variable used in pip’s requests.
Requirements Files#
“Requirements files” are files containing a list of items to be
installed using pip install like so:
Unix/macOS
python -m pip install -r requirements.txt
Windows
py -m pip install -r requirements.txt
Details on the format of the files are here: Requirements File Format.
Logically, a Requirements file is just a list of pip install arguments
placed in a file. Note that you should not rely on the items in the file being
installed by pip in any particular order.
Requirements files can also be served via a URL, e.g.
http://example.com/requirements.txt besides as local files, so that they can
be stored and served in a centralized place.
In practice, there are 4 common uses of Requirements files:
-
Requirements files are used to hold the result from pip freeze for the
purpose of achieving Repeatable Installs. In
this case, your requirement file contains a pinned version of everything that
was installed whenpip freeze
was run.Unix/macOS
python -m pip freeze > requirements.txt python -m pip install -r requirements.txt
Windows
py -m pip freeze > requirements.txt py -m pip install -r requirements.txt
-
Requirements files are used to force pip to properly resolve dependencies.
pip 20.2 and earlier doesn’t have true dependency resolution, but instead simply uses the first
specification it finds for a project. E.g. ifpkg1
requires
pkg3>=1.0
andpkg2
requirespkg3>=1.0,<=2.0
, and ifpkg1
is
resolved first, pip will only usepkg3>=1.0
, and could easily end up
installing a version ofpkg3
that conflicts with the needs ofpkg2
.
To solve this problem, you can placepkg3>=1.0,<=2.0
(i.e. the correct
specification) into your requirements file directly along with the other top
level requirements. Like so:pkg1 pkg2 pkg3>=1.0,<=2.0
-
Requirements files are used to force pip to install an alternate version of a
sub-dependency. For example, supposeProjectA
in your requirements file
requiresProjectB
, but the latest version (v1.3) has a bug, you can force
pip to accept earlier versions like so: -
Requirements files are used to override a dependency with a local patch that
lives in version control. For example, suppose a dependency
SomeDependency
from PyPI has a bug, and you can’t wait for an upstream
fix.
You could clone/copy the src, make the fix, and place it in VCS with the tag
sometag
. You’d reference it in your requirements file with a line like
so:git+https://myvcs.com/some_dependency@sometag#egg=SomeDependency
If
SomeDependency
was previously a top-level requirement in your
requirements file, then replace that line with the new line. If
SomeDependency
is a sub-dependency, then add the new line.
It’s important to be clear that pip determines package dependencies using
install_requires metadata,
not by discovering requirements.txt
files embedded in projects.
See also:
-
Requirements File Format
-
pip freeze
-
“setup.py vs requirements.txt” (an article by Donald Stufft)
Constraints Files#
Constraints files are requirements files that only control which version of a
requirement is installed, not whether it is installed or not. Their syntax and
contents is a subset of Requirements Files, with several kinds of syntax
not allowed: constraints must have a name, they cannot be editable, and they
cannot specify extras. In terms of semantics, there is one key difference:
Including a package in a constraints file does not trigger installation of the
package.
Use a constraints file like so:
Unix/macOS
python -m pip install -c constraints.txt
Windows
py -m pip install -c constraints.txt
Constraints files are used for exactly the same reason as requirements files
when you don’t know exactly what things you want to install. For instance, say
that the “helloworld” package doesn’t work in your environment, so you have a
local patched version. Some things you install depend on “helloworld”, and some
don’t.
One way to ensure that the patched version is used consistently is to
manually audit the dependencies of everything you install, and if “helloworld”
is present, write a requirements file to use when installing that thing.
Constraints files offer a better way: write a single constraints file for your
organisation and use that everywhere. If the thing being installed requires
“helloworld” to be installed, your fixed version specified in your constraints
file will be used.
Constraints file support was added in pip 7.1. In Changes to the pip dependency resolver in 20.3 (2020) we did a fairly comprehensive overhaul, removing several
undocumented and unsupported quirks from the previous implementation,
and stripped constraints files down to being purely a way to specify
global (version) limits for packages.
Same as requirements files, constraints files can also be served via a URL,
e.g. http://example.com/constraints.txt, so that your organization can store and
serve them in a centralized place.
Installing from Wheels#
“Wheel” is a built, archive format that can greatly speed installation compared
to building and installing from source archives. For more information, see the
Wheel docs , PEP 427, and PEP 425.
pip prefers Wheels where they are available. To disable this, use the
—no-binary flag for pip install.
If no satisfactory wheels are found, pip will default to finding source
archives.
To install directly from a wheel archive:
Unix/macOS
python -m pip install SomePackage-1.0-py2.py3-none-any.whl
Windows
py -m pip install SomePackage-1.0-py2.py3-none-any.whl
To include optional dependencies provided in the provides_extras
metadata in the wheel, you must add quotes around the install target
name:
Unix/macOS
python -m pip install './somepackage-1.0-py2.py3-none-any.whl[my-extras]'
Windows
py -m pip install './somepackage-1.0-py2.py3-none-any.whl[my-extras]'
Note
In the future, the path[extras]
syntax may become deprecated. It is
recommended to use PEP 508 syntax wherever possible.
For the cases where wheels are not available, pip offers pip wheel as a
convenience, to build wheels for all your requirements and dependencies.
pip wheel requires the wheel package to be installed, which provides the
“bdist_wheel” setuptools extension that it uses.
To build wheels for your requirements and all their dependencies to a local
directory:
Unix/macOS
python -m pip install wheel python -m pip wheel --wheel-dir=/local/wheels -r requirements.txt
Windows
py -m pip install wheel py -m pip wheel --wheel-dir=/local/wheels -r requirements.txt
And then to install those requirements just using your local directory of
wheels (and not from PyPI):
Unix/macOS
python -m pip install --no-index --find-links=/local/wheels -r requirements.txt
Windows
py -m pip install --no-index --find-links=/local/wheels -r requirements.txt
Uninstalling Packages#
pip is able to uninstall most packages like so:
Unix/macOS
python -m pip uninstall SomePackage
Windows
py -m pip uninstall SomePackage
pip also performs an automatic uninstall of an old version of a package
before upgrading to a newer version.
For more information and examples, see the pip uninstall reference.
Listing Packages#
To list installed packages:
Unix/macOS
$ python -m pip list docutils (0.9.1) Jinja2 (2.6) Pygments (1.5) Sphinx (1.1.2)
Windows
C:\> py -m pip list docutils (0.9.1) Jinja2 (2.6) Pygments (1.5) Sphinx (1.1.2)
To list outdated packages, and show the latest version available:
Unix/macOS
$ python -m pip list --outdated docutils (Current: 0.9.1 Latest: 0.10) Sphinx (Current: 1.1.2 Latest: 1.1.3)
Windows
C:\> py -m pip list --outdated docutils (Current: 0.9.1 Latest: 0.10) Sphinx (Current: 1.1.2 Latest: 1.1.3)
To show details about an installed package:
Unix/macOS
$ python -m pip show sphinx --- Name: Sphinx Version: 1.1.3 Location: /my/env/lib/pythonx.x/site-packages Requires: Pygments, Jinja2, docutils
Windows
C:\> py -m pip show sphinx --- Name: Sphinx Version: 1.1.3 Location: /my/env/lib/pythonx.x/site-packages Requires: Pygments, Jinja2, docutils
For more information and examples, see the pip list and pip show
reference pages.
Searching for Packages#
pip can search PyPI for packages using the pip search
command:
Unix/macOS
python -m pip search "query"
Windows
The query will be used to search the names and summaries of all
packages.
For more information and examples, see the pip search reference.
Configuration
This is now covered in Configuration.
Config file
This is now covered in Configuration.
Environment Variables
This is now covered in Configuration.
Config Precedence
This is now covered in Configuration.
Command Completion#
pip comes with support for command line completion in bash, zsh and fish.
To setup for bash:
python -m pip completion --bash >> ~/.profile
To setup for zsh:
python -m pip completion --zsh >> ~/.zprofile
To setup for fish:
python -m pip completion --fish > ~/.config/fish/completions/pip.fish
To setup for powershell:
python -m pip completion --powershell | Out-File -Encoding default -Append $PROFILE
Alternatively, you can use the result of the completion
command directly
with the eval function of your shell, e.g. by adding the following to your
startup file:
eval "`pip completion --bash`"
Installing from local packages#
In some cases, you may want to install from local packages only, with no traffic
to PyPI.
First, download the archives that fulfill your requirements:
Unix/macOS
python -m pip download --destination-directory DIR -r requirements.txt
Windows
py -m pip download --destination-directory DIR -r requirements.txt
Note that pip download
will look in your wheel cache first, before
trying to download from PyPI. If you’ve never installed your requirements
before, you won’t have a wheel cache for those items. In that case, if some of
your requirements don’t come as wheels from PyPI, and you want wheels, then run
this instead:
Unix/macOS
python -m pip wheel --wheel-dir DIR -r requirements.txt
Windows
py -m pip wheel --wheel-dir DIR -r requirements.txt
Then, to install from local only, you’ll be using —find-links and —no-index like so:
Unix/macOS
python -m pip install --no-index --find-links=DIR -r requirements.txt
Windows
py -m pip install --no-index --find-links=DIR -r requirements.txt
“Only if needed” Recursive Upgrade#
pip install --upgrade
now has a --upgrade-strategy
option which
controls how pip handles upgrading of dependencies. There are 2 upgrade
strategies supported:
-
eager
: upgrades all dependencies regardless of whether they still satisfy
the new parent requirements -
only-if-needed
: upgrades a dependency only if it does not satisfy the new
parent requirements
The default strategy is only-if-needed
. This was changed in pip 10.0 due to
the breaking nature of eager
when upgrading conflicting dependencies.
It is important to note that --upgrade
affects direct requirements (e.g.
those specified on the command-line or via a requirements file) while
--upgrade-strategy
affects indirect requirements (dependencies of direct
requirements).
As an example, say SomePackage
has a dependency, SomeDependency
, and
both of them are already installed but are not the latest available versions:
-
pip install SomePackage
: will not upgrade the existingSomePackage
or
SomeDependency
. -
pip install --upgrade SomePackage
: will upgradeSomePackage
, but not
SomeDependency
(unless a minimum requirement is not met). -
pip install --upgrade SomePackage --upgrade-strategy=eager
: upgrades both
SomePackage
andSomeDependency
.
As an historic note, an earlier “fix” for getting the only-if-needed
behaviour was:
Unix/macOS
python -m pip install --upgrade --no-deps SomePackage python -m pip install SomePackage
Windows
py -m pip install --upgrade --no-deps SomePackage py -m pip install SomePackage
A proposal for an upgrade-all
command is being considered as a safer
alternative to the behaviour of eager upgrading.
User Installs#
With Python 2.6 came the “user scheme” for installation,
which means that all Python distributions support an alternative install
location that is specific to a user. The default location for each OS is
explained in the python documentation for the site.USER_BASE variable.
This mode of installation can be turned on by specifying the —user option to pip install
.
Moreover, the “user scheme” can be customized by setting the
PYTHONUSERBASE
environment variable, which updates the value of
site.USER_BASE
.
To install “SomePackage” into an environment with site.USER_BASE
customized to
‘/myappenv’, do the following:
Unix/macOS
export PYTHONUSERBASE=/myappenv python -m pip install --user SomePackage
Windows
set PYTHONUSERBASE=c:/myappenv py -m pip install --user SomePackage
pip install --user
follows four rules:
-
When globally installed packages are on the python path, and they conflict
with the installation requirements, they are ignored, and not
uninstalled. -
When globally installed packages are on the python path, and they satisfy
the installation requirements, pip does nothing, and reports that
requirement is satisfied (similar to how global packages can satisfy
requirements when installing packages in a--system-site-packages
virtualenv). -
pip will not perform a
--user
install in a--no-site-packages
virtualenv (i.e. the default kind of virtualenv), due to the user site not
being on the python path. The installation would be pointless. -
In a
--system-site-packages
virtualenv, pip will not install a package
that conflicts with a package in the virtualenv site-packages. The —user
installation would lack sys.path precedence and be pointless.
To make the rules clearer, here are some examples:
From within a --no-site-packages
virtualenv (i.e. the default kind):
Unix/macOS
$ python -m pip install --user SomePackage Can not perform a '--user' install. User site-packages are not visible in this virtualenv.
Windows
C:\> py -m pip install --user SomePackage Can not perform a '--user' install. User site-packages are not visible in this virtualenv.
From within a --system-site-packages
virtualenv where SomePackage==0.3
is already installed in the virtualenv:
Unix/macOS
$ python -m pip install --user SomePackage==0.4 Will not install to the user site because it will lack sys.path precedence
Windows
C:\> py -m pip install --user SomePackage==0.4 Will not install to the user site because it will lack sys.path precedence
From within a real python, where SomePackage
is not installed globally:
Unix/macOS
$ python -m pip install --user SomePackage [...] Successfully installed SomePackage
Windows
C:\> py -m pip install --user SomePackage [...] Successfully installed SomePackage
From within a real python, where SomePackage
is installed globally, but
is not the latest version:
Unix/macOS
$ python -m pip install --user SomePackage [...] Requirement already satisfied (use --upgrade to upgrade) $ python -m pip install --user --upgrade SomePackage [...] Successfully installed SomePackage
Windows
C:\> py -m pip install --user SomePackage [...] Requirement already satisfied (use --upgrade to upgrade) C:\> py -m pip install --user --upgrade SomePackage [...] Successfully installed SomePackage
From within a real python, where SomePackage
is installed globally, and
is the latest version:
Unix/macOS
$ python -m pip install --user SomePackage [...] Requirement already satisfied (use --upgrade to upgrade) $ python -m pip install --user --upgrade SomePackage [...] Requirement already up-to-date: SomePackage # force the install $ python -m pip install --user --ignore-installed SomePackage [...] Successfully installed SomePackage
Windows
C:\> py -m pip install --user SomePackage [...] Requirement already satisfied (use --upgrade to upgrade) C:\> py -m pip install --user --upgrade SomePackage [...] Requirement already up-to-date: SomePackage # force the install C:\> py -m pip install --user --ignore-installed SomePackage [...] Successfully installed SomePackage
Ensuring Repeatability
This is now covered in Repeatable Installs.
Fixing conflicting dependencies
This is now covered in Dependency Resolution.
Using pip from your program#
As noted previously, pip is a command line program. While it is implemented in
Python, and so is available from your Python code via import pip
, you must
not use pip’s internal APIs in this way. There are a number of reasons for this:
-
The pip code assumes that it is in sole control of the global state of the
program.
pip manages things like the logging system configuration, or the values of
the standard IO streams, without considering the possibility that user code
might be affected. -
pip’s code is not thread safe. If you were to run pip in a thread, there
is no guarantee that either your code or pip’s would work as you expect. -
pip assumes that once it has finished its work, the process will terminate.
It doesn’t need to handle the possibility that other code will continue to
run after that point, so (for example) calling pip twice in the same process
is likely to have issues.
This does not mean that the pip developers are opposed in principle to the idea
that pip could be used as a library — it’s just that this isn’t how it was
written, and it would be a lot of work to redesign the internals for use as a
library, handling all of the above issues, and designing a usable, robust and
stable API that we could guarantee would remain available across multiple
releases of pip. And we simply don’t currently have the resources to even
consider such a task.
What this means in practice is that everything inside of pip is considered an
implementation detail. Even the fact that the import name is pip
is subject
to change without notice. While we do try not to break things as much as
possible, all the internal APIs can change at any time, for any reason. It also
means that we generally won’t fix issues that are a result of using pip in an
unsupported way.
It should also be noted that installing packages into sys.path
in a running
Python process is something that should only be done with care. The import
system caches certain data, and installing new packages while a program is
running may not always behave as expected. In practice, there is rarely an
issue, but it is something to be aware of.
Having said all of the above, it is worth covering the options available if you
decide that you do want to run pip from within your program. The most reliable
approach, and the one that is fully supported, is to run pip in a subprocess.
This is easily done using the standard subprocess
module:
subprocess.check_call([sys.executable, '-m', 'pip', 'install', 'my_package'])
If you want to process the output further, use one of the other APIs in the module.
We are using freeze here which outputs installed packages in requirements format.:
reqs = subprocess.check_output([sys.executable, '-m', 'pip', 'freeze'])
If you don’t want to use pip’s command line functionality, but are rather
trying to implement code that works with Python packages, their metadata, or
PyPI, then you should consider other, supported, packages that offer this type
of ability. Some examples that you could consider include:
-
packaging
— Utilities to work with standard package metadata (versions,
requirements, etc.) -
setuptools
(specificallypkg_resources
) — Functions for querying what
packages the user has installed on their system. -
distlib
— Packaging and distribution utilities (including functions for
interacting with PyPI).
Changes to the pip dependency resolver in 20.3 (2020)#
pip 20.3 has a new dependency resolver, on by default for Python 3
users. (pip 20.1 and 20.2 included pre-release versions of the new
dependency resolver, hidden behind optional user flags.) Read below
for a migration guide, how to invoke the legacy resolver, and the
deprecation timeline. We also made a two-minute video explanation
you can watch.
We will continue to improve the pip dependency resolver in response to
testers’ feedback. Please give us feedback through the resolver
testing survey.
Watch out for#
The big change in this release is to the pip dependency resolver
within pip.
Computers need to know the right order to install pieces of software
(“to install x
, you need to install y
first”). So, when Python
programmers share software as packages, they have to precisely describe
those installation prerequisites, and pip needs to navigate tricky
situations where it’s getting conflicting instructions. This new
dependency resolver will make pip better at handling that tricky
logic, and make pip easier for you to use and troubleshoot.
The most significant changes to the resolver are:
-
It will reduce inconsistency: it will no longer install a
combination of packages that is mutually inconsistent. In older
versions of pip, it is possible for pip to install a package which
does not satisfy the declared requirements of another installed
package. For example, in pip 20.0,pip install "six<1.12"
does the wrong thing, “successfully” installing
"virtualenv==20.0.2"
six==1.11
, even thoughvirtualenv==20.0.2
requires
six>=1.12.0,<2
(defined here).
The new resolver, instead, outright rejects installing anything if it
gets that input. -
It will be stricter — if you ask pip to install two packages with
incompatible requirements, it will refuse (rather than installing a
broken combination, like it did in previous versions).
So, if you have been using workarounds to force pip to deal with
incompatible or inconsistent requirements combinations, now’s a good
time to fix the underlying problem in the packages, because pip will
be stricter from here on out.
This also means that, when you run a pip install
command, pip only
considers the packages you are installing in that command, and may
break already-installed packages. It will not guarantee that your
environment will be consistent all the time. If you pip install x
and then pip install y
, it’s possible that the version of y
you get will be different than it would be if you had run pip
in a single command. We are considering changing this
install x y
behavior (per #7744) and would like your thoughts on what
pip’s behavior should be; please answer our survey on upgrades that
create conflicts.
We are also changing our support for Constraints Files,
editable installs, and related functionality. We did a fairly
comprehensive overhaul and stripped constraints files down to being
purely a way to specify global (version) limits for packages, and so
some combinations that used to be allowed will now cause
errors. Specifically:
-
Constraints don’t override the existing requirements; they simply
constrain what versions are visible as input to the resolver (see
#9020) -
Providing an editable requirement (
-e .
) does not cause pip to
ignore version specifiers or constraints (see #8076), and if
you have a conflict between a pinned requirement and a local
directory then pip will indicate that it cannot find a version
satisfying both (see #8307) -
Hash-checking mode requires that all requirements are specified as a
==
match on a version and may not work well in combination with
constraints (see #9020 and #8792) -
If necessary to satisfy constraints, pip will happily reinstall
packages, upgrading or downgrading, without needing any additional
command-line options (see #8115 and Options that control the installation process) -
Unnamed requirements are not allowed as constraints (see #6628 and #8210)
-
Links are not allowed as constraints (see #8253)
-
Constraints cannot have extras (see #6628)
Per our Python 2 Support policy, pip 20.3 users who are using
Python 2 will use the legacy resolver by default. Python 2 users
should upgrade to Python 3 as soon as possible, since in pip 21.0 in
January 2021, pip dropped support for Python 2 altogether.
How to upgrade and migrate#
-
Install pip 20.3 with
python -m pip install --upgrade pip
. -
Validate your current environment by running
pip check
. This
will report if you have any inconsistencies in your set of installed
packages. Having a clean installation will make it much less likely
that you will hit issues with the new resolver (and may
address hidden problems in your current environment!). If you run
pip check
and run into stuff you can’t figure out, please ask
for help in our issue tracker or chat. -
Test the new version of pip.
While we have tried to make sure that pip’s test suite covers as
many cases as we can, we are very aware that there are people using
pip with many different workflows and build processes, and we will
not be able to cover all of those without your help.-
If you use pip to install your software, try out the new resolver
and let us know if it works for you withpip install
. Try:-
installing several packages simultaneously
-
re-creating an environment using a
requirements.txt
file -
using
pip install --force-reinstall
to check whether
it does what you think it should -
using constraints files
-
the “Setups to test with special attention” and “Examples to try” below
-
-
If you have a build pipeline that depends on pip installing your
dependencies for you, check that the new resolver does what you
need. -
Run your project’s CI (test suite, build process, etc.) using the
new resolver, and let us know of any issues. -
If you have encountered resolver issues with pip in the past,
check whether the new resolver fixes them, and read Dealing with dependency conflicts. Also, let us know if the new resolver
has issues with any workarounds you put in to address the
current resolver’s limitations. We’ll need to ensure that people
can transition off such workarounds smoothly. -
If you develop or support a tool that wraps pip or uses it to
deliver part of your functionality, please test your integration
with pip 20.3.
-
-
Troubleshoot and try these workarounds if necessary.
-
If pip is taking longer to install packages, read Dependency
resolution backtracking for ways to
reduce the time pip spends backtracking due to dependency conflicts. -
If you don’t want pip to actually resolve dependencies, use the
--no-deps
option. This is useful when you have a set of package
versions that work together in reality, even though their metadata says
that they conflict. For guidance on a long-term fix, read
Dealing with dependency conflicts. -
If you run into resolution errors and need a workaround while you’re
fixing their root causes, you can choose the old resolver behavior using
the flag--use-deprecated=legacy-resolver
. This will work until we
release pip 21.0 (see
Deprecation timeline).
-
-
Please report bugs through the resolver testing survey.
Setups to test with special attention#
-
Requirements files with 100+ packages
-
Installation workflows that involve multiple requirements files
-
Requirements files that include hashes (Hash-checking Mode)
or pinned dependencies (perhaps as output frompip-compile
within
pip-tools
) -
Using Constraints Files
-
Continuous integration/continuous deployment setups
-
Installing from any kind of version control systems (i.e., Git, Subversion, Mercurial, or CVS), per VCS Support
-
Installing from source code held in local directories
Examples to try#
Install:
-
tensorflow
-
hacking
-
pycodestyle
-
pandas
-
tablib
-
elasticsearch
andrequests
together -
six
andcherrypy
together -
pip install flake8-import-order==0.17.1 flake8==3.5.0 --use-feature=2020-resolver
-
pip install tornado==5.0 sprockets.http==1.5.0 --use-feature=2020-resolver
Try:
-
pip install
-
pip uninstall
-
pip check
-
pip cache
Tell us about#
Specific things we’d love to get feedback on:
-
Cases where the new resolver produces the wrong result,
obviously. We hope there won’t be too many of these, but we’d like
to trap such bugs before we remove the legacy resolver. -
Cases where the resolver produced an error when you believe it
should have been able to work out what to do. -
Cases where the resolver gives an error because there’s a problem
with your requirements, but you need better information to work out
what’s wrong. -
If you have workarounds to address issues with the current resolver,
does the new resolver let you remove those workarounds? Tell us!
Please let us know through the resolver testing survey.
Deprecation timeline#
We plan for the resolver changeover to proceed as follows, using
Feature Flags and following our Release Cadence:
-
pip 20.1: an alpha version of the new resolver was available,
opt-in, using the optional flag
--unstable-feature=resolver
. pip defaulted to legacy
behavior. -
pip 20.2: a beta of the new resolver was available, opt-in, using
the flag--use-feature=2020-resolver
. pip defaulted to legacy
behavior. Users of pip 20.2 who want pip to default to using the
new resolver can runpip config set global.use-feature
(for more on that and the alternate
2020-resolver
PIP_USE_FEATURE
environment variable option, see issue
8661). -
pip 20.3: pip defaults to the new resolver in Python 3 environments,
but a user can opt-out and choose the old resolver behavior,
using the flag--use-deprecated=legacy-resolver
. In Python 2
environments, pip defaults to the old resolver, and the new one is
available using the flag--use-feature=2020-resolver
. -
pip 21.0: pip uses new resolver by default, and the old resolver is
no longer supported. It will be removed after a currently undecided
amount of time, as the removal is dependent on pip’s volunteer
maintainers’ availability. Python 2 support is removed per our
Python 2 Support policy.
Since this work will not change user-visible behavior described in the
pip documentation, this change is not covered by the Deprecation Policy.
Context and followup#
As discussed in our announcement on the PSF blog, the pip team are
in the process of developing a new “dependency resolver” (the part of
pip that works out what to install based on your requirements).
We’re tracking our rollout in #6536 and you can watch for
announcements on the low-traffic packaging announcements list and
the official Python blog.
Using system trust stores for verifying HTTPS
This is now covered in HTTPS Certificates.
Contents
- User Guide
- Installing Packages
- Requirements Files
- Constraints Files
- Installing from Wheels
- Uninstalling Packages
- Listing Packages
- Searching for Packages
- Configuration
- Config file
- Environment Variables
- Config Precedence
- Command Completion
- Installing from local packages
- «Only if needed» Recursive Upgrade
- User Installs
- Ensuring Repeatability
- Pinned Version Numbers
- Hash-checking Mode
- Installation Bundles
Installing Packages¶
pip supports installing from PyPI, version control, local projects, and
directly from distribution files.
The most common scenario is to install from PyPI using Requirement Specifiers
$ pip install SomePackage # latest version $ pip install SomePackage==1.0.4 # specific version $ pip install 'SomePackage>=1.0.4' # minimum version
For more information and examples, see the pip install reference.
Requirements Files¶
«Requirements files» are files containing a list of items to be
installed using pip install like so:
pip install -r requirements.txt
Details on the format of the files are here: Requirements File Format.
Logically, a Requirements file is just a list of pip install arguments
placed in a file. Note that you should not rely on the items in the file being
installed by pip in any particular order.
In practice, there are 4 common uses of Requirements files:
-
Requirements files are used to hold the result from pip freeze for the
purpose of achieving repeatable installations. In
this case, your requirement file contains a pinned version of everything that
was installed when pip freeze was run.pip freeze > requirements.txt pip install -r requirements.txt
-
Requirements files are used to force pip to properly resolve dependencies.
As it is now, pip doesn’t have true dependency resolution, but instead simply uses the first
specification it finds for a project. E.g if pkg1 requires pkg3>=1.0 and
pkg2 requires pkg3>=1.0,<=2.0, and if pkg1 is resolved first, pip will
only use pkg3>=1.0, and could easily end up installing a version of pkg3
that conflicts with the needs of pkg2. To solve this problem, you can
place pkg3>=1.0,<=2.0 (i.e. the correct specification) into your
requirements file directly along with the other top level requirements. Like
so:pkg1 pkg2 pkg3>=1.0,<=2.0
-
Requirements files are used to force pip to install an alternate version of a
sub-dependency. For example, suppose ProjectA in your requirements file
requires ProjectB, but the latest version (v1.3) has a bug, you can force
pip to accept earlier versions like so: -
Requirements files are used to override a dependency with a local patch that
lives in version control. For example, suppose a dependency,
SomeDependency from PyPI has a bug, and you can’t wait for an upstream fix.
You could clone/copy the src, make the fix, and place it in VCS with the tag
sometag. You’d reference it in your requirements file with a line like so:git+https://myvcs.com/some_dependency@sometag#egg=SomeDependency
If SomeDependency was previously a top-level requirement in your
requirements file, then replace that line with the new line. If
SomeDependency is a sub-dependency, then add the new line.
It’s important to be clear that pip determines package dependencies using
install_requires metadata,
not by discovering requirements.txt files embedded in projects.
See also:
- Requirements File Format
- pip freeze
- «setup.py vs requirements.txt» (an article by Donald Stufft)
Constraints Files¶
Constraints files are requirements files that only control which version of a
requirement is installed, not whether it is installed or not. Their syntax and
contents is nearly identical to Requirements Files. There is one key
difference: Including a package in a constraints file does not trigger
installation of the package.
Use a constraints file like so:
pip install -c constraints.txt
Constraints files are used for exactly the same reason as requirements files
when you don’t know exactly what things you want to install. For instance, say
that the «helloworld» package doesn’t work in your environment, so you have a
local patched version. Some things you install depend on «helloworld», and some
don’t.
One way to ensure that the patched version is used consistently is to
manually audit the dependencies of everything you install, and if «helloworld»
is present, write a requirements file to use when installing that thing.
Constraints files offer a better way: write a single constraints file for your
organisation and use that everywhere. If the thing being installed requires
«helloworld» to be installed, your fixed version specified in your constraints
file will be used.
Constraints file support was added in pip 7.1.
Installing from Wheels¶
«Wheel» is a built, archive format that can greatly speed installation compared
to building and installing from source archives. For more information, see the
Wheel docs ,
PEP427, and
PEP425
Pip prefers Wheels where they are available. To disable this, use the
—no-binary flag for pip install.
If no satisfactory wheels are found, pip will default to finding source archives.
To install directly from a wheel archive:
pip install SomePackage-1.0-py2.py3-none-any.whl
For the cases where wheels are not available, pip offers pip wheel as a
convenience, to build wheels for all your requirements and dependencies.
pip wheel requires the wheel package to be installed, which provides the
«bdist_wheel» setuptools extension that it uses.
To build wheels for your requirements and all their dependencies to a local directory:
pip install wheel pip wheel --wheel-dir=/local/wheels -r requirements.txt
And then to install those requirements just using your local directory of wheels (and not from PyPI):
pip install --no-index --find-links=/local/wheels -r requirements.txt
Uninstalling Packages¶
pip is able to uninstall most packages like so:
$ pip uninstall SomePackage
pip also performs an automatic uninstall of an old version of a package
before upgrading to a newer version.
For more information and examples, see the pip uninstall reference.
Listing Packages¶
To list installed packages:
$ pip list docutils (0.9.1) Jinja2 (2.6) Pygments (1.5) Sphinx (1.1.2)
To list outdated packages, and show the latest version available:
$ pip list --outdated docutils (Current: 0.9.1 Latest: 0.10) Sphinx (Current: 1.1.2 Latest: 1.1.3)
To show details about an installed package:
$ pip show sphinx --- Name: Sphinx Version: 1.1.3 Location: /my/env/lib/pythonx.x/site-packages Requires: Pygments, Jinja2, docutils
For more information and examples, see the pip list and pip show
reference pages.
Searching for Packages¶
pip can search PyPI for packages using the pip search
command:
The query will be used to search the names and summaries of all
packages.
For more information and examples, see the pip search reference.
Configuration¶
Config file¶
pip allows you to set all command line option defaults in a standard ini
style config file.
The names and locations of the configuration files vary slightly across
platforms. You may have per-user, per-virtualenv or site-wide (shared amongst
all users) configuration:
Per-user:
- On Unix the default configuration file is:
$HOME/.config/pip/pip.conf
which respects theXDG_CONFIG_HOME
environment variable. - On macOS the configuration file is
$HOME/Library/Application Support/pip/pip.conf
. - On Windows the configuration file is
%APPDATA%\pip\pip.ini
.
There are also a legacy per-user configuration file which is also respected,
these are located at:
- On Unix and macOS the configuration file is:
$HOME/.pip/pip.conf
- On Windows the configuration file is:
%HOME%\pip\pip.ini
You can set a custom path location for this config file using the environment
variable PIP_CONFIG_FILE
.
Inside a virtualenv:
- On Unix and macOS the file is
$VIRTUAL_ENV/pip.conf
- On Windows the file is:
%VIRTUAL_ENV%\pip.ini
Site-wide:
- On Unix the file may be located in
/etc/pip.conf
. Alternatively
it may be in a «pip» subdirectory of any of the paths set in the
environment variableXDG_CONFIG_DIRS
(if it exists), for example
/etc/xdg/pip/pip.conf
. - On macOS the file is:
/Library/Application Support/pip/pip.conf
- On Windows XP the file is:
C:\Documents and Settings\All Users\Application Data\pip\pip.ini
- On Windows 7 and later the file is hidden, but writeable at
C:\ProgramData\pip\pip.ini
- Site-wide configuration is not supported on Windows Vista
If multiple configuration files are found by pip then they are combined in
the following order:
- Firstly the site-wide file is read, then
- The per-user file is read, and finally
- The virtualenv-specific file is read.
Each file read overrides any values read from previous files, so if the
global timeout is specified in both the site-wide file and the per-user file
then the latter value is the one that will be used.
The names of the settings are derived from the long command line option, e.g.
if you want to use a different package index (--index-url
) and set the
HTTP timeout (--default-timeout
) to 60 seconds your config file would
look like this:
[global] timeout = 60 index-url = http://download.zope.org/ppix
Each subcommand can be configured optionally in its own section so that every
global setting with the same name will be overridden; e.g. decreasing the
timeout
to 10
seconds when running the freeze
(Freezing Requirements) command and using
60
seconds for all other commands is possible with:
[global] timeout = 60 [freeze] timeout = 10
Boolean options like --ignore-installed
or --no-dependencies
can be
set like this:
[install] ignore-installed = true no-dependencies = yes
To enable the boolean options --no-compile
and --no-cache-dir
, falsy
values have to be used:
[global] no-cache-dir = false [install] no-compile = no
Appending options like --find-links
can be written on multiple lines:
[global] find-links = http://download.example.com [install] find-links = http://mirror1.example.com http://mirror2.example.com
Environment Variables¶
pip’s command line options can be set with environment variables using the
format PIP_<UPPER_LONG_NAME>
. Dashes (-
) have to be replaced with
underscores (_
).
For example, to set the default timeout:
export PIP_DEFAULT_TIMEOUT=60
This is the same as passing the option to pip directly:
pip --default-timeout=60 [...]
To set options that can be set multiple times on the command line, just add
spaces in between values. For example:
export PIP_FIND_LINKS="http://mirror1.example.com http://mirror2.example.com"
is the same as calling:
pip install --find-links=http://mirror1.example.com --find-links=http://mirror2.example.com
Config Precedence¶
Command line options have precedence over environment variables, which have precedence over the config file.
Within the config file, command specific sections have precedence over the global section.
Examples:
--host=foo
overridesPIP_HOST=foo
PIP_HOST=foo
overrides a config file with[global] host = foo
- A command specific section in the config file
[<command>] host = bar
overrides the option with same name in the[global]
config file section
Command Completion¶
pip comes with support for command line completion in bash, zsh and fish.
To setup for bash:
$ pip completion --bash >> ~/.profile
To setup for zsh:
$ pip completion --zsh >> ~/.zprofile
To setup for fish:
$ pip completion --fish > ~/.config/fish/completions/pip.fish
Alternatively, you can use the result of the completion
command
directly with the eval function of your shell, e.g. by adding the following to your startup file:
eval "`pip completion --bash`"
Installing from local packages¶
In some cases, you may want to install from local packages only, with no traffic
to PyPI.
First, download the archives that fulfill your requirements:
$ pip install --download DIR -r requirements.txt
Note that pip install --download
will look in your wheel cache first, before
trying to download from PyPI. If you’ve never installed your requirements
before, you won’t have a wheel cache for those items. In that case, if some of
your requirements don’t come as wheels from PyPI, and you want wheels, then run
this instead:
$ pip wheel --wheel-dir DIR -r requirements.txt
Then, to install from local only, you’ll be using —find-links and —no-index like so:
$ pip install --no-index --find-links=DIR -r requirements.txt
«Only if needed» Recursive Upgrade¶
pip install --upgrade
is currently written to perform an eager recursive
upgrade, i.e. it upgrades all dependencies regardless of whether they still
satisfy the new parent requirements.
E.g. supposing:
- SomePackage-1.0 requires AnotherPackage>=1.0
- SomePackage-2.0 requires AnotherPackage>=1.0 and OneMorePackage==1.0
- SomePackage-1.0 and AnotherPackage-1.0 are currently installed
- SomePackage-2.0 and AnotherPackage-2.0 are the latest versions available on PyPI.
Running pip install --upgrade SomePackage
would upgrade SomePackage and
AnotherPackage despite AnotherPackage already being satisfied.
pip doesn’t currently have an option to do an «only if needed» recursive
upgrade, but you can achieve it using these 2 steps:
pip install --upgrade --no-deps SomePackage pip install SomePackage
The first line will upgrade SomePackage, but not dependencies like
AnotherPackage. The 2nd line will fill in new dependencies like
OneMorePackage.
See #59 for a plan of making «only if needed» recursive the default
behavior for a new pip upgrade
command.
User Installs¶
With Python 2.6 came the «user scheme» for installation,
which means that all Python distributions support an alternative install
location that is specific to a user. The default location for each OS is
explained in the python documentation for the site.USER_BASE variable. This mode
of installation can be turned on by specifying the —user option to pip install
.
Moreover, the «user scheme» can be customized by setting the
PYTHONUSERBASE
environment variable, which updates the value of site.USER_BASE
.
To install «SomePackage» into an environment with site.USER_BASE customized to ‘/myappenv’, do the following:
export PYTHONUSERBASE=/myappenv pip install --user SomePackage
pip install --user
follows four rules:
- When globally installed packages are on the python path, and they conflict
with the installation requirements, they are ignored, and not
uninstalled. - When globally installed packages are on the python path, and they satisfy
the installation requirements, pip does nothing, and reports that
requirement is satisfied (similar to how global packages can satisfy
requirements when installing packages in a--system-site-packages
virtualenv). - pip will not perform a
--user
install in a--no-site-packages
virtualenv (i.e. the default kind of virtualenv), due to the user site not
being on the python path. The installation would be pointless. - In a
--system-site-packages
virtualenv, pip will not install a package
that conflicts with a package in the virtualenv site-packages. The —user
installation would lack sys.path precedence and be pointless.
To make the rules clearer, here are some examples:
From within a --no-site-packages
virtualenv (i.e. the default kind):
$ pip install --user SomePackage Can not perform a '--user' install. User site-packages are not visible in this virtualenv.
From within a --system-site-packages
virtualenv where SomePackage==0.3
is already installed in the virtualenv:
$ pip install --user SomePackage==0.4 Will not install to the user site because it will lack sys.path precedence
From within a real python, where SomePackage
is not installed globally:
$ pip install --user SomePackage [...] Successfully installed SomePackage
From within a real python, where SomePackage
is installed globally, but is not the latest version:
$ pip install --user SomePackage [...] Requirement already satisfied (use --upgrade to upgrade) $ pip install --user --upgrade SomePackage [...] Successfully installed SomePackage
From within a real python, where SomePackage
is installed globally, and is the latest version:
$ pip install --user SomePackage [...] Requirement already satisfied (use --upgrade to upgrade) $ pip install --user --upgrade SomePackage [...] Requirement already up-to-date: SomePackage # force the install $ pip install --user --ignore-installed SomePackage [...] Successfully installed SomePackage
Ensuring Repeatability¶
pip can achieve various levels of repeatability:
Pinned Version Numbers¶
Pinning the versions of your dependencies in the requirements file
protects you from bugs or incompatibilities in newly released versions:
SomePackage == 1.2.3 DependencyOfSomePackage == 4.5.6
Using pip freeze to generate the requirements file will ensure that not
only the top-level dependencies are included but their sub-dependencies as
well, and so on. Perform the installation using —no-deps for an extra dose of insurance against installing
anything not explicitly listed.
This strategy is easy to implement and works across OSes and architectures.
However, it trusts PyPI and the certificate authority chain. It
also relies on indices and find-links locations not allowing
packages to change without a version increase. (PyPI does protect
against this.)
Hash-checking Mode¶
Beyond pinning version numbers, you can add hashes against which to verify
downloaded packages:
FooProject == 1.2 --hash=sha256:2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824
This protects against a compromise of PyPI or the HTTPS
certificate chain. It also guards against a package changing
without its version number changing (on indexes that allow this).
This approach is a good fit for automated server deployments.
Hash-checking mode is a labor-saving alternative to running a private index
server containing approved packages: it removes the need to upload packages,
maintain ACLs, and keep an audit trail (which a VCS gives you on the
requirements file for free). It can also substitute for a vendor library,
providing easier upgrades and less VCS noise. It does not, of course,
provide the availability benefits of a private index or a vendor library.
For more, see pip install’s discussion of hash-checking mode.
Installation Bundles¶
Using pip wheel, you can bundle up all of a project’s dependencies, with
any compilation done, into a single archive. This allows installation when
index servers are unavailable and avoids time-consuming recompilation. Create
an archive like this:
$ tempdir=$(mktemp -d /tmp/wheelhouse-XXXXX) $ pip wheel -r requirements.txt --wheel-dir=$tempdir $ cwd=`pwd` $ (cd "$tempdir"; tar -cjvf "$cwd/bundled.tar.bz2" *)
You can then install from the archive like this:
$ tempdir=$(mktemp -d /tmp/wheelhouse-XXXXX) $ (cd $tempdir; tar -xvf /path/to/bundled.tar.bz2) $ pip install --force-reinstall --ignore-installed --upgrade --no-index --no-deps $tempdir/*
Note that compiled packages are typically OS- and architecture-specific, so
these archives are not necessarily portable across machines.
Hash-checking mode can be used along with this method to ensure that future
archives are built with identical packages.
Warning
Finally, beware of the setup_requires
keyword arg in setup.py
.
The (rare) packages that use it will cause those dependencies to be
downloaded by setuptools directly, skipping pip’s protections. If you need
to use such a package, see Controlling
setup_requires.
Что представляют собой пакеты и модули, откуда их брать и что с ними делать.
https://gbcdn.mrgcdn.ru/uploads/post/1340/og_cover_image/a9b1c9e84cf2c603aa80f227403c4177
Прежде чем что-то устанавливать, давайте разберёмся, что такое пакет, чем он отличается от модуля, и как с ним работать. У слова «пакет» применительно к Python два значения.
C одной стороны, пакеты Python — это Py-приложения, дополнения или утилиты, которые можно установить из внешнего репозитория: Github, Bitbucket, Google Code или официального Python Package Index. На сервере пакеты хранятся в .zip и .tar архивах, либо в дополнительной упаковке — «яйцах» (.egg, старый формат) или «колесах» (.whl). В составе пакета, как правило, есть сценарий установки setup.py, который хранит сведения о зависимостях — других пакетах и модулях, без которых пакет работать не будет.
С другой стороны, если речь об архитектуре Python-приложения, пакет — это каталог, внутри которого файл __init__.py и, опционально, другие каталоги и файлы .py. Так большую Python-программу разбивают на пакеты и модули. Модуль — файл с исходным кодом, который можно использовать в других приложениях: как «заготовку» для будущих проектов или как часть библиотеки/фреймворка. Но к теме статьи это прямого отношения не имеет, поэтому дальше мы будем говорить только о пакетах из репозиториев.
Чтобы за секунды устанавливать пакеты со всеми зависимостями, используют менеджер пакетов pip или модуль easy_install. В большинстве случаев рекомендуется использовать pip. И только если у вас есть инфраструктура на пакетах .egg, которые pip не открывает, нужен easy_install.
Установка PIP для Python 3 и 2
Если вы используете виртуальные окружения на базе venv или virtualenv, pip уже установлен. Начиная с Python 3.4 (для Python 2 — с версии 2.7.9) pip поставляется вместе с интерпретатором. Для более ранних версий устанавливать менеджер пакетов нужно вручную. Вариантов два:
-
C помощью скрипта get_pip.py — быстро.
-
Через setuptools — кроме pip сможем использовать easy_install.
Вариант 1. Скачиваем скрипт get_pip.py и запускаем в консоли. Для этого открываем терминал через Win+R>»cmd»>OK и пишем:
python get_pip.py
Остальное установщик сделает сам: если нужно, попутно установит wheel (для распаковки .whl-колес) и setuptools. Чтобы запретить инсталляцию дополнительных инструментов, можно добавить в строку ключи —no-setuptools и/или —no-wheels.
Если возникает ошибка, путь к Python не прописан в переменной среды $PATH. Нужно либо найти эту переменную в системном реестре и задать её значение, либо каждый раз указывать полный путь до python.exe, а за ним уже имя исполняемого Py-файла:
C:/python32/python.exe get_pip.py
Полный путь полезен и в том случае, если у вас на компьютере несколько версий Python и вы ставите пакет для одной из них.
Вариант 2. Скачиваем архив с setuptools из PYPI и распаковываем в отдельный каталог. В терминале переходим в директорию setuptools c файлом setup.py и пишем:
python setup.py install
Обновить pip для Python в Windows можно так:
python pip install -U pip
Если это не работает, нужно добавить путь к папке с pip в $PATH.
Установка пакета в pip
Пора запустить pip в Python и начать устанавливать пакеты короткой командой из консоли:
pip install имя_пакета
При установке в Windows, перед pip нужно добавить «python -m».
Обновить пакет не сложнее:
pip install имя_пакета -U
Если у вас последняя версия пакета, но вы хотите принудительно переустановить его:
pip install --force-reinstall
Посмотреть список установленных пакетов Python можно с помощью команды:
pip list
Найти конкретный пакет по имени можно командой «pip search». О других командах можно прочесть в справке, которая выдается по команде «pip help».
Удаление пакета Python
Когда пакет больше не нужен, пишем:
pip uninstall имя_пакета
Как установить пакеты в Python без pip
Формат .egg сейчас используют не часто, поэтому pip его не поддерживает. Модуль easy_install умеет устанавливать как .egg, так и обычные пакеты, но есть у него важные минусы:
-
он не удаляет пакеты,
-
он может пытаться установить недозагруженный пакет.
Использовать easy_install можно сразу после установки setuptools. Хранится модуль в папке Scripts вашего интерпретатора. Если у вас в $PATH верно прописан путь, ставить пакеты из PYPI можно короткой командой:
easy_install имя_пакета
Для обновления после install и перед именем пакета нужно ставить ключ -U. Откатиться до нужной версии можно так:
easy_install имя_пакета=0.2.3
Если нужно скачать пакет из альтернативного источника, вы можете задать URL или локальный адрес на компьютере:
easy_install http://адрес_репозитория.ру/директория/пакет-1.1.2.zip
Чтобы узнать об опциях easy_install, запустим его с ключом -h:
easy_install -h
Список пакетов, установленных через easy_install, хранится в файле easy-install.pth в директории /libs/site-packages/ вашего Python.
К счастью, удалять установленные через easy_install пакеты можно с помощью pip. Если же его нет, потребуется удалить пакет вручную и стереть сведения о нем из easy-install.pth.
Теперь вы умеете ставить и удалять пакеты для вашей версии Python.
Кстати, для тех, кто изучает Python, мы подготовили список полезных и практичных советов.
Содержание:развернуть
- Pip или pip3?
- Если pip не установлен
-
Windows
-
Linux (Ubuntu и Debian)
-
MacOS
- Как обновить PIP
- Команды PIP
- Пример работы с пакетами
PIP — это менеджер пакетов. Он позволяет устанавливать и управлять пакетами на Python.
Представьте себе ситуацию: вы собираете проект и подключаете множество сторонних библиотек для реализации своей задачи. Если это делать вручную, процесс выглядит примерно так:
- вы заходите на сайт, выбираете нужную версию пакета;
- скачиваете ее, разархивируете, перекидываете в папку проекта;
- подключаете, прописываете пути, тестируете.
Вполне вероятно, что эта версия библиотеки вообще не подходит, и весь процесс повторяется заново. А если таких библиотек 10? Устанавливать их вручную?
Нет 🙅🏻♂️
Менеджер пакетов PIP — решает данную проблему. Весь процесс установки пакета сводится к выполнению консольной команды pip install package-name
. Несложно представить, сколько времени это экономит.
Если вы работали с другими языками программирования, концепция pip может показаться вам знакомой. Pip похож на npm (в Javascript), composer (в PHP) или gem (в Ruby).
Pip является стандартным менеджером пакетов в Python
Pip или pip3?
В зависимости от того, какая версия Python установлена в системе, может потребоваться использовать pip3 вместо pip.
Если вы не знаете какая версия Python установлена на вашей системе, выполните следующие команды:
python --version
— для Python 2.xpython3 --version
— для Python 3.xpython3.8 --version
— для Python 3.8.x
Советуем использовать версию Python 3.6 и выше
Если команда «python» не найдена, установите Python по инструкции из предыдущей статьи.
Далее нужно убедиться, что сам PIP установлен и работает корректно. Узнать это поможет команда:
pip --version
Команда отобразит в консоли версию pip, путь до pip и версию python, для которой в дальнейшем будут устанавливаться пакеты:
pip 19.2.3 from /usr/local/lib/python3.8/site-packages/pip (python 3.8)
☝️ Важный момент: в зависимости от того, какую версию Python вы будете использовать, команда может выглядеть как pip
, pip3
или pip3.8
Альтернативный вариант вызова pip:
python3.7 -m pip install package-name
Флаг -m
сообщает Python-у запустить pip как исполняемый модуль.
Если pip не установлен
Pip поставляется вместе с Python, и доступен после его установки. Если по какой-то причине pip не установлен на вашей системе, установить его будет не сложно.
Windows
- Скачайте файл get-pip.py и сохраните у себя на компьютере.
- Откройте командную строку и перейдите в папку, в которой сохранен
get-pip.py
. - В командной строке выполните команду:
python get-pip.py
илиpython3 get-pip.py
. - PIP установлен 🎉!
Linux (Ubuntu и Debian)
Прежде, чем перейти к непосредственному описанию, хотим отметить, что все команды, описанные ниже, используются от имени root пользователя. Если же вы являетесь обычным пользователем на компьютере, то потребуется использовать команду sudo
, чтобы получить привилегии root.
Для Питона 2-й версии, выполните команду:
apt-get install python-pip
Для Питона 3-ей версии:
apt-get install python3-pip
MacOS
- скачайте файл
get-pip.py
командойcurl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
; - запустите скачанный файл командой:
python get-pip.py
илиpython3 get-pip.py
.
Должна появиться запись Successfully Installed. Процесс закончен, можно приступать к работе с PIP на MacOS!
Как обновить PIP
Иногда, при установке очередного пакета, можно видеть сообщение о том, что доступна новая версия pip.
WARNING: You are using pip version 19.2.3, however version 19.3.1 is available.
А в следующей за ней строке
You should consider upgrading via the 'python -m pip install --upgrade pip' command.
указана команда для обновления pip:
python -m pip install --upgrade pip
Команды PIP
Синтаксис pip выглядит следующим образом: pip + команда + доп. опции
pip <command> [options]
Со всеми командами pip можно ознакомиться, выполнив pip help . Информацию по конкретной команде выведет pip help <command>
.
Рассмотрим команды pip:
pip install package-name
— устанавливает последнюю версию пакета;pip install package-name==4.8.2
— устанавливает пакет версии 4.8.2;pip install package-name --upgrade
— обновляет версию пакета;pip download
— скачивает пакеты;pip uninstall
— удаляет пакеты;pip freeze
— выводит список установленных пакетов в необходимом формате ( обычно используется для записи вrequirements.txt
);pip list
— выводит список установленных пакетов;pip list --outdated
— выводит список устаревших пакетов;pip show
— показывает информацию об установленном пакете;pip check
— проверяет установленные пакеты на совместимость зависимостей;pip search
— по введенному названию, ищет пакеты, опубликованные в PyPI;pip wheel
— собирает wheel-архив по вашим требованиям и зависимостям;pip hash
— вычисляет хеши архивов пакетов;pip completion
— вспомогательная команда используется для завершения основной команды;pip help
— помощь по командам.
Пример работы с пакетами
PIP позволяет устанавливать, обновлять и удалять пакеты на компьютере. Ниже попробуем разобраться с работой менеджера pip на примере парсинга названий свежих статей на сайте habr.com.
- установим нужные пакеты;
- импортируем пакет в свой скрипт;
- разберемся, что такое
requirements.txt
; - обновим/удалим установленные пакеты.
Приступим 🙎🏻♂️
Шаг #1 Установка.
Для начала, нам необходимо установить beautifulsoup4 — библиотеку для парсинга информации с веб-сайтов.
pip3 install beautifulsoup4
pip найдет последнюю версию пакета в официальном репозитории pypi.org. После скачает его со всеми необходимыми зависимостями и установит в вашу систему. Если вам нужно установить определенную версию пакета, укажите её вручную:
pip3 install beautifulsoup4==4.8.2
Данная команда способна даже перезаписать текущую версию на ту, что вы укажите.
Также для работы beautifulsoup нам понадобится пакет lxml:
pip install lxml
☝️ Важный момент: по умолчанию pip устанавливает пакеты глобально. Это может привести к конфликтам между версиями пакетов. На практике, чтобы изолировать пакеты текущего проекта, создают виртуальное окружение (virtualenv).
Шаг #2 Импортирование в скрипте.
Для того чтобы воспользоваться функционалом установленного пакета, подключим его в наш скрипт, и напишем простой парсер:
from urllib.request import urlopen
from bs4 import BeautifulSoup
# скачиваем html
page = urlopen("https://habr.com/ru/top/")
content = page.read()
# сохраняем html в виде объекта BeautifulSoup
soup = BeautifulSoup(content, "lxml")
# Находим все теги "a" с классом "post__title_link"
all_a_titles = soup.findAll("a", { "class" : "post__title_link" })
# Проходим по каждому найденному тегу и выводим на экран название статьи
for a_title in all_a_titles:
print(a_title.text)
Шаг #3 requirements.txt.
Если вы просматривали какие-либо проекты Python на Github или где-либо еще, вы, вероятно, заметили файл под названием requirements.txt. Этот файл используется для указания того, какие пакеты необходимы для запуска проекта (в нашем случае beautifulsoup4 и lxml).
Файл requirements.txt
создается командой:
pip freeze > requirements.txt
и выглядит следующим образом:
beautifulsoup4==4.8.2
lxml==4.4.2
soupsieve==1.9.5
Теперь ваш скрипт вместе с файлом requirements.txt можно сохранить в системе контроля версий (например git).
Для работы парсера в новом месте (например на компьютере другого разработчика или на удаленном сервере) необходимо затянуть файлы из системы контроля версий и выполнить команду:
pip install -r requirements.txt
Шаг #4 Обновление/удаление установленных пакетов.
Команда pip list --outdated
выведет список всех устаревших пакетов. Обновить отдельно выбранный пакет поможет команда:
pip install package-name --upgrade
Однако бывают ситуации, когда нужно обновить сразу все пакеты из requirements.txt. Достаточно выполнить команду:
pip install -r requirements.txt --upgrade
Для удаления пакета выполните:
pip uninstall package-name
Для удаления всех пакетов из requirements.txt:
pip uninstall -r requirements.txt -y
Мы разобрали основы по работе с PIP. Как правило, этого достаточно для работы с большей частью проектов.