Package install tools: pip and easy_install, by means of distribute

Tuesday 08 November 2011, 00:09:00 | python

This article is in English because it is about a topic that can be useful for a lot of people. It is about getting easy_install and pip going on your different Python versions.

Note: easy_install originally comes from the setuptools package, that has recently been superseded by distribute. Distribute is where it all starts, see its homepage: http://packages.python.org/distribute/

(>) TL;DR

  1. use your system's package manager if possible, to skip step 2-4 and perhaps even 5
  2. download and install distribute via its bootstrap script distribute_setup.py
  3. use easy_install to install pip (or use its bootstrap script get-pip.py) and after that, only use pip to install stuff. Forget about easy_install.
  4. use pip to install virtualenv
  5. create and active a virutalenv and install the rest of your packages there

Read on for more details about each of these steps.

(!) Before everything else: use your package manager!

If your system provides a package manager it is strongly advised to use that if it provides the packages you need. Don't screw around by installing stuff yourself if the package manager can do it for you. We're talking about the following:

(or similar names). Notice that often this only provides these packages for the system default python, so if you have another python version installed, read on. If you don't have a a package manager because you are on Windows or Mac OS X, read on.

Installing stuff in your system Python installations

Normally when installing third party packages they become part of the system Python installation. You will probably need to do these installation steps with administrator/root privileges, otherwise you can't write to the system library directories. It is preferred to not modify your system Python installation at all (other than using your system's package manager, if it has one). There's a better and safer way to customize your Python installations that doesn't mess with your system. It's called virtualenv, and is briefly discussed at the end of this article.

Installing distribute

Getting distribute often proves to be a non trivial task. Obviously, distribute can't depend on a package management tool to install itself. So it needs a bootstrapping script to do the dirty work. How to get this script? On the distribute homepage it provides a command using curl, but you can also use wget if you want:

curl -O http://python-distribute.org/distribute_setup.py
wget http://python-distribute.org/distribute_setup.py

If you don't have any of these tools (a common situation on Windows), don't worry. You can download the setup file with a normal web browser, or you can use Python itself:

python -c "import urllib; urllib.urlretrieve('http://python-distribute.org/distribute_setup.py', 'distribute_setup.py')"

For Python 3.x you should use this command instead:

python3 -c "import urllib.request as r; r.urlretrieve('http://python-distribute.org/distribute_setup.py', 'distribute_setup.py')"

Once you have the distribute_setup.py file you need to execute it with the correct python interpreter. This will install the distribute package for that version of python. If you want to install it into different versions of python that are available on your machine, you will have to execute the distribute_setup.py script with every version of the python interpreter that you want to install it in. For instance, if you have a system default python (let's assume 2.7), and another python 2.6 installed, you can do:

python2.6 distribute_setup.py
python distribute_setup.py

Make sure you run the command with the system default python last! This is because the install process creates a few command line tools (easy_install) and if you do it in the wrong order, the default easy_install script will point to a different python version.

You now have the following commands:

If you ever need to run the easy_install script for a specific version of Python on your system, use the one with the appropriate version suffix. The default easy_install script should point to your system's default Python version. Windows users: the easy_install scripts are actually executables on Windows and are located in the 'scripts' directory in the installation path of your Python version, for instance c:\Python27\Scripts\easy_install.exe

A better install tool: pip

While the easy_install is often enough to be able to do basic package installations, it is suggested to use pip instead. It is a replacement for easy_install and it provides more features, two of the most important ones being the ability to uninstall packages, and leaving you in a clean state if an install fails halfway trough. Because we already have easy_install it is now trivial to install pip:

easy_install-2.6 pip
easy_install pip

Again, use the command for your system default Python last, otherwise the default pip script will point to a different version of Python. Notice that you don't even need to use easy_install because pip also provides a bootstrap install script. See http://www.pip-installer.org/en/latest/installing.html for more details if you want to install it using this method. Anyway, after this you will have the following scripts:

As with easy_install, use the pip script with the desired version suffix to install stuff for that specific Python version on your system. The default pip should point to your system's default Python. On Windows, these commands are again located in the 'scripts' directory in your Python installation paths.

Installing packages

Installing a third party package should now be a breeze but don't forget that you need to install it separately for every Python version on your system that you want to use it in. For instance, to install the Pyro4 package for both the 2.6 and the system default Python, you use:

pip-2.6 install Pyro4
pip install Pyro4

Not messing with the system: virtualenv

Install virtualenv. It's good for you:

pip install virtualenv

This provides you with the virtualenv command line program (on Windows, it's in the scripts directory once again). With this tool you can create virtual Python installations on your system that you can modify, add and remove packages in, without messing with the system Python installation. It is a powerful tool so read its documentation, but it boils down to:

virtualenv myenv
cd myenv
. bin/activate
 ...now you are working in your virtual python environment...
deactivate
 ...back to normal...

Virtualenv copies pip (and easy_install) into the virtual environments so you can use them to install stuff. No need to be administrator now because everything is installed into the virtual environment.

Conclusion

Once you have virtualenv and pip running, all should be well. Getting them to work and installing the required distribute package can be a bit confusing. Also the way they interact with the different Python versions on your system is something to be aware of. I hope this article is helpful about both of these subjects and gets you going with using virtualenv and pip easily.

A short url for a direct link to this article is: http://tinyurl.com/pythonpip