Installing Python Packages Offline

Suppose we want to install Ansible offline.

Creating a Virtualenv ¶

We will create a virtual environment to download all dependencies without accidentally including unnecessary ones. First, let’s install virtualenv.

1
2
python3 -m pip install --upgrade --user virtualenv
python3 -m virtualenv venv

Now we can activate it.

1
. ./venv/bin/activate

Now, for the duration of the terminal session, we are in an isolated Python virtual environment.

Preparing Required Packages ¶

By installing only what is needed in a clean environment, we will also pull in all dependencies. Additionally, make sure to include setuptools and pip.

1
python3 -m pip install --upgrade pip setuptools ansible

Now we can create a list of what we have.

1
python3 -m pip freeze > requirements.txt

You can simply add pip and setuptools to requirements.txt without specifying versions (if they aren’t already in the list).

Example requirements.txt

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
ansible==3.4.0
ansible-base==2.10.11
cryptography==2.8
jinja2==2.11.3
netaddr==0.7.19
pbr==5.4.4
jmespath==0.9.5
ruamel.yaml==0.16.10
ruamel.yaml.clib==0.2.2
MarkupSafe==1.1.1
pip
setuptools

Having the list of packages with dependencies, we can download them even without venv.

1
python3 -m pip download --dest my-pip-packages -r requirements.txt

For convenience, let’s pack everything into an archive.

1
tar -cpvzf pips.tgz my-pip-packages requirements.txt

Installing in an Offline Environment ¶

Python should already be installed in advance.

Unpack your archive and install everything you brought. Pip and setuptools should be installed separately at the beginning, followed by everything else. If the command is not run as a superuser, you need to either:

Add sudo at the beginning
Add the --user argument to pip to install packages only for the user running the command

1
2
3
tar -xpzvf pips.tgz
python3 -m pip install --upgrade --no-index --find-links python-packages pip setuptools
python3 -m pip install --upgrade --no-index --find-links my-pip-packages -r requirements.txt

Now you can run Ansible!

1
2
3
4
5
6
7
root@289f34a67cbd:/host# ansible --version
ansible 2.10.11
  config file = None
  configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.6/dist-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 3.6.9 (default, Jan 26 2021, 15:33:00) [GCC 8.4.0]

Locales ¶

There have been cases where in a severely stripped-down environment, like Docker Ubuntu 18.04, the package installation fails with the following error.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
ERROR: Exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/pip/_internal/cli/base_command.py", line 173, in _main
    status = self.run(options, args)
  File "/usr/local/lib/python3.6/dist-packages/pip/_internal/cli/req_command.py", line 203, in wrapper
    return func(self, options, args)
  File "/usr/local/lib/python3.6/dist-packages/pip/_internal/commands/install.py", line 316, in run
    reqs, check_supported_wheels=not options.target_dir
...
  File "/usr/local/lib/python3.6/dist-packages/pip/_internal/operations/prepare.py", line 249, in unpack_url
    unpack_file(file.path, location, file.content_type)
  File "/usr/local/lib/python3.6/dist-packages/pip/_internal/utils/unpacking.py", line 256, in unpack_file
    untar_file(filename, location)
  File "/usr/local/lib/python3.6/dist-packages/pip/_internal/utils/unpacking.py", line 226, in untar_file
    with open(path, "wb") as destfp:
UnicodeEncodeError: 'ascii' codec can't encode character '\xe9' in position 117: ordinal not in range(128)

This can be resolved by setting environment variables before installing packages.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
export LANGUAGE=""
export LANG="C.UTF-8"
export LC_CTYPE="C.UTF-8"
export LC_NUMERIC="C.UTF-8"
export LC_TIME="C.UTF-8"
export LC_COLLATE="C.UTF-8"
export LC_MONETARY="C.UTF-8"
export LC_MESSAGES="C.UTF-8"
export LC_PAPER="C.UTF-8"
export LC_NAME="C.UTF-8"
export LC_ADDRESS="C.UTF-8"
export LC_TELEPHONE="C.UTF-8"
export LC_MEASUREMENT="C.UTF-8"
export LC_IDENTIFICATION="C.UTF-8"
export LC_ALL="C.UTF-8"