OnCallReady

Lesson 21.29 · Spring Boot Runtime, Resilience & Python Ops · 11 min read

Python for operations: the subset, venv, and why never system pip

In plain words

Imagine a shared family toolbox that your dad keeps in exact order, because the car and the boiler depend on those exact tools. If you swap his screwdriver for a newer one you like, something may stop working next month and nobody will know why. So you get your own small toolbox for your project, and fill it with whatever you need.

That is a Python virtual environment. The system Python in /usr/lib/python3 belongs to apt, because Ubuntu tools like unattended-upgrades and cloud-init depend on it, and PEP 668 makes pip install refuse to touch it. python3 -m venv .venv makes your own toolbox, a directory with its own bin/python and packages, and pip freeze > requirements.txt writes down exactly what's in it.

The subset, and nothing else

The problem. Bash gets painful past 100 lines: no real data structures, fragile error handling, no tests. Platform teams write their health checkers, cleanup jobs and report generators in Python. You already know JavaScript; this is the small part of Python those jobs need, and the one installation rule that protects the OS.

What you need to know already: bash scripts and exit codes (6.1-6.5), apt and sudo (1.11), PATH and which (1.9), from your frontend work: npm, package.json and node_modules.

Python in JavaScript terms, before anything else:

Pythonthe npm world
pip install Xnpm install X - the package installer
a venv (virtual environment)a project-local node_modules - packages for one project only
requirements.txt with pinned versionsa lockfile (package-lock.json)
a module / importa module / import
an exception (raise, try/except)throw, try/catch

You already program. Python for platform work is vocabulary: a CLI that parses arguments, logs properly, calls an HTTP API with timeouts, runs a command, reads some files, exits with a meaningful code, and has tests. That is the whole list:

venv, pip, requirements.txt / pyproject.toml     environments
argparse (or click)                              CLIs
requests + Session + timeouts + retry adapters   HTTP
subprocess.run([...], check=True)               running commands
pathlib.Path                                    files
dataclasses, json, type hints                   data
logging                                         output that survives being unattended
custom exceptions + exit codes                  failure
pytest + fixtures                               tests
kubernetes client                               the SDK you will use most (lesson 21.34)

(An SDK is a library for talking to one system's API from code.)

Not: Django, Flask, pandas, notebooks, asyncio frameworks. None of it is the job.

About this box (simulator). Real Ubuntu has CPython. This trainer runs a restricted interpreter for exactly this subset - the standard modules above plus requests, pytest, click, kubernetes and cloud-SDK stubs from a small offline package index. Anything outside the subset fails loudly with a (simulator) message rather than pretending to work. Everything you type here is real Python and works the same on the VM.

System Python is not yours

$ python3 --version
Python 3.14.4
$ pip install requests
Command 'pip' not found, but can be installed with:
sudo apt install python3-pip
$ sudo apt install -y python3-pip && pip install requests
error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.
    ...
    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.

That is PEP 668 (a PEP = Python Enhancement Proposal, a numbered design document; 668 is the "externally managed environment" rule), on by default since Ubuntu 23.04. /usr/lib/python3 belongs to apt: tools like unattended-upgrades and cloud-init are Python programs that depend on exact versions of those packages. A sudo pip install that upgrades one of them breaks the OS in ways that show up weeks later. --break-system-packages exists; the name is the documentation.

venv

$ mkdir -p ~/oncall-lab/labs/4a-runtime/python && cd ~/oncall-lab/labs/4a-runtime/python
$ python3 -m venv .venv          # -m = run a module as a program; this one creates .venv/
The virtual environment was not created successfully because ensurepip is not
available.  On Debian/Ubuntu systems, you need to install the python3-venv
package using the following command.

    apt install python3.14-venv
...
$ sudo apt install -y python3-venv
$ python3 -m venv .venv
$ ls .venv
bin  include  lib  lib64  pyvenv.cfg
$ source .venv/bin/activate
# the prompt now starts with (.venv)
$ which python pip
/home/learner/oncall-lab/labs/4a-runtime/python/.venv/bin/python
/home/learner/oncall-lab/labs/4a-runtime/python/.venv/bin/pip
$ pip install requests pytest
...
Successfully installed certifi-2026.8.14 charset-normalizer-3.4.3 idna-3.10 ... requests-2.34.2 urllib3-2.5.0
$ deactivate

Pinning

(.venv) $ pip freeze > requirements.txt
(.venv) $ cat requirements.txt
certifi==2026.8.14
charset-normalizer==3.4.3
idna==3.10
requests==2.34.2
urllib3==2.5.0
(.venv) $ pip install -r requirements.txt        # on the next machine, the same versions

pip freeze prints every installed package with its exact version; pip install -r FILE installs from such a list. requirements.txt with exact versions is the simple lock. A pyproject.toml (Python's package.json) declares the project and its (looser) dependencies; tools like pip-tools, uv or poetry produce locks from it. For an ops tool, a venv plus a pinned requirements.txt is enough.

pipx for tools you just want to run

# on the real VM (pipx is not simulated here)
sudo apt install -y pipx
pipx install httpie        # its own venv, its command on your PATH

What you can now do

Why it helps

Platform engineers write a lot of small Python tools: health checkers, cleanup jobs, scripts that query the Kubernetes API, CI helpers. The first time you run sudo pip install on a server and break apt or cloud-init, you lose an afternoon or a VM. Knowing venv and PEP 668 prevents that, and lets you deploy a tool properly: its own venv, pinned requirements, and a systemd unit calling /opt/tool/.venv/bin/python by path.

It also comes up in CI images and Dockerfiles you'll review: pip into a venv or use pip install --user in a container, pin versions, don't rely on activate in scripts. Keeping to the ops subset of Python means you learn exactly what you'll use, without wandering into frameworks you won't.

Commands in this lesson

python3 pip apt mkdir ls source which deactivate

FAQ

What does "externally-managed-environment" mean?

It is PEP 668, enabled on Ubuntu since 23.04: the system Python is marked as managed by apt, and pip refuses to install into it. The reason is that OS tools are Python programs depending on exact package versions, and a global pip install that upgrades one of them can break the system later. Use apt install python3-xyz for system-wide packages, a venv for your own projects, or pipx for command-line tools. --break-system-packages exists and means what it says.

Do I have to activate the venv in scripts and cron jobs?

No. source .venv/bin/activate only prepends .venv/bin to PATH for your interactive shell. Scripts, cron jobs and systemd units should call the venv's interpreter directly by path, for example ExecStart=/opt/tool/.venv/bin/python /opt/tool/check.py. That interpreter automatically uses the venv's packages. A shebang of #!/usr/bin/env python3 only picks the venv's Python when the venv is on PATH, which it usually isn't under cron.

Should I commit the venv to Git?

No. A venv contains absolute paths and links to the system interpreter, so it doesn't move between machines, and it is large. Commit requirements.txt with pinned versions, produced by pip freeze, or a pyproject.toml plus a lock file, and recreate the venv on each machine with python3 -m venv .venv and pip install -r requirements.txt. Add .venv/ to .gitignore.

What's the difference between requirements.txt and pyproject.toml?

requirements.txt is a list of packages, usually with exact versions from pip freeze, which works as a simple lock file. pyproject.toml describes a project: its name, Python version and dependencies, typically with looser version ranges. Tools like pip-tools, uv or poetry generate a lock from it. For a small ops tool, a venv and a pinned requirements.txt are enough; for a package others install, use pyproject.toml.

When should I use pipx instead of a venv?

When you want to install a command-line tool rather than build a project. pipx install httpie creates a dedicated venv for that tool and puts its command on your PATH, so you can run it anywhere without affecting the system Python or your projects. A venv is for code you're developing, with dependencies you import. On Ubuntu, pipx itself comes from apt.

In an interview Mid

What is a Python virtual environment, and how would you install a small Python tool on a server?

A venv is a directory with its own bin/python (a link to the system interpreter) and its own site-packages - packages for one project only, like a project-local node_modules. Nothing global changes.

Why it matters on a server: system Python belongs to apt. Tools like unattended-upgrades and cloud-init depend on exact versions of its packages, so sudo pip install can break the OS weeks later. Ubuntu blocks it (PEP 668, "externally managed environment"); --break-system-packages is named honestly.

Installing a tool:

  1. python3 -m venv /opt/tool/.venv (needs python3-venv).
  2. /opt/tool/.venv/bin/pip install -r requirements.txt - a list pinned with pip freeze (exact versions = a lockfile).
  3. Run it by the venv's interpreter path: ExecStart=/opt/tool/.venv/bin/python /opt/tool/check.py. systemd and cron do not activate anything; activate only prepends .venv/bin to PATH.

Never commit the venv; recreate it from the pinned list. For CLI tools you just want to run: pipx.

Also asked: Why is running sudo pip install on a production server dangerous? · How do you pin a Python tool's dependencies? · How would you run a Python script from a systemd timer so it uses the right packages?

Practise this lesson in the terminal Free, in your browser - a real Ubuntu terminal to try it in, with missions that check your work.