I am trying to install the dependencies for a project which uses Poetry, inside a docker container.
If I remove the line /root/.local/bin/poetry config virtualenvs.create false Poetry installs, but then then project is installed inside a venv. Because this is inside a docker container, I don’t need or want the packages installed in a venv.
The reason for this is I want other apps and tools inside the container to have access to the installed dependencies. If the deps installed by poetry are in a venv, one has to run a program using poetry run foo to access those deps, but if foo forks new threads or subprocess, they aren’t inside the poetry shell environment and can’t won’t have access to the deps, so a multi-threaded or multi-process application started with poetry run foo doesn’t work.
Therefore I just want to install the poetry deps globally (which is fine, because this is contained in a container).
This is the Dockerfile:
FROM ubuntu:24.04
SHELL ["/bin/bash", "-c"] # Poetry doesn't work with sh
RUN apt-get update &&
apt-get install -y ca-certificates build-essential python3-dev pipx gpg libenchant-2-2
RUN pipx install poetry && pipx ensurepath
COPY ./pyproject.toml /pyproject.toml
COPY ./poetry.lock /poetry.lock
RUN /root/.local/bin/poetry config virtualenvs.create false &&
/root/.local/bin/poetry install -vv --no-interaction
The docker-compose file is very basic:
services:
app:
build:
context: ../
dockerfile: docker/Dockerfile
Build with: docker compose build
The is the error I get at build time:
...
2.506 - Installing sphinxcontrib-serializinghtml (1.1.5)
2.532 - Installing wrapt (1.16.0)
2.561 - Installing yarl (1.11.1)
3.539
3.539 Stack trace:
3.540
3.540 2 ~/.local/share/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/utils/env/base_env.py:338 in _run
3.540 output = subprocess.check_output(
3.540
3.540 1 /usr/lib/python3.12/subprocess.py:466 in check_output
3.540 return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
3.540
3.540 CalledProcessError
3.540
3.540 Command '['/usr/bin/python3.12', '/root/.local/share/pipx/venvs/poetry/lib/python3.12/site-packages/virtualenv/seed/wheels/embed/pip-24.2-py3-none-any.whl/pip', 'uninstall', 'packaging', '-y']' returned non-zero exit status 1.
3.540
3.540 at /usr/lib/python3.12/subprocess.py:571 in run
3.564 567│ # We don't call process.wait() as .__exit__ does that for us.
3.564 568│ raise
3.564 569│ retcode = process.poll()
3.564 570│ if check and retcode:
3.565 → 571│ raise CalledProcessError(retcode, process.args,
3.565 572│ output=stdout, stderr=stderr)
3.565 573│ return CompletedProcess(process.args, retcode, stdout, stderr)
3.565 574│
3.565 575│
3.565
3.565 The following error occurred when trying to handle this error:
3.565
3.565
3.565 Stack trace:
3.565
3.565 9 ~/.local/share/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/installation/executor.py:285 in _execute_operation
3.565 result = self._do_execute_operation(operation)
3.565
3.565 8 ~/.local/share/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/installation/executor.py:395 in _do_execute_operation
3.566 result: int = getattr(self, f"_execute_{method}")(operation)
3.566
3.566 7 ~/.local/share/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/installation/executor.py:527 in _execute_update
3.566 status_code = self._update(operation)
3.566
3.566 6 ~/.local/share/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/installation/executor.py:586 in _update
3.566 return self._install(operation)
3.566
3.566 5 ~/.local/share/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/installation/executor.py:576 in _install
3.566 self._remove(operation.initial_package)
3.566
3.566 4 ~/.local/share/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/installation/executor.py:596 in _remove
3.566 return self.run_pip("uninstall", package.name, "-y")
3.566
3.566 3 ~/.local/share/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/installation/executor.py:418 in run_pip
3.567 self._env.run_pip(*args, **kwargs)
3.567
3.567 2 ~/.local/share/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/utils/env/base_env.py:310 in run_pip
3.567 return self._run(cmd, **kwargs)
3.567
3.567 1 ~/.local/share/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/utils/env/generic_env.py:99 in _run
3.567 return super(VirtualEnv, self)._run(cmd, **kwargs)
3.567
3.567 EnvCommandError
3.567
3.567 Command ['/usr/bin/python3.12', '/root/.local/share/pipx/venvs/poetry/lib/python3.12/site-packages/virtualenv/seed/wheels/embed/pip-24.2-py3-none-any.whl/pip', 'uninstall', 'packaging', '-y'] errored with the following return code 1
3.567
3.567 Output:
3.567 error: externally-managed-environment
3.567
3.567 × This environment is externally managed
3.567 ╰─> To install Python packages system-wide, try apt install
3.567 python3-xyz, where xyz is the package you are trying to
3.567 install.
3.567
3.567 If you wish to install a non-Debian-packaged Python package,
3.567 create a virtual environment using python3 -m venv path/to/venv.
3.567 Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
3.567 sure you have python3-full installed.
3.567
3.567 If you wish to install a non-Debian packaged Python application,
3.567 it may be easiest to use pipx install xyz, which will manage a
3.567 virtual environment for you. Make sure you have pipx installed.
3.567
3.567 See /usr/share/doc/python3.12/README.venv for more information.
3.567
3.567 note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
3.567 hint: See PEP 668 for the detailed specification.
3.567
3.567
3.567 at ~/.local/share/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/utils/env/base_env.py:342 in _run
3.572 338│ output = subprocess.check_output(
3.572 339│ cmd, stderr=stderr, env=env, text=True, **kwargs
3.572 340│ )
3.572 341│ except CalledProcessError as e:
3.572 → 342│ raise EnvCommandError(e)
3.572 343│
3.572 344│ return output
3.572 345│
3.572 346│ def execute(self, bin: str, *args: str, **kwargs: Any) -> int:
3.572
3.572 Cannot install packaging.
What is interesting about this, is if I start a blank Ubuntu container with docker run -it --rm ubuntu:22.04 and copy and paste all the commands in the Dockerfile into the shell, everything works. So maybe this is some Docker issue?
From the error I guess that poetry is using pip or pipx behind the scenes and there is no way that I can find to pass the --break-system-packages option.