August 2024 Update

I found that Everett Toews added python -m pip install --upgrade pip setuptools wheel from the answer /a/61980136/15233792 to solve my problem.

Original Question

I run my gitlab-ci pipeline locally in a ubuntu machine which is also a gitlab-runner. But inside the gitlab-runner env the job failed with ModuleNotFoundError: No module named 'pkg_resources'

The only command I execute:
gitlab-runner exec docker build_test

.gitlab-ci.yml

image: python:latest

workflow:
  rules:
    - if: $CI_PIPELINE_SOURCE == 'merge_request_event'
    - if: $CI_COMMIT_REF_NAME == 'main'

# Change pip's cache directory to be inside the project directory since we can
# only cache local items.
variables:
  PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pip"

# https://pip.pypa.io/en/stable/topics/caching/
cache:
  paths:
    - .cache/pip

before_script:
  - python --version ; pip --version  # For debugging
  - pip install virtualenv
  - virtualenv venv
  - source venv/bin/activate

stages:
  - build_test

build_test:
  stage: build_test
  script:
    - pip install -r requirements.txt --no-cache-dir
    - pip freeze
    - python run.py
    - sleep 5  # wait for server
    - python test.py

requirements.txt

Flask==3.0.2
Flask_Admin==1.6.1
Flask_Cors==4.0.0
Flask_Login
Flask-Session==0.6.0
Flask-SQLAlchemy==3.0.5
flask-security-too
bleach
email_validator
Werkzeug==3.0.1
PyJWT==2.8.0
bcrypt==4.1.2
wtforms

The job logs:

Runtime platform                                    arch=amd64 os=linux pid=3662085 revision=782e15da version=16.2.0
WARNING: You most probably have uncommitted changes. 
WARNING: These changes will not be tested.         
Running with gitlab-runner 16.2.0 (782e15da)
Preparing the "docker" executor
Using Docker executor with image python:latest ...
Authenticating with credentials from /home/kewen_zhu/.docker/config.json
Pulling docker image python:latest ...
Using docker image sha256:0218518c77bef087f9111b988785f0cef566c072474ee5324c7bb57faf07d93d for python:latest with digest python@sha256:e3d5b6f95ce66923b5e48a06ee5755abb097de96a8617c3f2f7d431d48e63d35 ...
Preparing environment
Running on runner--project-0-concurrent-0 via shark...
Getting source from Git repository
Fetching changes...
Initialized empty Git repository in /builds/project-0/.git/
Created fresh repository.
Checking out 2b33a34b as detached HEAD (ref is test/ci-pipeline)...

Skipping Git submodules setup
Restoring cache
Checking cache for build_test/test/ci-pipeline...
No URL provided, cache will not be downloaded from shared cache server. Instead a local version of cache will be extracted. 
Successfully extracted cache
Executing "step_script" stage of the job script
Using docker image sha256:0218518c77bef087f9111b988785f0cef566c072474ee5324c7bb57faf07d93d for python:latest with digest python@sha256:e3d5b6f95ce66923b5e48a06ee5755abb097de96a8617c3f2f7d431d48e63d35 ...
$ python --version ; pip --version
Python 3.12.5
pip 24.2 from /usr/local/lib/python3.12/site-packages/pip (python 3.12)
$ pip install virtualenv
Collecting virtualenv
  Downloading virtualenv-20.26.3-py3-none-any.whl.metadata (4.5 kB)
Collecting distlib<1,>=0.3.7 (from virtualenv)
  Downloading distlib-0.3.8-py2.py3-none-any.whl.metadata (5.1 kB)
Collecting filelock<4,>=3.12.2 (from virtualenv)
  Downloading filelock-3.15.4-py3-none-any.whl.metadata (2.9 kB)
Collecting platformdirs<5,>=3.9.1 (from virtualenv)
  Downloading platformdirs-4.2.2-py3-none-any.whl.metadata (11 kB)
Downloading virtualenv-20.26.3-py3-none-any.whl (5.7 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.7/5.7 MB 27.9 MB/s eta 0:00:00
Downloading distlib-0.3.8-py2.py3-none-any.whl (468 kB)
Downloading filelock-3.15.4-py3-none-any.whl (16 kB)
Downloading platformdirs-4.2.2-py3-none-any.whl (18 kB)
Installing collected packages: distlib, platformdirs, filelock, virtualenv
Successfully installed distlib-0.3.8 filelock-3.15.4 platformdirs-4.2.2 virtualenv-20.26.3
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable.It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
$ virtualenv venv
created virtual environment CPython3.12.5.final.0-64 in 227ms
  creator CPython3Posix(dest=/builds/project-0/venv, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, via=copy, app_data_dir=/root/.local/share/virtualenv)
    added seed packages: pip==24.1
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
$ source venv/bin/activate
$ pip install -r requirements.txt --no-cache-dir
Collecting Flask==3.0.2 (from -r requirements.txt (line 1))
  Downloading flask-3.0.2-py3-none-any.whl.metadata (3.6 kB)
Collecting Flask_Admin==1.6.1 (from -r requirements.txt (line 2))
  Downloading Flask_Admin-1.6.1-py3-none-any.whl.metadata (5.8 kB)
Collecting Flask_Cors==4.0.0 (from -r requirements.txt (line 3))
  Downloading Flask_Cors-4.0.0-py2.py3-none-any.whl.metadata (5.4 kB)
Collecting Flask_Login (from -r requirements.txt (line 4))
  Downloading Flask_Login-0.6.3-py3-none-any.whl.metadata (5.8 kB)
Collecting Flask-Session==0.6.0 (from -r requirements.txt (line 5))
  Downloading flask_session-0.6.0-py3-none-any.whl.metadata (2.7 kB)
Collecting Flask-SQLAlchemy==3.0.5 (from -r requirements.txt (line 6))
  Downloading flask_sqlalchemy-3.0.5-py3-none-any.whl.metadata (3.3 kB)
Collecting flask-security-too (from -r requirements.txt (line 7))
  Downloading flask_security_too-5.5.2-py3-none-any.whl.metadata (7.9 kB)
Collecting bleach (from -r requirements.txt (line 8))
  Downloading bleach-6.1.0-py3-none-any.whl.metadata (30 kB)
Collecting email_validator (from -r requirements.txt (line 9))
  Downloading email_validator-2.2.0-py3-none-any.whl.metadata (25 kB)
Collecting Werkzeug==3.0.1 (from -r requirements.txt (line 10))
  Downloading werkzeug-3.0.1-py3-none-any.whl.metadata (4.1 kB)
Collecting PyJWT==2.8.0 (from -r requirements.txt (line 11))
  Downloading PyJWT-2.8.0-py3-none-any.whl.metadata (4.2 kB)
Collecting bcrypt==4.1.2 (from -r requirements.txt (line 12))
  Downloading bcrypt-4.1.2-cp39-abi3-manylinux_2_28_x86_64.whl.metadata (9.5 kB)
Collecting wtforms (from -r requirements.txt (line 13))
  Downloading wtforms-3.1.2-py3-none-any.whl.metadata (5.3 kB)
Collecting Jinja2>=3.1.2 (from Flask==3.0.2->-r requirements.txt (line 1))
  Downloading jinja2-3.1.4-py3-none-any.whl.metadata (2.6 kB)
Collecting itsdangerous>=2.1.2 (from Flask==3.0.2->-r requirements.txt (line 1))
  Downloading itsdangerous-2.2.0-py3-none-any.whl.metadata (1.9 kB)
Collecting click>=8.1.3 (from Flask==3.0.2->-r requirements.txt (line 1))
  Downloading click-8.1.7-py3-none-any.whl.metadata (3.0 kB)
Collecting blinker>=1.6.2 (from Flask==3.0.2->-r requirements.txt (line 1))
  Downloading blinker-1.8.2-py3-none-any.whl.metadata (1.6 kB)
Collecting cachelib (from Flask-Session==0.6.0->-r requirements.txt (line 5))
  Downloading cachelib-0.13.0-py3-none-any.whl.metadata (2.0 kB)
Collecting sqlalchemy>=1.4.18 (from Flask-SQLAlchemy==3.0.5->-r requirements.txt (line 6))
  Downloading SQLAlchemy-2.0.32-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (9.6 kB)
Collecting MarkupSafe>=2.1.1 (from Werkzeug==3.0.1->-r requirements.txt (line 10))
  Downloading MarkupSafe-2.1.5-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.0 kB)
Collecting Flask-Principal>=0.4.0 (from flask-security-too->-r requirements.txt (line 7))
  Downloading Flask-Principal-0.4.0.tar.gz (5.5 kB)
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Collecting Flask-WTF>=1.1.2 (from flask-security-too->-r requirements.txt (line 7))
  Downloading flask_wtf-1.2.1-py3-none-any.whl.metadata (3.4 kB)
Collecting passlib>=1.7.4 (from flask-security-too->-r requirements.txt (line 7))
  Downloading passlib-1.7.4-py2.py3-none-any.whl.metadata (1.7 kB)
Collecting importlib_resources>=5.10.0 (from flask-security-too->-r requirements.txt (line 7))
  Downloading importlib_resources-6.4.2-py3-none-any.whl.metadata (3.9 kB)
Collecting six>=1.9.0 (from bleach->-r requirements.txt (line 8))
  Downloading six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB)
Collecting webencodings (from bleach->-r requirements.txt (line 8))
  Downloading webencodings-0.5.1-py2.py3-none-any.whl.metadata (2.1 kB)
Collecting dnspython>=2.0.0 (from email_validator->-r requirements.txt (line 9))
  Downloading dnspython-2.6.1-py3-none-any.whl.metadata (5.8 kB)
Collecting idna>=2.0.0 (from email_validator->-r requirements.txt (line 9))
  Downloading idna-3.7-py3-none-any.whl.metadata (9.9 kB)
Collecting typing-extensions>=4.6.0 (from sqlalchemy>=1.4.18->Flask-SQLAlchemy==3.0.5->-r requirements.txt (line 6))
  Downloading typing_extensions-4.12.2-py3-none-any.whl.metadata (3.0 kB)
Collecting greenlet!=0.4.17 (from sqlalchemy>=1.4.18->Flask-SQLAlchemy==3.0.5->-r requirements.txt (line 6))
  Downloading greenlet-3.0.3-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata (3.8 kB)
Downloading flask-3.0.2-py3-none-any.whl (101 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 101.3/101.3 kB 15.1 MB/s eta 0:00:00
Downloading Flask_Admin-1.6.1-py3-none-any.whl (7.5 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.5/7.5 MB 93.6 MB/s eta 0:00:00
Downloading Flask_Cors-4.0.0-py2.py3-none-any.whl (14 kB)
Downloading flask_session-0.6.0-py3-none-any.whl (8.6 kB)
Downloading flask_sqlalchemy-3.0.5-py3-none-any.whl (24 kB)
Downloading werkzeug-3.0.1-py3-none-any.whl (226 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 226.7/226.7 kB 165.4 MB/s eta 0:00:00
Downloading PyJWT-2.8.0-py3-none-any.whl (22 kB)
Downloading bcrypt-4.1.2-cp39-abi3-manylinux_2_28_x86_64.whl (698 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 698.9/698.9 kB 39.5 MB/s eta 0:00:00
Downloading Flask_Login-0.6.3-py3-none-any.whl (17 kB)
Downloading flask_security_too-5.5.2-py3-none-any.whl (418 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 418.9/418.9 kB 31.2 MB/s eta 0:00:00
Downloading bleach-6.1.0-py3-none-any.whl (162 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 162.8/162.8 kB 25.7 MB/s eta 0:00:00
Downloading email_validator-2.2.0-py3-none-any.whl (33 kB)
Downloading wtforms-3.1.2-py3-none-any.whl (145 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 146.0/146.0 kB 17.2 MB/s eta 0:00:00
Downloading blinker-1.8.2-py3-none-any.whl (9.5 kB)
Downloading click-8.1.7-py3-none-any.whl (97 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 97.9/97.9 kB 50.9 MB/s eta 0:00:00
Downloading dnspython-2.6.1-py3-none-any.whl (307 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 307.7/307.7 kB 22.4 MB/s eta 0:00:00
Downloading flask_wtf-1.2.1-py3-none-any.whl (12 kB)
Downloading idna-3.7-py3-none-any.whl (66 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 66.8/66.8 kB 31.9 MB/s eta 0:00:00
Downloading importlib_resources-6.4.2-py3-none-any.whl (34 kB)
Downloading itsdangerous-2.2.0-py3-none-any.whl (16 kB)
Downloading jinja2-3.1.4-py3-none-any.whl (133 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 133.3/133.3 kB 10.9 MB/s eta 0:00:00
Downloading MarkupSafe-2.1.5-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (28 kB)
Downloading passlib-1.7.4-py2.py3-none-any.whl (525 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 525.6/525.6 kB 8.4 MB/s eta 0:00:00
Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Downloading SQLAlchemy-2.0.32-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.2 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.2/3.2 MB 3.9 MB/s eta 0:00:00
Downloading cachelib-0.13.0-py3-none-any.whl (20 kB)
Downloading webencodings-0.5.1-py2.py3-none-any.whl (11 kB)
Downloading greenlet-3.0.3-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (625 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 625.7/625.7 kB 5.2 MB/s eta 0:00:00
Downloading typing_extensions-4.12.2-py3-none-any.whl (37 kB)
Building wheels for collected packages: Flask-Principal
  Building wheel for Flask-Principal (pyproject.toml): started
  Building wheel for Flask-Principal (pyproject.toml): finished with status 'done'
  Created wheel for Flask-Principal: filename=Flask_Principal-0.4.0-py3-none-any.whl size=5576 sha256=6af5476b363495ebc0bf3e4356fed8c68e1bb33df76e6fb0c5426b9d9f76400b
  Stored in directory: /tmp/pip-ephem-wheel-cache-jj0iastq/wheels/88/f9/fa/de59e4a5a9d8c319c22f841229ee4563907603d8ce819ea52b
Successfully built Flask-Principal
Installing collected packages: webencodings, passlib, typing-extensions, six, PyJWT, MarkupSafe, itsdangerous, importlib_resources, idna, greenlet, dnspython, click, cachelib, blinker, bcrypt, wtforms, Werkzeug, sqlalchemy, Jinja2, email_validator, bleach, Flask, Flask-WTF, Flask-SQLAlchemy, Flask-Session, Flask-Principal, Flask_Login, Flask_Cors, Flask_Admin, flask-security-too
Successfully installed Flask-3.0.2 Flask-Principal-0.4.0 Flask-SQLAlchemy-3.0.5 Flask-Session-0.6.0 Flask-WTF-1.2.1 Flask_Admin-1.6.1 Flask_Cors-4.0.0 Flask_Login-0.6.3 Jinja2-3.1.4 MarkupSafe-2.1.5 PyJWT-2.8.0 Werkzeug-3.0.1 bcrypt-4.1.2 bleach-6.1.0 blinker-1.8.2 cachelib-0.13.0 click-8.1.7 dnspython-2.6.1 email_validator-2.2.0 flask-security-too-5.5.2 greenlet-3.0.3 idna-3.7 importlib_resources-6.4.2 itsdangerous-2.2.0 passlib-1.7.4 six-1.16.0 sqlalchemy-2.0.32 typing-extensions-4.12.2 webencodings-0.5.1 wtforms-3.1.2

[notice] A new release of pip is available: 24.1 -> 24.2
[notice] To update, run: pip install --upgrade pip
$ pip freeze
bcrypt==4.1.2
bleach==6.1.0
blinker==1.8.2
cachelib==0.13.0
click==8.1.7
dnspython==2.6.1
email_validator==2.2.0
Flask==3.0.2
Flask-Admin==1.6.1
Flask-Cors==4.0.0
Flask-Login==0.6.3
Flask-Principal==0.4.0
Flask-Security-Too==5.5.2
Flask-Session==0.6.0
Flask-SQLAlchemy==3.0.5
Flask-WTF==1.2.1
greenlet==3.0.3
idna==3.7
importlib_resources==6.4.2
itsdangerous==2.2.0
Jinja2==3.1.4
MarkupSafe==2.1.5
passlib==1.7.4
PyJWT==2.8.0
six==1.16.0
SQLAlchemy==2.0.32
typing_extensions==4.12.2
webencodings==0.5.1
Werkzeug==3.0.1
WTForms==3.1.2
$ python run.py
Traceback (most recent call last):
  File "/builds/project-0/run.py", line 1, in <module>
    from user_api import app
  File "/builds/project-0/user_api/__init__.py", line 1, in <module>
    from user_api.app import app
  File "/builds/project-0/user_api/app.py", line 13, in <module>
    from flask_security import (
  File "/builds/project-0/venv/lib/python3.12/site-packages/flask_security/__init__.py", line 16, in <module>
    from .core import (
  File "/builds/project-0/venv/lib/python3.12/site-packages/flask_security/core.py", line 88, in <module>
    from .totp import Totp
  File "/builds/project-0/venv/lib/python3.12/site-packages/flask_security/totp.py", line 18, in <module>
    from passlib.pwd import genword
  File "/builds/project-0/venv/lib/python3.12/site-packages/passlib/pwd.py", line 16, in <module>
    import pkg_resources
ModuleNotFoundError: No module named 'pkg_resources'
ERROR: Job failed: exit code 1

FATAL: exit code 1

What I’ve tried:

pip install setuptools

or

pip install --upgrade setuptools
pip install --upgrade distribute 

but neither works on my side.