Fix "ImportError: cannot import name 'appengine' from 'requests.packages.urllib3.contrib'"

Problem statement

You're running Kubeflow Pipelines (KFP) on Vertex AI using the kfp.v2 SDK (specifically version 2.0.0b15) and suddenly encounter the following error without any major code changes:

ImportError: cannot import name 'appengine' from 'requests.packages.urllib3.contrib'

This error occurs during pipeline execution immediately when components start to run. The key characteristics are:

• Pipelines were previously working but broke suddenly
• The error appears even with newly created components
• No pipeline code modifications were made except trivial changes (like configuration tweaks)
• The stack trace doesn't point to your application code

Why the sudden break?

This error emerged suddenly in May 2023 due to an unexpected dependency conflict between new releases (urllib3==2.0.0 and requests-toolbelt==1.0.0) pushed to PyPI and Kubeflow Pipelines versions.

Recommended solutions

Choose the most appropriate solution based on your environment constraints:

Solution 1: Upgrade the KFP SDK (recommended)

The KFP team fixed the compatibility issue in:

• kfp>=1.8.21 for V1 API
• kfp>=2.0.0-b16 for V2 API

pip install --upgrade kfp==2.0.0b16

If you're using V1 SDK:

pip install --upgrade kfp==1.8.21

Verification

After upgrading, check your installed version:

pip show kfp

Solution 2: Pin problematic dependencies (temporary workaround)

If you can't upgrade KFP immediately, pin these dependency versions in your Dockerfile:

FROM python:3.9

RUN pip install urllib3==1.26.15 requests-toolbelt==0.10.1 \
    && pip install kfp==2.0.0b15  # Your existing KFP version

Build and push this custom container to your registry, then reference it in your components:

@component(
    base_image="gcr.io/YOUR_REGISTRY/custom-kfp-image:latest"
)
def your_component(...):

Solution 3: Add appengine-python-standard dependency

Explicitly install the missing dependency in all components:

@component(
    base_image="python:3.7",
    packages_to_install=[
        "appengine-python-standard",  # Critical fix
        # Your other dependencies
    ]
)
def train_model(...):

Solution 4: Upgrade Python base image

Newer Python images include compatible dependency versions:

@component(
    base_image="python:3.11",  # Upgrade to latest supported
)

Why these solutions work

The root cause was requests==2.38.0's incompatibility with urllib3>=2.0. Specifically:

1. requests-toolbelt>=1.0.0 dropped the appengine module
2. KFP versions before 1.8.21/2.0.0b16 implicitly relied on this module
3. Auto-updating dependencies broke existing pipelines

The solutions solve this by either:

• Updating to KFP versions with dependency fixes
• Pinning known-working dependency versions
• Explicitly providing the removed module
• Using base images with modern dependency resolutions

Additional contexts

The same error pattern appears in other tools with dependency chains involving requests and urllib3:

Twine users:

pip install --upgrade twine requests-toolbelt

Poetry users (in GitHub Actions):
Update to Poetry 1.4.2+ in your workflow

gql package users:
Update dependencies:

pip install --upgrade gql==3.5.0 requests-toolbelt==1.0.0

System package conflicts:
If using system utilities like certbot:

sudo pip3 uninstall urllib3 requests requests-toolbelt
sudo apt update && sudo apt install --reinstall python3-certbot

Best practices to prevent recurrence

1. Pin exact versions in production Dockerfiles:

   RUN pip install kfp=2.0.0b16 \
       requests-toolbelt=1.0.0 \
       urllib3=1.26

2. Use distinct virtual environments for different projects to prevent dependency conflicts

3. Implement CI/CD dependency caching to prevent unexpected upgrades

On base images

Always explicitly declare Python versions (FROM python:3.10-slim) rather than using floating tags (FROM python:latest)

For a complete dependency resolution reference, consult the KFP compatibility matrix for your SDK version.