NumPy Binary Incompatibility Error: Expected 88 from C header, got 80 from PyObject

This error occurs when there's a mismatch between the NumPy version used to build a Python package and the version used at runtime. It's a common issue with packages that depend on NumPy's C API, particularly when mixing NumPy versions before and after the significant ABI change in version 1.20.0.

Problem Overview

The error message typically appears as:

ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 88 from C header, got 80 from PyObject

This occurs because:

• NumPy 1.20.0 introduced changes to the internal structure of PyArrayObject
• Packages built with NumPy ≥1.20.0 are incompatible with NumPy ≤1.19.5 at runtime
• The size of the internal structure changed from 80 bytes to 88 bytes

Root Cause: NumPy ABI Change

In NumPy 1.20.0, the development team made significant changes to the C API:

// Before NumPy 1.20.0 (ndarraytypes.h)
#define NPY_SIZEOF_PYARRAYOBJECT (sizeof(PyArrayObject_fields))

// After NumPy 1.20.0 - this define was removed

The PyArrayObject_fields structure gained a new member:

void *_buffer_info;  /* private buffer info, tagged to allow warning */

This change increased the structure size from 80 to 88 bytes, creating a binary incompatibility between versions.

Solutions

1. Upgrade NumPy (Recommended)

The simplest solution is to ensure you're using NumPy ≥1.20.0:

pip install numpy>=1.20.0

Or install a specific compatible version:

pip install numpy==1.26.4

2. Rebuild Problematic Packages

If you cannot upgrade NumPy (e.g., due to other dependencies), rebuild the problematic package against your current NumPy version:

pip uninstall pyxdameraulevenshtein
pip install pyxdameraulevenshtein --no-binary pyxdameraulevenshtein

3. Install NumPy First

When setting up a new environment, install NumPy first before other packages:

# Create fresh environment
python -m venv myenv
source myenv/bin/activate

# Install NumPy first
pip install numpy==1.21.5

# Then install other packages
pip install your-other-packages

4. Use Compatible Package Versions

Some packages have newer versions that avoid this issue. For example, with pyxdameraulevenshtein:

pip install pyxdameraulevenshtein>=1.7.0

5. Environment-Specific Solutions

pip

# Force reinstall NumPy
pip uninstall numpy -y
pip install numpy

# Or upgrade specifically
pip install --upgrade numpy

conda

# Conda solution
pip uninstall numpy
conda install -y -c conda-forge numpy

Dockerfile

# Docker solution
RUN pip install --upgrade --no-binary numpy numpy

Common Scenarios and Fixes

Scenario 1: TensorFlow Compatibility

When using TensorFlow with specific NumPy requirements:

# TensorFlow may require older NumPy
pip install numpy~=1.19.2

# Then rebuild problematic packages
pip install pyxdameraulevenshtein --no-binary pyxdameraulevenshtein

Scenario 2: Python Version Differences

The issue often appears in Python 3.7+ but may not occur in Python 3.6 because NumPy 1.19.5 was the latest version supporting Python 3.6.

Scenario 3: Package-Specific Solutions

For specific packages:

gensim

pip install gensim==3.8.3

pycocotools

pip install pycocotools==2.0.0

personal packages

# For custom packages, modify pyproject.toml
requires = ["numpy==1.21.0"]  # Instead of "numpy>=1.21.0"

Prevention Best Practices

1. Pin NumPy Version: Specify exact NumPy versions in your requirements
2. Isolated Environments: Use virtual environments for each project
3. Build Isolation: Use --no-binary or --no-build-isolation when appropriate
4. Order Matters: Install NumPy before other scientific packages
5. Regular Updates: Keep packages updated to benefit from compatibility fixes

Under the Hood: Technical Details

The error occurs because:

1. Python packages with C extensions compile against specific NumPy headers
2. The compiled extension expects certain memory layouts and structure sizes
3. When runtime NumPy version differs, memory access becomes incorrect
4. This leads to the size mismatch error or potential crashes

WARNING

Mixing NumPy versions before and after 1.20.0 creates undefined behavior that may cause crashes or incorrect results, not just error messages.

Conclusion

The "numpy.ndarray size changed" error stems from NumPy's ABI change in version 1.20.0. The most reliable solutions are:

1. Using NumPy ≥1.20.0 consistently across all packages
2. Rebuilding packages against your specific NumPy version when upgrading isn't possible
3. Maintaining version consistency throughout your environment

For most users, upgrading to the latest compatible NumPy version resolves the issue effectively.