Scipy error: numpy.dtype size changed, may indicate binary incompatibility (and associated strange behavior)

Question

I am installing numpy/scipy/scikit-learn on OS X 10.9.4, and am getting errors about "numpy.dtype size changed, may indicate binary incompatibility".

Here's what I did to construct the repo:

mkvirtualenv thm
workon thm
pip install numpy scipy pandas ipython # and some other stuff
cd /path/to/our/repo
# run tests

Here's a traceback excerpt of a relevant warning (turned into an error because we use warnings.simplefilter('error') at the beginning of our tests):

======================================================================
ERROR: Failure: RuntimeWarning (numpy.dtype size changed, may indicate binary in
compatibility)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/ben/.virtualenvs/thm/lib/python2.7/site-packages/nose/loader.py",
 line 414, in loadTestsFromName
    addr.filename, addr.module)
  File "/Users/ben/.virtualenvs/thm/lib/python2.7/site-packages/nose/importer.py
", line 47, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/Users/ben/.virtualenvs/thm/lib/python2.7/site-packages/nose/importer.py
", line 94, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "/Users/ben/code/thm/alpha/prosper/base/stats/test_auc.py", line 3, in <m
odule>
    import sklearn.metrics
  File "/Users/ben/.virtualenvs/thm/lib/python2.7/site-packages/sklearn/metrics/
__init__.py", line 6, in <module>
    from .metrics import (accuracy_score,
  File "/Users/ben/.virtualenvs/thm/lib/python2.7/site-packages/sklearn/metrics/metrics.py", line 27, in <module>
    from scipy.spatial.distance import hamming as sp_hamming
  File "/Users/ben/.virtualenvs/thm/lib/python2.7/site-packages/scipy/spatial/__init__.py", line 90, in <module>
    from .ckdtree import *
  File "__init__.pxd", line 155, in init scipy.spatial.ckdtree (scipy/spatial/ckdtree.c:20570)
RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility

I'm told that this warning is caused by scipy being compiled against a different version of numpy than the one installed. However, I installed them all with pip in what I thought was a pretty standard way, so this shouldn't be a problem, I would think.

Weirdly, although running our entire test suite as a whole (via python -m unittest discover) gives these errors, running the individual tests (via python -m unittest <module>) works fine.

According to the tests, here's some relevant version info:

numpy version 1.9.0 (rev 07601a64cdfeb1c0247bde1294ad6380413cab66)
scipy version 0.14.0 (built against numpy 1.9.0)
sklearn version 0.15.2
pandas version 0.14.1

Happy to provide more info on request!

Answer 1

How did you build sklearn 0.14.1? Did you build it against the same version of numpy as you did for scipy?

Recent versions of scikit-learn, scipy and numpy have prebuilt-packages. In particular scikit-learn 0.15.2 should be binary compatible with numpy 1.7+. I think the same is true with scipy 0.14.0 but you said you built it yourself from source, which is not what pip should do by default (it should just install the prebuilt whl package).

Edit: have you tried to do:

pip install -U scipy scikit-learn pandas

to make sure that you are using the latest stable versions of the whl for those packages?

Edit: The comment below has the actual answer that works and is presumably why this answer was accepted. Namely:

pip uninstall -y scipy scikit-learn
pip install --no-binary scipy scikit-learn