pymatgen fails to load MPRester

[jjkas]

Python version

Python 3.12.5

Pymatgen version

2024.9.17

Operating system version

Linux Mint V22, windows10 64 bit,

Current behavior

from pymatgen.ext.matproj import MPRester

A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.1.1 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled with NumPy 2.0.
Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.

If you are a user of the module, the easiest solution will be to
downgrade to 'numpy<2' or try to upgrade the affected module.
We expect that some modules will need time to support NumPy 2.

Traceback (most recent call last): File "", line 1, in
File "/home/jjkas/miniconda3/envs/Corvus/lib/python3.12/site-packages/pymatgen/ext/matproj.py", line 25, in
from pymatgen.core import SETTINGS
File "/home/jjkas/miniconda3/envs/Corvus/lib/python3.12/site-packages/pymatgen/core/init.py", line 13, in
from pymatgen.core.lattice import Lattice
File "/home/jjkas/miniconda3/envs/Corvus/lib/python3.12/site-packages/pymatgen/core/lattice.py", line 19, in
from pymatgen.util.coord import pbc_shortest_vectors
File "/home/jjkas/miniconda3/envs/Corvus/lib/python3.12/site-packages/pymatgen/util/coord.py", line 15, in
from pymatgen.util import coord_cython
Traceback (most recent call last):
File "", line 1, in
File "/home/jjkas/miniconda3/envs/Corvus/lib/python3.12/site-packages/pymatgen/ext/matproj.py", line 25, in
from pymatgen.core import SETTINGS
File "/home/jjkas/miniconda3/envs/Corvus/lib/python3.12/site-packages/pymatgen/core/init.py", line 13, in
from pymatgen.core.lattice import Lattice
File "/home/jjkas/miniconda3/envs/Corvus/lib/python3.12/site-packages/pymatgen/core/lattice.py", line 19, in
from pymatgen.util.coord import pbc_shortest_vectors
File "/home/jjkas/miniconda3/envs/Corvus/lib/python3.12/site-packages/pymatgen/util/coord.py", line 15, in
from pymatgen.util import coord_cython
File "pymatgen/util/coord_cython.pyx", line 1, in init pymatgen.util.coord_cython
ImportError: numpy.core.multiarray failed to import (auto-generated because you didn't call 'numpy.import_array()' after cimporting numpy; use 'numpy._import_array' to disable if you are certain you don't need it).

Expected Behavior

Since pymatgen change log states explicit fixes for version v2024.9.10, I would expect its requirements also to work with numpy >2.0. Not sure if this is a bug, but perhaps changing installation via pypi to require numpy<2.0 would be a good idea?
Notes:

1. Seems to be due to requirement of emmet-core of numpy<2.0.
2. Works on mac m2 with the same versions of python/pymatgen.
3. Works on windows 11 running as a virtual machine on the same mac m2.

Minimal example

>> from pymatgen.ext.matproj import MPRester
fails with numpy 2.1.1
works with numpy 1.26.

Relevant files to reproduce this bug

No response

[shyuep]

Pls state whether you are using MPRester basic or the mp-api version. If it is the mp-api version, then you will have to post this in that repo. If it is basic, then let us know.

[jjkas]

It happens with both, but it's really a problem with pymatgen rather than mp-api. If numpy>2.0 is installed, the following two commands both fail:

from pymatgen.ext.matproj import MPRester
from mp_api.client import MPRester

However, the problem in general is not a problem for mp-api since there is a requirement there to install numpy<2.0. Thus if both mp-api and pymatgen are pip installed, both work, since mp-api will downgrade numpy to <2.0, and pymatgen does not require numpy>2.0. If only pymatgen is installed in a clean environment, numpy does not have the <2.0 requirement and MPRester fails.

[DanielYang59]

If only pymatgen is installed in a clean environment, numpy does not have the <2.0 requirement and MPRester fails.

@jjkas I cannot reproduce this issue. Can you please confirm your pymatgen version? I would expect version v2024.9.10 and afterwards to fully support numpy 2.

import numpy as np
from pymatgen.ext.matproj import MPRester
import pkg_resources


print("NumPy version:", np.__version__)  # >>> NumPy version: 2.1.1
print("pymatgen version:", pkg_resources.get_distribution("pymatgen").version)  # >>> pymatgen version: 2024.9.17.1
print(dir(MPRester))  

[jjkas]

Windows 10 64 bit
As my environment is now, MPRester loads just fine.

Python 3.12.5 | packaged by Anaconda, Inc. | (main, Sep 12 2024, 18:18:29) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> from pymatgen.ext.matproj import MPRester
>>> import pkg_resources
<stdin>:1: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/>\pkg_resources.html
>>>
>>>
>>> print("NumPy version:", np.version) # >>> NumPy version: 2.1.1
\NumPy version: 1.26.4
>>> print("pymatgen version:", pkg_resources.get_distribution("pymatgen").version) # >>> pymatgen version: 2024.9.17.1
\pymatgen version: 2024.9.17.1
>>> print(dir(MPRester))
['class', 'delattr', 'dict', 'dir', 'doc', 'eq', 'format', 'ge', 'getattribute', 'getstate', 'gt', >'hash', 'init', 'init_subclass', 'le', 'lt', 'module', 'ne', 'new', 'reduce', 'reduce_ex', 'repr', >'setattr', 'sizeof', 'str', 'subclasshook', 'weakref']

[jjkas]

I'm now having trouble reproducing the problem myself. I'll see if I can get it to occur again.

[DanielYang59]

Cool, let me know if you found anything.

NumPy version: 1.26.4

Note you appear to be running numpy 1.26 while the original issue was reported on numpy 2.x, so please make sure you have the correct version of numpy installed.

I tried to run both numpy 1.26 and numpy 2.1 on my Windows machine and cannot reproduce this.