Change to non-deprecated NumPy C API

This is an issue to summarize the current status of NumPy C API support and what can/should be done to update Cython. (cc @mattip, this is what I was talking about yesterday - could be a useful thing to work on).

Currently Cython uses a NumPy API that has been deprecated since NumPy 1.6.0 in 2013.

• Example: https://stackoverflow.com/questions/25789055/cython-numpy-warning-about-npy-no-deprecated-api-when-using-memoryview

The Cython docs say here: “For the time being, it is just a warning that you can ignore.”.

It is about time to update this, it will save users a lot of confusion and shorten the build logs of many projects by hundreds of lines.

@nouiz wrote a summary of how to support both old (<1.6.0) and new API:

• https://github.com/scikit-learn/scikit-learn/issues/2573 (meant as a Cython issue, but never reopened here as far as I can tell)
• https://mail.python.org/pipermail/numpy-discussion/2013-November/068172.html

I would suggest that it’s either no longer needed to support the old <1.6 API, or make that opt-in rather than the default (to avoid all the build warnings while remaining compatible with very old NumPy versions). Would be good to get the opinion of the Cython team on this before starting work on this.

Here is an example of changing to the new API and avoiding the build warnings for scipy.optimize: https://github.com/scipy/scipy/pull/4351/files. Linking because the use of numpy_nodepr_api there (the defines and passing them to distutils) may be useful to other projects; it’s a bit nontrivial to get right.

[mattip]

I think we can close this. Things like this PR in scipy can be finished once cython releases 3.0.

[scoder]

Is this done now, or is there still anything missing?

Cython doesn’t currently set the NumPy deprecated API marker itself, so that’s left to users. Is that ok, or can/should we do anything else to help them?