FTBFS with LLVM 9 and 10

/usr/bin/python3 /builddir/build/BUILD/llvmlite-0.32.0/ffi/build.py
LLVM version... 10.0.0

Traceback (most recent call last):
  File "/builddir/build/BUILD/llvmlite-0.32.0/ffi/build.py", line 168, in <module>
    main()
  File "/builddir/build/BUILD/llvmlite-0.32.0/ffi/build.py", line 158, in main
    main_posix('linux', '.so')
  File "/builddir/build/BUILD/llvmlite-0.32.0/ffi/build.py", line 120, in main_posix
    raise RuntimeError(msg)
RuntimeError: Building llvmlite requires LLVM 7.0.x, 7.1.x or 8.0.x, got '10.0.0'. Be sure to set LLVM_CONFIG to the right executable path.
Read the documentation at http://llvmlite.pydata.org/ for more information about building llvmlite.

LLVM 9 was released back in Sept 2019. LLVM 10 was released 2 weeks ago.

Please update.

Your project is using LLVM. That was your choice. LLVM updates very quickly and does not promise source compatibility. Therefore, your project must test frequently with LLVM trunk and make sure you’re making releases at least as often as LLVM itself (that is, every 6 months). Anything else is irresponsible.

I recommend a CI hook that tests periodically with LLVM.

[seibert]

Hi, Numba dev here. We appreciate the desire to stay up to date with LLVM (we also want to stay current) but to be clear, these are our priorities (ranked in descending order):

1. Ensure that llvmlite delivers a version of LLVM that will work for Numba users on all of the platforms we support. That means the LLVM version we ship with needs to work with x86, x86_64, armv7, aarch64, and ppc64le, as well as on some combinations of Windows, Mac, and Linux for those architectures. As a result, we aggressively pin to LLVM releases we have tested extensively, statically link to LLVM to insulate us from untested system libraries, and even ship with patched LLVM when we have no other choice to fix problems. LLVM’s primary consumer is clang, and so bugs that only manifest with JIT use cases will appear from time to time, and we have to work around them until fixed. Julia also has to extensively patch LLVM (much more than we do) as well. Even now, we have to ship PPC with LLVM 7 because there are critical bugs in LLVM 8 and 9 (haven’t checked 10 yet) that corrupt code generated by the JIT on that platform.

2. Ensure llvmlite works for non-Numba users. We are excited to see people find other uses for llvmlite, and those users likely would prefer faster LLVM updates, but this is a much smaller user group.

3. Track the most recent release of LLVM. Newer LLVM potentially (but not always) will generate better performing code, and support newer chips.

We do have a CI system to check all these combinations with new LLVM releases, but we have also been humbled by really hard to debug problems deep in the compiler backend that affected our users. As a result, we have adopted a very conservative approach to updating LLVM. We prefer to update LLVM early in a development cycle so we have more time to catch subtle bugs that emerge from novel combinations of user code, and slip through the Numba test suite (7000+ tests) on initial evaluation.

LLVM 9 also brought with it some concerns about how to accommodate newer compiler requirements on Windows along side the compiler requirements for ABI compatibility of Python 2 and 3 extensions. Ultimately we decided to punt on LLVM 9 until after we dropped Python 2 in January, and then during the last development cycle we decided there were enough high risk changes going out that an LLVM update was not worth the additional risk unless it would also fix PPC. Now that LLVM 10 is out and we’ve dropped Python 2 and Python 3.5, we almost certainly skip LLVM 9 and go straight to 10.

All that said, we have definitely heard the feedback that for people who are building llvmlite from source, they want to experiment with other LLVM versions at their own risk. #582 will allow people to bypass the version checking with only a warning.

We appreciate the frustration expressed here, but we would ask that you not assume incompetence or irresponsibility when the situation might be due to developers juggling requirements beyond your own. In return, we’ll try to be more transparent about where we are in evaluating new LLVM releases, as it clearly is causing confusion for the community.

As a first step, I’ve created #583 where we will track our progress on LLVM 10, and I’m closing this issue so we can focus our discussion over there.

[flying-sheep]

While the technical assessment might well be true (I don’t know, there might have been no other choice than to go this way), your demanding attitude is completely wrong. You aren’t paying for support here, you can’t demand someone to pour in free time.

I propose you rephrase this until it only contains constructive ideas on how to improve this project (if you indeed have any) without the entitled attitude. It’s OK to be annoyed by a fact of life. But don’t blame the ones who worked towards achieving the thing you want, that’s neither smart nor very nice.