Allow llvmlite to also link object code

Currently llvmlite can produce the LLVM IR source code by:

str(module)

it can also read this IR source code and generate a machine code object file using:

    llvm.initialize()
    llvm.initialize_native_asmprinter()
    llvm.initialize_native_target()
    target = llvm.Target.from_triple(v.module.triple)
    target_machine = target.create_target_machine()
    mod = llvm.parse_assembly(str(module))
    mod.verify()
    with open("%s.o" % basename, "wb") as o:
        o.write(target_machine.emit_object(mod))

But it seems llvmlite does not have the functionality to actually link these object files into an executable. One has to do it by hand, e.g. using gcc:

gcc -o a.out file.o

or clang:

clang -o a.out file.o

or using ld directly, though the exact invocation of ld is platform dependent (one has to link the C library manually, as well as the crt1.o). One can pass the -v option to either gcc or clang to figure out the platform dependent line for ld, e.g. on my machine it is:

 "/usr/bin/ld" -z relro --hash-style=gnu --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o c /usr/lib/gcc/x86_64-linux-gnu/6.1.1/../../../x86_64-linux-gnu/crt1.o /usr/lib/gcc/x86_64-linux-gnu/6.1.1/../../../x86_64-linux-gnu/crti.o /usr/lib/gcc/x86_64-linux-gnu/6.1.1/crtbegin.o -L/usr/lib/gcc/x86_64-linux-gnu/6.1.1 -L/usr/lib/gcc/x86_64-linux-gnu/6.1.1/../../../x86_64-linux-gnu -L/lib/x86_64-linux-gnu -L/lib/../lib64 -L/usr/lib/x86_64-linux-gnu -L/usr/lib/gcc/x86_64-linux-gnu/6.1.1/../../.. -L/home/certik/repos/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-7.2.0/llvm-5.0.0-vulzawogiwpyst64drjcp5wxgl5inldr/bin/../lib -L/lib -L/usr/lib expr2.o -L/home/certik/repos/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-7.2.0/llvm-5.0.0-vulzawogiwpyst64drjcp5wxgl5inldr/lib -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed /usr/lib/gcc/x86_64-linux-gnu/6.1.1/crtend.o /usr/lib/gcc/x86_64-linux-gnu/6.1.1/../../../x86_64-linux-gnu/crtn.o

However, since clang knows how to link object files into an executable, there must be a way to link from C++, it might not be very well exposed, but at least the clang driver must know how to do that.

It would be nice to expose this from llvmlite.

Alternatively, if that is out of scope of llvmlite, why does llvmlite have the functionality to produce object files in the first place, if they can’t be linked? One does not need the object file for the JIT function (for numba) it seems. So if the philosophy behind llvmlite is to just expose JIT, then it doesn’t need to emit object files at all.

I personally think if there was a way to expose the linking from llvmlite, then one can write a full compiler using llvmlite only (e.g. conda install llvmlite), so that would be very nice. As it is now, one can generate object files, but the final step still requires either gcc or clang to be installed to do the linking.

[alendit]

@certik this is surprisingly tricky using the MCJIT, since the API isn’t exposed. This become straight forward with ORC, which is the new JIT for LLVM>=6. #245 ist relevant here.

[gmarkall]

As per https://github.com/numba/llvmlite/pull/419#issuecomment-998057666 - if anyone would like to take over the completion of #419 (@overdev / @appcypher / @matthieugouel / @alendit perhaps?) - please do let me know (further directions in the linked comment).