DOC: better document the spawn interface, compare and contrast it to Jax's "split"

Jax, a new deep learning machine learning library has copied numpy’s excellent interface. They added one innovation to random number generation: splitting.

This is useful when parallel processes that should not be serialized need access to random numbers from a reproducible stream.

I suggest adding a method to BitGenerator:

b: BitGenerator
b.split(n)  # returns a list of n BitGenerators, each differently initialized from b in a unique, reproducible way

and a similar method on Generator that splits the underlying bit-generator and returns

[type(self)(big_generator[i]) for range(i)]

[rkern]

While the current API forces you to work with SeedSequences directly to access its spawn() method, we did make sure that it would be possible to expose spawn() methods on BitGenerator and Generator, which will be more convenient. We wanted to wait until we got a little more experience with the concept before exposing it so prominently.

Just to provide some mathematical background, Jax’s PRNG is in the same weak-crypto family as our Philox BitGenerator. The method by which it splits only works well for that family of weak-crypto PRNGs because that family keeps its initial seed around as the key value and only evolves a counter as one draws numbers from it. The other PRNGs iterate the state.

SeedSequence implements a similar scheme, but separates that scheme out from the PRNG implementation. By keeping the original seed state around as SeedSequence and by implementing good integer hashing techniques, we can get the same benefits without needing the full weak-crypto functionality of the PRNG. This may be of particular interest to the Jax developers since the ThreeFry algorithm that they use is not the fastest, nor necessarily the most GPU-friendly (once the massive-parallelization playing field is leveled by SeedSequence). They might be able to use a faster PRNG like SFC64 (our default of PCG64 may not be appropriate on a GPU because it requires 128-bit multiplication).

[peteroupc]

Note that Philox, and the Threefry PRNG, and SFC are merely examples of a general construction called the “counter-based PRNG” (as used in the “Random123” paper). In general, counter-based PRNGs use an underlying hash function or block cipher to hash a seed and an incrementing counter. And any other hash function can substitute for Threefry (or whatever underlying function the counter-based PRNG uses) as long as the resulting PRNG provides adequate randomness.

Also, splittable PRNGs are far from being an “innovation” of JAX (see also the JAX PRNG design notes); they have existed, for example, in Haskell and Java for years. Some of the known constructions for splittable PRNGs are surveyed in “Evaluation of Splittable Pseudo-Random Generators” by H. G. Schaathun, 2015. Some of them are general enough to be used by any PRNG (including Mersenne Twister and PCG), but do not necessarily lead to high-quality splittable PRNGs. Another example of a splittable PRNG is found in JuliaLang/julia#34852. See also https://github.com/idontgetoutmuch/random/issues/7.