RFE: Add a public ByteStreams.toByteArray(InputStream in, int expectedSize)

1. rename the current package-private ByteStreams.toByteArray(InputStream in, int expectedSize)

2. add public ByteStreams.toByteArray(InputStream in, long expectedSize) that does exactly this: https://github.com/google/guava/blob/e102fa4a86dad34f375e06c8ce0f52ea264588c3/guava/src/com/google/common/io/Files.java#L159-L175

3. except that this line: https://github.com/google/guava/blob/e102fa4a86dad34f375e06c8ce0f52ea264588c3/guava/src/com/google/common/io/Files.java#L172 should be: return expectedSize <= 0

4. remove package-private Files.readFile(InputStream in, long expectedSize)

rationale for 2):

it is impossible to efficiently read an input stream of known size to a byte array, but sometimes it is needed. example a): efficiently reading a binary file after checking that its header is valid. only options are: reading the header, closing the file, open it again and read it fully with Files.readFile(...). or: reading the header then passing the stream to inefficient ByteStreams.toByteArray(InputStream in). example b): reading an InputStream of a zip file’s ZipEntry, where again the uncompress length is known ahead of time.

both examples a) and b) are from real code i am wirting.

rationale for 3):

ZipFile entry returns -1 rather than 0 for unknown lengths: https://docs.oracle.com/javase/8/docs/api/index.html?java/util/zip/ZipEntry.html Given that 0 is a valid length, that makes total sense. so it also makes sense that expectedSize should handle <0 values (and 0 too) as unknown lengths.

(rationales for 1) and 4) are obvious.)

[Lanchon]

i also want to comment on this:

In most other cases, people either:

• …
• …
• know the exact length (in which case they can use ByteStreams.readFully)

ByteStreams.readFully is not an option. it will throw if the file is shorter than expected, but it will silently fail (misbehave) if the file length was increased under its feet, potentially returning a byte array that is not representative of any file state.

what is needed is a performance hint to a robust helper, not some brittle alternative that you cannot justify choosing solely on the grounds of increased performance. i will keep rolling my code for this as guava does not provide useful help for this case.

the fact that you actually need this yourself and do not share it is frustrating.

[cpovirk]

I don’t know if this will make you feel better or worse, but we dedicated a couple engineer hours to discussing this over the past couple days, and we’ll probably spend a couple more on it in the next week or so. I can’t speak for everyone, by my main reservation is that ZipFile seems to be the main use case: In most other cases, people either:

• won’t have opened the stream yet (in which case they can implement ByteSource and get this behavior for free once we make the appropriate optimization there)
• won’t care about allocations, or will care about allocations enough to read the file in chunks
• know the exact length (in which case they can use ByteStreams.readFully)