JsonReader.nextString() producer/consumer alternative

Hi,

JsonReader is a great class to read JSON data in streaming fashion. However, there are some cases when tokens are too big to fit in JVM memory making the JSON reader streaming idea neutralized. For example, here is a question at StackOverflow asking for managing huge fields somehow: http://stackoverflow.com/questions/39615673/best-way-to-handle-huge-fields-with-gson-jsonreader – String JsonReader.nextString() reads the whole huge literal input into a StringBuilder instance that makes the JVM crash on OutOfMemoryError.

Do you consider adding a streaming alternative, say void JsonReader.nextString(StringChunksConsumer) where the StringChunksConsumer interface is expected to be something like the interface below, a valuable idea?

interface StringChunksConsumer {

    void accept(char[] buffer, int offset, int length)
            throws IOException;

}

The idea of this method would be passing parsed string literal chunks to the consumer rather than aggregating them into a string builder. The StringChunksConsumer.accept method signature would perfectly match:

• StringBuilder.append(char[], int, int)
• Writer.write(char[], int, int)
• etc.

For example, a simple redirecting from a reader to a writer could be just read.nextString(writer::write); consuming no more memory than JsonReader.nextString() uses internally.

Thanks!

Update 1

A similar problem: https://stackoverflow.com/questions/43470927/android-java-lang-outofmemoryerror-failed-to-allocate-with-free-bytes-and-70m

Update 2

This would also allow to work with inner string-encoded (for whatever reason) JSON documents more efficiently. Example: http://stackoverflow.com/questions/43929916/gson-parsing-error-for-my-json-string

Update 3

This possibly duplicates #415 and #718

[swankjesse]

Interesting idea. I think the most likely thing we’d build would be a method like nextReader() that returned a Reader on the characters of a string.

[Marcono1234]

@lyubomyr-shaydariv, even though Gson is now in maintenance mode, would you mind if this issue remains open? Maybe this could still be implemented eventually if the project members find this useful.