Unable to set a compression input/output decorator to a `SmileFactory`

I have a special need for the riak-java-client which only allows me to use an ObjectMapper to serialize/deserialize key-values, I would like to decorate a SmileFactory with compressors like LZ4, Snappy or GZip but at the moment this is not possible, when I try a mapper like the following:

public static final Charset UTF8=Charset.forName("UTF-8");

public static final ObjectMapper GZIP_JSON_MAPPER=new ObjectMapper(new SmileFactory().disable(ENCODE_BINARY_AS_7BIT)
   .setInputDecorator(new InputDecorator()
   {
     @Override
     public InputStream decorate(IOContext context,InputStream inputStream) throws IOException
     {
       return new GZIPInputStream(inputStream);
     }

     @Override
     public InputStream decorate(IOContext context,byte[] bytes,int offset,int length) throws IOException
     {
       return new GZIPInputStream(new ByteArrayInputStream(bytes,offset,length));
     }

     @Override
     public Reader decorate(IOContext context,Reader reader) throws IOException
     {
       return new InputStreamReader(new GZIPInputStream(new ReaderInputStream(reader)),UTF8);
     }
   })
   .setOutputDecorator(new OutputDecorator()
   {
     @Override
     public OutputStream decorate(IOContext context,OutputStream outputStream) throws IOException
     {
       return new GZIPOutputStream(outputStream);
     }

     @Override
     public Writer decorate(IOContext context,Writer writer) throws IOException
     {
       return new OutputStreamWriter(new GZIPOutputStream(new WriterOutputStream(writer,UTF8)));
     }
   }))
   .disable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES)
   .disable(SerializationFeature.FAIL_ON_EMPTY_BEANS)
   .setSerializationInclusion(JsonInclude.Include.NON_NULL)
   .disable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS);

This is the exception I get:

Exception in thread "main" java.util.zip.ZipException: Not in GZIP format
	at java.util.zip.GZIPInputStream.readHeader(GZIPInputStream.java:165)
	at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:79)
	at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:91)
	at ...JsonUtils$4.decorate(JsonUtils.java:162)
	at com.fasterxml.jackson.core.JsonFactory._decorate(JsonFactory.java:1459)
	at com.fasterxml.jackson.dataformat.smile.SmileFactory.createParser(SmileFactory.java:330)
	at com.fasterxml.jackson.dataformat.smile.SmileFactory.createParser(SmileFactory.java:320)
	at com.fasterxml.jackson.dataformat.smile.SmileFactory.createParser(SmileFactory.java:29)
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3091)

I used Gzip as an example, in reality I’m using both LZ4 and Gzip and both throw exceptions when I try with a SmileFactory, this works perfectly with a JsonFactory, the reason for me to prefer a SmileFactory over a JsonFactory is because it is notice-able faster than the JsonFactory so basically it’ll help compensate the price I pay for compression.

[guidomedina]

I’m using LZ4 double compression with 32KB block size, also because it is binary and Smile is faster than standard Json when I use some binary compression I’m using Smile straight forward, I’m using:

• org.lz4:lz4-java:1.7.1
• com.fasterxml.jackson.dataformat:jackson-dataformat-smile 2.11.1 (or 2.11.2 released yesterday)

Here is the code for LZ4 Json mapper and LZ4 Smile mapper:

import static com.fasterxml.jackson.dataformat.smile.SmileGenerator.Feature.ENCODE_BINARY_AS_7BIT;
import static java.nio.charset.StandardCharsets.UTF_8;
// there are more imports but I'm sure you will be able to figure them out

...
...

  public static final ParameterNamesModule PARAMETER_NAMES_MODULE = new ParameterNamesModule();
  public static final JavaTimeModule JAVA_TIME_MODULE = new JavaTimeModule();
  public static final Jdk8Module JDK_8_MODULE = new Jdk8Module();

  // Just to configure some modules
  public static ObjectMapper configureDefaultObjectMapper(ObjectMapper objectMapper)
  {
    return objectMapper
      .disable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES)
      .disable(SerializationFeature.FAIL_ON_EMPTY_BEANS)
      .setSerializationInclusion(JsonInclude.Include.NON_NULL)
      .disable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS)
      .disable(DeserializationFeature.ADJUST_DATES_TO_CONTEXT_TIME_ZONE)
      .registerModule(PARAMETER_NAMES_MODULE)
      .registerModule(JAVA_TIME_MODULE)
      .registerModule(JDK_8_MODULE);
  }

  public static final int LZ4_BLOCK_SIZE = 32 * 1024;

  public static final ObjectMapper LZ4_JSON_MAPPER = configureDefaultObjectMapperForRiak(JsonMapper.builder(new JsonFactoryBuilder()
    .inputDecorator(new InputDecorator()
    {
      @Override
      public InputStream decorate(IOContext context, InputStream inputStream)
      {
        return new LZ4BlockInputStream(new LZ4BlockInputStream(inputStream));
      }

      @Override
      public InputStream decorate(IOContext context, byte[] bytes, int offset, int length)
      {
        return new LZ4BlockInputStream(new LZ4BlockInputStream(new ByteArrayInputStream(bytes, offset, length)));
      }

      @Override
      public Reader decorate(IOContext context, Reader reader)
      {
        return new InputStreamReader(new LZ4BlockInputStream(new LZ4BlockInputStream(new ReaderInputStream(reader, UTF_8))), UTF_8);
      }
    })
    .outputDecorator(new OutputDecorator()
    {
      @Override
      public OutputStream decorate(IOContext context, OutputStream outputStream)
      {
        return new LZ4BlockOutputStream(new LZ4BlockOutputStream(outputStream,
          LZ4_BLOCK_SIZE, LZ4Factory.fastestInstance().fastCompressor()),
          LZ4_BLOCK_SIZE, LZ4Factory.fastestInstance().fastCompressor());
      }

      @Override
      public Writer decorate(IOContext context, Writer writer)
      {
        return new OutputStreamWriter(new LZ4BlockOutputStream(new LZ4BlockOutputStream(new WriterOutputStream(writer, UTF_8),
          LZ4_BLOCK_SIZE, LZ4Factory.fastestInstance().fastCompressor()),
          LZ4_BLOCK_SIZE, LZ4Factory.fastestInstance().fastCompressor())
        );
      }
    }).build()).build());

  public static final ObjectMapper LZ4_SMILE_MAPPER = configureDefaultObjectMapper(JsonMapper.builder(SmileFactory.builder()
    .disable(ENCODE_BINARY_AS_7BIT)
    .inputDecorator(new InputDecorator()
    {
      @Override
      public InputStream decorate(IOContext context, InputStream inputStream)
      {
        return new LZ4BlockInputStream(new LZ4BlockInputStream(inputStream));
      }

      @Override
      public InputStream decorate(IOContext context, byte[] bytes, int offset, int length)
      {
        return new LZ4BlockInputStream(new LZ4BlockInputStream(new ByteArrayInputStream(bytes, offset, length)));
      }

      @Override
      public Reader decorate(IOContext context, Reader reader)
      {
        return new InputStreamReader(new LZ4BlockInputStream(new LZ4BlockInputStream(new ReaderInputStream(reader, UTF_8))), UTF_8);
      }
    })
    .outputDecorator(new OutputDecorator()
    {
      @Override
      public OutputStream decorate(IOContext context, OutputStream outputStream)
      {
        return new LZ4BlockOutputStream(new LZ4BlockOutputStream(outputStream,
          LZ4_BLOCK_SIZE, LZ4Factory.fastestInstance().fastCompressor()),
          LZ4_BLOCK_SIZE, LZ4Factory.fastestInstance().fastCompressor());
      }

      @Override
      public Writer decorate(IOContext context, Writer writer)
      {
        return new OutputStreamWriter(new LZ4BlockOutputStream(new LZ4BlockOutputStream(new WriterOutputStream(writer, UTF_8),
          LZ4_BLOCK_SIZE, LZ4Factory.fastestInstance().fastCompressor()),
          LZ4_BLOCK_SIZE, LZ4Factory.fastestInstance().fastCompressor())
        );
      }
    }).build()).build());

[howardem]

Thank so much for posting the code and for your detailed explanation. I’m gonna your configurations and see how it goes!!