question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Improve performance of Unpooled.copiedBuffer from CharSequence and char[]

See original GitHub issue

Hi, there. As we known ByteBufUtil.wirteUtf8() is faster than ByteBufUtil.encodeString() when the charset is UTF-8. So, If we add some code in Unpooled.copiedBuffer(CharSequence string, Charset charset) method to predicate the charset is UTF-8 or not. If true call ByteBufUtil.writeUtf8 method . This will improve the perf evident

Affected methods:

ByteBuf copiedBuffer(CharSequence string, Charset charset) ByteBuf copiedBuffer(CharSequence string, int offset, int length, Charset charset) ByteBuf copiedBuffer(char[] array, Charset charset) ByteBuf copiedBuffer(char[] array, int offset, int length, Charset charset)

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:6 (3 by maintainers)

github_iconTop GitHub Comments

1reaction
timandycommented, Apr 23, 2020
@SuppressWarnings("unused")
@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.NANOSECONDS)
@Fork(1)
@State(Scope.Benchmark)
@Warmup(iterations = 2, time = 5)
@Measurement(iterations = 2, time = 5)
public class UnpooledPerfTest {
    private static final String SOURCE = "hello, world; hello, world; hello, world; hello, world;" +
            " hello, world; hello, world; hello, world; hello, world;";
    private static final UnpooledByteBufAllocator ALLOC = UnpooledByteBufAllocator.DEFAULT;

    public static ByteBuf copiedBuffer(CharSequence string, Charset charset) {
        ObjectUtil.checkNotNull(string, "string");

        if (charset == CharsetUtil.UTF_8) {
            return ByteBufUtil.writeUtf8(ALLOC, string);
        }

        if (string instanceof CharBuffer) {
            return copiedBuffer((CharBuffer) string, charset);
        }

        return copiedBuffer(CharBuffer.wrap(string), charset);
    }

    private static ByteBuf copiedBuffer(CharBuffer buffer, Charset charset) {
        return ByteBufUtil.encodeString0(ALLOC, true, buffer, charset, 0);
    }

    @Benchmark
    public ByteBuf utf8Origin() {
        ByteBuf byteBuf = Unpooled.copiedBuffer(SOURCE, CharsetUtil.UTF_8);
        byteBuf.release();
        return byteBuf;
    }

    @Benchmark
    public ByteBuf utf8Perf() {
        ByteBuf byteBuf = copiedBuffer(SOURCE, CharsetUtil.UTF_8);
        byteBuf.release();
        return byteBuf;
    }

    @Benchmark
    public ByteBuf noUtf8Origin() {
        ByteBuf byteBuf = Unpooled.copiedBuffer(SOURCE, CharsetUtil.UTF_16);
        byteBuf.release();
        return byteBuf;
    }

    @Benchmark
    public ByteBuf noUtf8Perf() {
        ByteBuf byteBuf = copiedBuffer(SOURCE, CharsetUtil.UTF_16);
        byteBuf.release();
        return byteBuf;
    }

    @Test
    void test() throws RunnerException {
        Options opt = new OptionsBuilder()
                .include(this.getClass().getSimpleName())
                .build();

        new Runner(opt).run();
    }
}

# JMH version: 1.23
# VM version: JDK 1.8.0_231, Java HotSpot(TM) 64-Bit Server VM, 25.231-b11
# VM invoker: C:\Program Files\Java\jdk1.8.0_231\jre\bin\java.exe
# VM options: -ea -Didea.test.cyclic.buffer.size=1048576 -javaagent:C:\Program Files\JetBrains\IntelliJ IDEA 2020.1\lib\idea_rt.jar=49611:C:\Program Files\JetBrains\IntelliJ IDEA 2020.1\bin -Dfile.encoding=UTF-8
# Warmup: 2 iterations, 5 s each
# Measurement: 2 iterations, 5 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: io.netty.buffer.UnpooledPerfTest.noUtf8Origin

# Run progress: 0.00% complete, ETA 00:01:20
# Fork: 1 of 1
# Warmup Iteration   1: 755.285 ns/op
# Warmup Iteration   2: 702.164 ns/op
Iteration   1: 703.226 ns/op
Iteration   2: 707.050 ns/op


Result "io.netty.buffer.UnpooledPerfTest.noUtf8Origin":
  705.138 ns/op


# JMH version: 1.23
# VM version: JDK 1.8.0_231, Java HotSpot(TM) 64-Bit Server VM, 25.231-b11
# VM invoker: C:\Program Files\Java\jdk1.8.0_231\jre\bin\java.exe
# VM options: -ea -Didea.test.cyclic.buffer.size=1048576 -javaagent:C:\Program Files\JetBrains\IntelliJ IDEA 2020.1\lib\idea_rt.jar=49611:C:\Program Files\JetBrains\IntelliJ IDEA 2020.1\bin -Dfile.encoding=UTF-8
# Warmup: 2 iterations, 5 s each
# Measurement: 2 iterations, 5 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: io.netty.buffer.UnpooledPerfTest.noUtf8Perf

# Run progress: 25.00% complete, ETA 00:01:04
# Fork: 1 of 1
# Warmup Iteration   1: 750.255 ns/op
# Warmup Iteration   2: 711.880 ns/op
Iteration   1: 714.150 ns/op
Iteration   2: 717.284 ns/op


Result "io.netty.buffer.UnpooledPerfTest.noUtf8Perf":
  715.717 ns/op


# JMH version: 1.23
# VM version: JDK 1.8.0_231, Java HotSpot(TM) 64-Bit Server VM, 25.231-b11
# VM invoker: C:\Program Files\Java\jdk1.8.0_231\jre\bin\java.exe
# VM options: -ea -Didea.test.cyclic.buffer.size=1048576 -javaagent:C:\Program Files\JetBrains\IntelliJ IDEA 2020.1\lib\idea_rt.jar=49611:C:\Program Files\JetBrains\IntelliJ IDEA 2020.1\bin -Dfile.encoding=UTF-8
# Warmup: 2 iterations, 5 s each
# Measurement: 2 iterations, 5 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: io.netty.buffer.UnpooledPerfTest.utf8Origin

# Run progress: 50.00% complete, ETA 00:00:42
# Fork: 1 of 1
# Warmup Iteration   1: 507.022 ns/op
# Warmup Iteration   2: 472.329 ns/op
Iteration   1: 474.891 ns/op
Iteration   2: 477.625 ns/op


Result "io.netty.buffer.UnpooledPerfTest.utf8Origin":
  476.258 ns/op


# JMH version: 1.23
# VM version: JDK 1.8.0_231, Java HotSpot(TM) 64-Bit Server VM, 25.231-b11
# VM invoker: C:\Program Files\Java\jdk1.8.0_231\jre\bin\java.exe
# VM options: -ea -Didea.test.cyclic.buffer.size=1048576 -javaagent:C:\Program Files\JetBrains\IntelliJ IDEA 2020.1\lib\idea_rt.jar=49611:C:\Program Files\JetBrains\IntelliJ IDEA 2020.1\bin -Dfile.encoding=UTF-8
# Warmup: 2 iterations, 5 s each
# Measurement: 2 iterations, 5 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: io.netty.buffer.UnpooledPerfTest.utf8Perf

# Run progress: 75.00% complete, ETA 00:00:21
# Fork: 1 of 1
# Warmup Iteration   1: 314.817 ns/op
# Warmup Iteration   2: 300.476 ns/op
Iteration   1: 301.387 ns/op
Iteration   2: 300.663 ns/op


Result "io.netty.buffer.UnpooledPerfTest.utf8Perf":
  301.025 ns/op


# Run complete. Total time: 00:01:25

REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
experiments, perform baseline and negative tests that provide experimental control, make sure
the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
Do not assume the numbers tell you what you want them to tell.

Benchmark                      Mode  Cnt    Score   Error  Units
UnpooledPerfTest.noUtf8Origin  avgt    2  705.138          ns/op
UnpooledPerfTest.noUtf8Perf    avgt    2  715.717          ns/op
UnpooledPerfTest.utf8Origin    avgt    2  476.258          ns/op
UnpooledPerfTest.utf8Perf      avgt    2  301.025          ns/op


Process finished with exit code 0

Read more comments on GitHub >

github_iconTop Results From Across the Web

Netty :Unpooled.copiedBuffer sending messages to client in ...
I am writing messages to client by using Unpooled.copiedBuffer(myMsg.getByte() , CharsetUtil_UTF_16) . The problem is for string messages ...
Read more >
Uses of Class io.netty.buffer.ByteBuf
static ByteBuf, Unpooled. copiedBuffer(CharSequence string, Charset charset). Creates a new big-endian buffer whose content is the specified string encoded in ...
Read more >
Index (Netty/All-in-One 4.1.23.Final API) - javadoc.io
addChar(CharSequence, char) - Method in class io.netty.handler.codec.http2. ... BIG_ENDIAN - Static variable in class io.netty.buffer.Unpooled.
Read more >
io.netty.handler.codec.http.HttpContent Java Examples
getBytes(); HttpContent chunk1 = new DefaultHttpContent(Unpooled.copiedBuffer(chunk1Bytes)); HttpContent lastChunk = new DefaultLastHttpContent(Unpooled.
Read more >
Diff - e9e4e90..f07b6b8 - chromium/src - Git at Google
-77,6 +77,17 @@ def ParseGnList(gn_string): + # TODO(brettw) bug 573132: This doesn't handle GN escaping properly, so any + # weird characters like...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found