question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Netty 4.1.19 http encode header cost too many time reach 10ms

See original GitHub issue

Expected behavior

I use netty send http request ,find cost too much time ,I add log for the follower,found too many time is cost encode header part

Actual behavior

[28 18:33:59,778 INFO ] [ioNioEventLoopGroup-Southgate-1-15] http.HttpObjectEncoder - Finish encode request initial line … [28 18:33:59,789 INFO ] [ioNioEventLoopGroup-Southgate-1-15] http.HttpObjectEncoder - Finish encode request header …

cost time 789-778 = 11ms

Steps to reproduce

Minimal yet complete reproducer code (or URL to code)

public static void main(String[] args){

    HttpVersion httpVersion = HttpVersion.HTTP_1_0;
    HttpMethod method = HttpMethod.GET;
    ByteBuf buf = ByteBufAllocator.DEFAULT.buffer();
    ByteBuf content = Unpooled.EMPTY_BUFFER;
   // new DefaultFullHttpRequest(httpVersion,method, urlAddress,content);
    DefaultFullHttpRequest nettyHttpRequest = new DefaultFullHttpRequest(httpVersion,method,"http://localhost/aaa/bbb",content);
    //头里加入host信息
    nettyHttpRequest.headers().add(HttpHeaderNames.HOST, new AsciiString("192.168.3.52:8081"));
    nettyHttpRequest.headers().add(HttpHeaderNames.USER_AGENT, new AsciiString("Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:53.0) Gecko/20100101 Firefox/53.0"));
    nettyHttpRequest.headers().add(HttpHeaderNames.CONTENT_TYPE, new AsciiString("text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"));
    nettyHttpRequest.headers().add(HttpHeaderNames.ACCEPT_LANGUAGE, new AsciiString("zh-CN,zh;q=0.8,en-US;q=0.5,en;q=0.3"));
    nettyHttpRequest.headers().add(HttpHeaderNames.ACCEPT_ENCODING, new AsciiString("gzip, deflate"));
    nettyHttpRequest.headers().add(HttpHeaderNames.CONTENT_LENGTH, HttpHeaderValues.ZERO);
    nettyHttpRequest.headers().add(HttpHeaderNames.CONNECTION, HttpHeaderValues.KEEP_ALIVE);
    try {
        long st = System.currentTimeMillis();
        encodeHeaders(nettyHttpRequest.headers(), buf);
        long et = System.currentTimeMillis();
        System.out.println("cost time :"+(et-st));
    }catch (Exception e){

    }

}


protected static void encodeHeaders(HttpHeaders headers, ByteBuf buf) throws Exception {
    Iterator<Map.Entry<CharSequence, CharSequence>> iter = headers.iteratorCharSequence();
    while (iter.hasNext()) {
        Map.Entry<CharSequence, CharSequence> header = iter.next();
        long st = System.currentTimeMillis();
        encoderHeader(header.getKey(), header.getValue(), buf);
        long et = System.currentTimeMillis();
        System.out.println("encode header "+header.getKey()+" value "+header.getValue()+" cost time :"+(et-st));
    }
}


public static void encoderHeader(CharSequence name, CharSequence value, ByteBuf buf) throws Exception {
    final int nameLen = name.length();
    final int valueLen = value.length();
    final int entryLen = nameLen + valueLen + 4;
    buf.ensureWritable(entryLen);
    int offset = buf.writerIndex();
    writeAscii(buf, offset, name, nameLen);
    offset += nameLen;
    buf.setByte(offset ++, ':');
    buf.setByte(offset ++, ' ');
    writeAscii(buf, offset, value, valueLen);
    offset += valueLen;
    buf.setByte(offset ++, '\r');
    buf.setByte(offset ++, '\n');
    buf.writerIndex(offset);
}

private static void writeAscii(ByteBuf buf, int offset, CharSequence value, int valueLen) {
    if (value instanceof AsciiString) {
        long st = System.currentTimeMillis();
        ByteBufUtil.copy((AsciiString) value, 0, buf, offset, valueLen);
        long et = System.currentTimeMillis();
        //System.out.println("writeAscii header "+value+" value  cost time :"+(et-st));

    } else {

        writeCharSequence(buf, offset, value, valueLen);

    }
}

private static void writeCharSequence(ByteBuf buf, int offset, CharSequence value, int valueLen) {
    for (int i = 0; i < valueLen; ++i) {
        buf.setByte(offset ++, c2b(value.charAt(i)));
    }
}

Netty version

4.1.9

JVM version (e.g. java -version)

OS version (e.g. uname -a)

Issue Analytics

  • State:closed
  • Created 6 years ago
  • Comments:6 (6 by maintainers)

github_iconTop GitHub Comments

1reaction
slandellecommented, Jun 28, 2017

Several comments:

  • Never use System. currentTimeMillis() for measuring short durations: it’s a wall clock that can drift and can be updated any time by NTP. With a fixed version of your sample using System. nanoTime(), I have a cost of 3ms on my machine.
  • If you want to build microbenchmarks, use JMH
0reactions
normanmaurercommented, Jun 29, 2017

Please reopen once you have a proper jmh benchmark …

Read more comments on GitHub >

github_iconTop Results From Across the Web

http request smuggling, cause by obfuscating TE header #9571
Hi, it think the "Transfer-Encoding : chunked" header name with trailing whitespace is incorrect, but I'm not sure, maybe @slandelle can help.
Read more >
Package io.netty.handler.codec.http
Compresses an HttpMessage and an HttpContent in gzip or deflate encoding while respecting the "Accept-Encoding" header. Decodes the content of the received ...
Read more >
Comparing 38524ec3e2...37c03cce5e - netty5 - iGNUranza Git
beginEncode()` method has too many if else, so consider refactoring Modification: ... Motivation: netty needs to support zstd content-encoding http content ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found