ByteBuf.order() seems to have wrong behavior

Found unexpected behavior when calling

Unpooled.buffer(1024).order(ByteOrder.LITTLE_ENDIAN).writeShortLE(message)

since netty 4.1.54.Final.

The order() method has already been deprecated, but nevertheless it shouldn’t be broken.

Expected behavior

Test passes

Actual behavior

Test failed

expected: -12208
but was : 20688
	at NettyEndianTest.endianTestCallingOrder(NettyEndianTest.java:26)

Steps to reproduce

Create the following project

./build.gradle

plugins {
    id 'java'
}

group 'org.example'
version '1.0-SNAPSHOT'

repositories {
    mavenCentral()
}

dependencies {
    testImplementation 'com.google.truth:truth:1.0.1'
    testImplementation 'io.netty:netty-buffer:4.1.68.Final'
    testImplementation 'org.junit.jupiter:junit-jupiter-api:5.7.0'
    testRuntimeOnly 'org.junit.jupiter:junit-jupiter-engine:5.7.0'
}

test {
    useJUnitPlatform()
}

./src/NettyEndianTest.java

import static com.google.common.truth.Truth.assertThat;

import io.netty.buffer.ByteBuf;
import io.netty.buffer.Unpooled;
import java.nio.ByteOrder;
import org.junit.jupiter.api.Test;

class NettyEndianTest {

  // Test will fail for netty 4.1.54.Final and above 
  @SuppressWarnings("deprecation")
  @Test
  void endianTestCallingOrder() {
    short message = (short) 0xD050;
    short reverse = (short) 0x50D0;
    ByteBuf b = Unpooled.buffer(1024).order(ByteOrder.LITTLE_ENDIAN).writeShortLE(message);

    assertThat(b.getShortLE(0)).isEqualTo(message);
    assertThat(b.getShort(0)).isEqualTo(reverse);
  }

  // Test is passing
  @Test
  void endianTestWithoutCallingOrder() {
    short message = (short) 0xD050;
    short reverse = (short) 0x50D0;
    ByteBuf b = Unpooled.buffer(1024).writeShortLE(message);

    assertThat(b.getShortLE(0)).isEqualTo(message);
    assertThat(b.getShort(0)).isEqualTo(reverse);
  }
}

Run gradle test

Minimal yet complete reproducer code (or URL to code)

Netty version

4.1.54.Final - 4.1.68.Final

Earlier versions before 4.1.54 were good.

JVM version (e.g. java -version)

openjdk version “11.0.11” 2021-04-20 LTS OpenJDK Runtime Environment Zulu11.48+21-CA (build 11.0.11+9-LTS) OpenJDK 64-Bit Server VM Zulu11.48+21-CA (build 11.0.11+9-LTS, mixed mode)

Other JVMs the same.

OS version (e.g. uname -a)

Darwin Kernel Version 20.6.0 Other platforms the same.

[chrisvest]

Opened #11706 to fix this.

@dapengzhang0 Note the tests I added to UnpooledTest are different from the ones you posted above. Specifically, the test you said was failing is now making the opposite assertion, namely that LE methods always perform little-endian accesses.

[ejona86]

Looks like the entirety of SwappedByteBuf needs an audit. But it looks like it has always been broken https://github.com/netty/netty/commit/0f9492c9affc528c766f9677952412564d4a3f6d . It seems what changed is https://github.com/netty/netty/commit/96da45de2d53d74ad3a77e83c840410eb3f847c7 fixed many of the methods, but also missed many of them. Before the methods were wrong, but in agreement. Now some methods are just a bit more obviously broken.

https://github.com/netty/netty/blob/23405e2000427e9cad104913e7071d8d08e91a3c/buffer/src/main/java/io/netty/buffer/SwappedByteBuf.java#L247-L250

And getMediumLE, getIntLE, and getLongLE look busted. And then I see some other busted methods like:

https://github.com/netty/netty/blob/23405e2000427e9cad104913e7071d8d08e91a3c/buffer/src/main/java/io/netty/buffer/SwappedByteBuf.java#L402-L406