java.lang.IllegalArgumentException: clientSecret is null or empty on Azure Container Instance

java.lang.IllegalArgumentException: clientSecret is null or empty

Possibly related to https://github.com/microsoft/mssql-jdbc/issues/1512

The issue below happens only on Azure Container Instances, nowhere else.

Driver version

9.2.0.jre11

SQL Server version

current Azure SQL Database

Client Operating System

Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f) Default locale: en, platform encoding: UTF-8 OS name: “linux”, version: “4.15.0-1106-azure”, arch: “amd64”, family: “unix”

JAVA/JVM version

Java version: 11.0.10, vendor: Oracle Corporation, runtime: /usr/local/openjdk-11

Table schema

not relevant

Problem description

I am using this JDBC driver in combination with msal4j to connect my liquibase to a an Azure SQL Database using a service principal. This is executed in a Docker container on Azure Container Instance.

I have no issues running this locally on my machine. I have no issues running this in the Docker container on my machine either.

However, the exact same Docker container, gives me the mentioned error when I run it in an Azure Container Instance.

What I checked:

• the code works on my machine (macOS 11.2)
• the code works in the Docker container on my machine (FROM maven:3-openjdk-11)
• the issue occurs only on Azure Container Instance, both with and without vnet integration
• the environment variable with my JDBC string contains the same value on ACI and locally, the client secret is always there
• the client ID and client secret, as well as any other value in the connection string, do not contain any special characters
• the order of the connection string components does not make any difference
• the connection is not blocked by any firewalls
• you can see in the trace logs that the secret is correctly parsed from the connection string

JDBC trace logs

log.xml.log

Stacktrace

ful logs.txt

Reproduction code

Using liquibase with a connection string in this form: jdbc:sqlserver://dl-REDACTED-sqlserver.database.windows.net:1433;database=REDACTED;encrypt=true;trustServerCertificate=false;hostNameInCertificate=*.database.windows.net;loginTimeout=300;authentication=ActiveDirectoryServicePrincipal;aadSecurePrincipalId=51fcffe3-REDACTED;aadSecurePrincipalSecret=REDACTED

POM file

<project  xmlns="http://maven.apache.org/POM/4.0.0"  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>eu.lynxcare</groupId>
    <artifactId>REDACTED</artifactId>
    <version>1.0-SNAPSHOT</version>
    <build>
        <pluginManagement>
            <plugins>
                <plugin>
                    <groupId>org.liquibase</groupId>
                    <artifactId>liquibase-maven-plugin</artifactId>
                    <version>4.3.0</version>
                    <configuration>
                        <promptOnNonLocalDatabase>false</promptOnNonLocalDatabase>
                        <driver>com.microsoft.sqlserver.jdbc.SQLServerDriver</driver>
                        <logging>${env.LOGLEVEL}}</logging>
                        <url>${env.JDBC_STRING}</url>
                        <changeLogDirectory>changelogs</changeLogDirectory>
                        <changeLogFile>main.changelog.xml</changeLogFile>
                    </configuration>
                    <dependencies>
                        <dependency>
                            <groupId>com.microsoft.azure</groupId>
                            <artifactId>msal4j</artifactId>
                            <version>1.9.0</version>
                        </dependency>
                        <dependency>
                            <groupId>com.microsoft.sqlserver</groupId>
                            <artifactId>mssql-jdbc</artifactId>
                            <version>9.2.0.jre11</version>
                        </dependency>
                    </dependencies>
                </plugin>
            </plugins>
        </pluginManagement>
    </build>
</project>

Environment variables

KUBERNETES_SERVICE_PORT_HTTPS=443
KUBERNETES_SERVICE_PORT=443
MAVEN_CONFIG=/root/.m2
HOSTNAME=wk-caas-b8ed290fc52d4df8bd6e5047b2c06027-7032143a535651dc8335ce
JAVA_HOME=/usr/local/openjdk-11
reportBuilderPassword=REDACTED
AZCOPY_CONCURRENCY_VALUE=AUTO
PWD=/liquibase
HOME=/root
LANG=C.UTF-8
KUBERNETES_PORT_443_TCP=tcp://10.0.0.1:443
dqSchemaName=REDACTED
omopExtensions20201126=REDACTED
adGroupReader=REDACTED
cdmSchemaName=REDACTED
TERM=xterm
MAVEN_HOME=/usr/share/maven
AZCOPY_CRED_TYPE=Anonymous
SHLVL=1
storageAccountKey=REDACTED
KUBERNETES_PORT_443_TCP_PROTO=tcp
JDBC_STRING=jdbc:sqlserver://dl-REDACTED-sqlserver.database.windows.net:1433;database=REDACTED;encrypt=true;trustServerCertificate=false;hostNameInCertificate=*.database.windows.net;loginTimeout=300;aadSecurePrincipalId=51fcffe3-REDACTED;aadSecurePrincipalSecret=REDACTED;authentication=ActiveDirectoryServicePrincipal
KUBERNETES_PORT_443_TCP_ADDR=10.0.0.1
KUBERNETES_SERVICE_HOST=10.0.0.1
KUBERNETES_PORT=tcp://10.0.0.1:443
KUBERNETES_PORT_443_TCP_PORT=443
masterKey=REDACTED
PATH=/usr/local/openjdk-11/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
vocabulariesLocation=REDACTED
customerId=1
TARGET_VERSION=v23
JAVA_VERSION=11.0.10
LOGLEVEL=INFO
cdmv5SchemaName=REDACTED

Docker

FROM maven:3-openjdk-11

# curl
RUN apt -qq update \
 && apt -y --no-install-recommends install curl \
 && apt clean \
 && rm -rf /var/lib/apt/lists/*

# sqlcmd
RUN curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add - \
 && curl https://packages.microsoft.com/config/ubuntu/18.04/prod.list | tee /etc/apt/sources.list.d/msprod.list \
 && apt -qq update \
 && ACCEPT_EULA=Y apt -y --no-install-recommends install mssql-tools unixodbc-dev \
 && apt clean \
 && rm -rf /var/lib/apt/lists/* \
 && ln -s /opt/mssql-tools/bin/sqlcmd /usr/local/bin/sqlcmd

# AzCopy
RUN curl -sL -o /tmp/azcopy.tar.gz https://aka.ms/downloadazcopy-v10-linux \
 && tar -zxf /tmp/azcopy.tar.gz  -C /usr/local/bin --wildcards --no-anchored --strip-components=1 'azcopy' \
 && chmod +x /usr/local/bin/azcopy \
 && rm -rf /tmp/*
ENV AZCOPY_CONCURRENCY_VALUE=AUTO
ENV AZCOPY_CRED_TYPE=Anonymous

# project
RUN mkdir -p /liquibase
COPY . /liquibase

# Application Insights
ARG APPLICATION_INSIGHTS_RELEASE="3.0.2"
RUN curl -o /liquibase/ai.jar -sL "https://github.com/microsoft/ApplicationInsights-Java/releases/download/${APPLICATION_INSIGHTS_RELEASE}/applicationinsights-agent-${APPLICATION_INSIGHTS_RELEASE}.jar"

# default settings
ENV LOGLEVEL=INFO
ENV storageAccountKey="REDACTED"
ENV vocabulariesLocation="REDACTED"
ENV omopExtensions20201126="REDACTED"
ENV cdmSchemaName=REDACTED
ENV cdmv5SchemaName=REDACTED
ENV dqSchemaName=REDACTED

# update me when you add a new changelog
ENV TARGET_VERSION=v23

WORKDIR /liquibase
CMD /liquibase/scripts/entrypoint.sh

[lilgreenbird]

hi @sdebruyn,

I am testing this fix, can you please give these jars a try to see if this resolve the issue in your environment?

[lilgreenbird]

hi @sdebruyn

I can repro the issue can confirm it is a bug