(codebuild): CDK BuildSpec.FromObject Environment Secrets Not Working

What is the problem?

I am trying to use a private nuget package, which is stored in github and requires credentials to access For me to get this to work I need to create a NuGet.Config file and copy it to the docker container I would like to build the NuGet.Config from secrets stored in secrets manager

Below is how I generate the buildspec

BuildSpec.FromObject(new Dictionary<string, object> {
    { "version", "0.2" },

    { "env", new Dictionary<string, object> {
        { "secrets-manager", new Dictionary<string, object> {
            { "GITHUB_ACCESS_TOKEN", "my-secret-path/my-secret:Token" },
            { "GITHUB_USERNAME", "my-secret-path/my-secret:Username" },
            { "GITHUB_PACKAGE_URL", "my-secret-path/my-secret:Url" },
        }}
    }},

    { "phases", new Dictionary<string, object> {
        { "install", new Dictionary<string, object> {
            { "runtime-versions", new Dictionary<string, object> {
                { "dotnet", "latest" },
            }}
        }},
        { "pre_build", new Dictionary<string, object> {
            { "commands", new[] {
                "echo '<?xml version=\"1.0\" encoding=\"utf-8\"?><configuration><packageSources><add key=\"nuget.org\" value=\"https://api.nuget.org/v3/index.json\" protocolVersion=\"3\" /><add key=\"github\" value=\"$GITHUB_PACKAGE_URL\" /></packageSources><packageSourceCredentials><github><add key=\"Username\" value=\"$GITHUB_USERNAME\" /><add key=\"ClearTextPassword\" value=\"$GITHUB_ACCESS_TOKEN\" /></github></packageSourceCredentials></configuration>' | tee MyProject/NuGet.Config",
                "REPOSITORY_URI=MyAccountNumber.dkr.ecr.us-east-1.amazonaws.com/my-repo",
                "COMMIT_HASH=$(echo $CODEBUILD_RESOLVED_SOURCE_VERSION | cut -c 1-7)",
                "IMAGE_TAG=${COMMIT_HASH:=latest}",
            }}
        }},
        { "build", new Dictionary<string, object> {
            { "commands", new[] {
                "docker build -t $REPOSITORY_URI:latest -f MyProject/Dockerfile ."
            }}
        }},
        { "post_build", new Dictionary<string, object> {
            { "commands", new[] {
                "aws ecr get-login-password --region $AWS_DEFAULT_REGION | docker login --username AWS --password-stdin MyAccountNumber.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com",
                "docker tag $REPOSITORY_URI:latest $REPOSITORY_URI:$IMAGE_TAG",
                "docker push $REPOSITORY_URI:latest",
                "docker push $REPOSITORY_URI:$IMAGE_TAG",
            }}
        }}
    }},
});

Below is my dockerfile

FROM mcr.microsoft.com/dotnet/runtime:5.0 AS base
WORKDIR /app

FROM mcr.microsoft.com/dotnet/sdk:5.0 AS build
WORKDIR /src
COPY ["MyProject/MyProject.csproj", "MyProject/"]
COPY ["MyProject/NuGet.Config", "MyProject/"]
RUN dotnet restore "MyProject/MyProject.csproj"
COPY . .
WORKDIR "/src/MyProject"
RUN dotnet build "MyProject.csproj" -c Release -o /app/build

FROM build AS publish
RUN dotnet publish "MyProject.csproj" -c Release -o /app/publish

FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "MyProject.dll"]

When Codebuild tries to run this it fails with these errors:

Step 7/16 : RUN dotnet restore "MyProject/MyProject.csproj"
 ---> Running in 1264ea8ab195
  Determining projects to restore...
/usr/share/dotnet/sdk/5.0.403/NuGet.targets(131,5): error : The local source '/src/MyProject/$GITHUB_PACKAGE_URL' doesn't exist. [/src/MyProject/MyProject.csproj]
The command '/bin/sh -c dotnet restore "MyProject/MyProject.csproj"' returned a non-zero code: 1

[Container] 2021/11/22 17:41:36 Command did not exit successfully docker build -t $REPOSITORY_URI:latest -f MyProject/Dockerfile . exit status 1
[Container] 2021/11/22 17:41:36 Phase complete: BUILD State: FAILED
[Container] 2021/11/22 17:41:36 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: docker build -t $REPOSITORY_URI:latest -f MyProject/Dockerfile .. Reason: exit status 1

If I replace $GITHUB_PACKAGE_URL, $GITHUB_USERNAME and $GITHUB_ACCESS_TOKEN with hard coded values in this part of the buildspec, everything works

"echo '<?xml version=\"1.0\" encoding=\"utf-8\"?><configuration><packageSources><add key=\"nuget.org\" value=\"https://api.nuget.org/v3/index.json\" protocolVersion=\"3\" /><add key=\"github\" value=\"$GITHUB_PACKAGE_URL\" /></packageSources><packageSourceCredentials><github><add key=\"Username\" value=\"$GITHUB_USERNAME\" /><add key=\"ClearTextPassword\" value=\"$GITHUB_ACCESS_TOKEN\" /></github></packageSourceCredentials></configuration>' | tee MyProject/NuGet.Config",

Reproduction Steps

See above

What did you expect to happen?

Nuget.Config should contain the secrets from secrets manager

What actually happened?

I get a build error, see above

CDK CLI Version

1.132.0 (build 5c75891)

Framework Version

.NET 5.0

Node.js Version

v14.17.4

OS

Windows 10

Language

.NET

Language Version

.NET 5.0.303

Other information

No response

[markreyndev]

Thank you so much, you’re right, $-expansion does not work inside single quotes. I knew it would be a stupid mistake on my part. I have surrounded the $ variables with single quotes and it’s all now working

[github-actions[bot]]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.