dotnet Data Sources `Pulumi.Aws.Iam.Invokes` could not resolve Resource `Output<T>`.

Description

Create Iam Role and pass it’s Arn to Pulumi.Aws.Iam.Invokes.GetPolicyDocument throws exception.

Pulumi.Deployment+InvokeException: Invoke of ‘aws:iam/getPolicyDocument:getPolicyDocument’ failed: “statement.1.principals.0.identifiers”: required field is not set ()

It seems like Pulumi.Aws.Invokes.GetArn also throws exception for same kind of error.

Error message

$ pulumi up

Previewing update (dev):
     Type                 Name        Plan     Info
     pulumi:pulumi:Stack  pulumi-dev           'dotnet build -nologo .' completed successfully

     pulumi:pulumi:Stack  pulumi-dev  running..   
     pulumi:pulumi:Stack  pulumi-dev             1 error; 2 messages
     Type                 Name        Plan       Info
 +   └─ aws:iam:Role      role        create

Diagnostics:
  pulumi:pulumi:Stack (pulumi-dev):

    error: Running program 'C:\git\infra\pulumi\bin\Debug\netcoreapp3.0\Infra.dll' failed with an unhandled exception:
    Pulumi.Deployment+InvokeException: Invoke of 'aws:iam/getPolicyDocument:getPolicyDocument' failed: "statement.1.principals.0.identifiers": required field is not set ()
       at Pulumi.Deployment.InvokeAsync[T](String token, ResourceArgs args, InvokeOptions options, Boolean convertResult)
       at Program.<>c.<<Main>b__0_0>d.MoveNext() in C:\git\cysharp\infra\pulumi\Program.cs:line 36
    --- End of stack trace from previous location where exception was thrown ---
       at Pulumi.Stack.RunInitAsync(Func`1 init)
       at Pulumi.Output`1.GetValueAsync()
       at Pulumi.Deployment.RegisterResourceOutputsAsync(Resource resource, Output`1 outputs)
       at Pulumi.Deployment.Runner.WhileRunningAsync()

Minimum reproduce

using System.Collections.Generic;
using System.Threading.Tasks;
using Pulumi;
using Pulumi.Aws.Iam;
using Pulumi.Aws.Iam.Inputs;

class Program
{
    static Task<int> Main()
    {
        return Deployment.RunAsync(async () =>
        {
            var policy = await Pulumi.Aws.Iam.Invokes.GetPolicyDocument(new GetPolicyDocumentArgs
            {
                Statements = new[] {
                    new GetPolicyDocumentStatementsArgs
                    {
                        Actions = "sts:AssumeRole",
                        Effect = "Allow",
                        Principals = new GetPolicyDocumentStatementsPrincipalsArgs
                        {
                            Type = "Service",
                            Identifiers = "ec2.amazonaws.com",
                        }
                    },
                },
            });
            var role = new Pulumi.Aws.Iam.Role($"role", new RoleArgs
            {
                AssumeRolePolicy = policy.Json,
            });

            var assumepolicy = await Pulumi.Aws.Iam.Invokes.GetPolicyDocument(new GetPolicyDocumentArgs
            {
                Statements = new[] {
                    new GetPolicyDocumentStatementsArgs
                    {
                        Actions = "sts:AssumeRole",
                        Effect = "Allow",
                        Principals = new GetPolicyDocumentStatementsPrincipalsArgs
                        {
                            Type = "Service",
                            Identifiers = "ec2.amazonaws.com",
                        }
                    },
                    new GetPolicyDocumentStatementsArgs
                    {
                        Actions = "sts:AssumeRole",
                        Effect = "Allow",
                        Principals = new GetPolicyDocumentStatementsPrincipalsArgs
                        {
                            Type = "AWS",
                            // throws exception here!
                            Identifiers = role.Arn,
                        }
                    }
                }
            });

            return new Dictionary<string, object>
            {
                { "arn", role.Arn },
                { "assumepolicy", assumepolicy.Json },
            };
        });
    }
}

Workaround

Issue happens only when Role is not yet created before. You can avoid with following steps.

1. Create role first and run pulumi up and execute change. (comment out Data Sources var assumepolicy section.)
2. Add Data Source and pass Output<T> Role.Arn to InputList<T> Identifiers.
3. pulumi up successfully run and GetPolicyDocumentResult will resolved as expected.

Expected behavior

Data Source wait for Resource Output<T> and resolve dependency.

Actual behavior

Exception happen when Data Source Input<T> receive Resource’s Output<T> when Resource is not yet created.

Terraform actually can resolve this resource -> data dependency. This enable me to create IAM Document with Role Arn restriction without consider resource -> data dependency.

data "aws_iam_policy_document" "main" {
  statement {
    actions = ["sts:AssumeRole"]
    principals {
      type        = "Service"
      identifiers = "ec2.amazonaws.com"
    }
  }
}
resource "aws_iam_role" "main" {
  name               = var.name
  assume_role_policy = data.aws_iam_policy_document.main.json
}
data "aws_iam_policy_document" "eks_kube2iam_role_assumerole_policy" {
  statement {
    actions = ["sts:AssumeRole"]
    effect  = "Allow"
    principals {
      identifiers = ["ec2.amazonaws.com"]
      type        = "Service"
    }
  }
  statement {
    actions = ["sts:AssumeRole"]
    effect  = "Allow"
    principals {
      identifiers = [aws_iam_role.main.arn]
      type        = "AWS"
    }
  }
}

[CyrusNajmabadi]

I’m @mikhailshilkov is doing hte codegen change. I’m makign the change to the core .NET Pulumi sdk to work here.

[pgavlin]

I believe that this is occurring because we expect all inputs to an invoke to be fully-resolved. As such, it would appear that we’ve mistyped the Identifiers field of GetPolicyDocumentStatementsPrincipalsArgs: rather than an InputList<T>, it should be an ImmutableArray<T>.

You should be able to make this work by invoking the data source inside an Apply, which looks something like this:

            var assumepolicy = role.Arn.Apply(roleArn => Pulumi.Aws.Iam.Invokes.GetPolicyDocument(new GetPolicyDocumentArgs
            {
                Statements = new[] {
                    new GetPolicyDocumentStatementsArgs
                    {
                        Actions = "sts:AssumeRole",
                        Effect = "Allow",
                        Principals = new GetPolicyDocumentStatementsPrincipalsArgs
                        {
                            Type = "Service",
                            Identifiers = "ec2.amazonaws.com",
                        }
                    },
                    new GetPolicyDocumentStatementsArgs
                    {
                        Actions = "sts:AssumeRole",
                        Effect = "Allow",
                        Principals = new GetPolicyDocumentStatementsPrincipalsArgs
                        {
                            Type = "AWS",
                            Identifiers = roleArn,
                        }
                    }
                }
            }));

cc @CyrusNajmabadi @MIkhailShilkov