local exec fails to parse terraform.py

There seems to be a problem parsing the terraform.py inside a local-exec provisioner in Terraform

Taking this as an example trigger in terraform to run ansible

resource "null_resource" "jenkins_ansible" {
  // count = 2 ### "${length(list(var.jenkins_private_ip))}"
  triggers {
    // jenkins_ips = "${var.jenkins_private_ip[count.index]}"
    jenkins_ips = "${var.jenkins_private_ip}"
  }

  provisioner "local-exec" {
    interpreter = ["/bin/bash", "-c"]
    command = "cd ${var.ansible_dir}; ANSIBLE_TF_DIR=${var.ANSIBLE_TF_DIR} ANSIBLE_HOST_KEY_CHECKING=False ansible-playbook --inventory-file=${var.ansible_dir}/terraform.py jenkins/master.yml"
  }
}

gives me this output

[WARNING]:  * Failed to parse /Users/moi/ansible/sandbox-
test/terraform.py with script plugin: Inventory script
(/Users/moi/ansible/sandbox-test/terraform.py) had an execution
error: 2018/03/14 13:44:52 [INFO] Terraform version: 0.11.3
3802b14260603f90c7a1faf55994dcc8933e2069 2018/03/14 13:44:52 [INFO] Go runtime
version: go1.9.1 2018/03/14 13:44:52 [INFO] CLI args: []string{"terraform",
"state", "pull", "-input=false"} 2018/03/14 13:44:52 [DEBUG] Attempting to open
CLI config file: /Users/moi/.terraformrc 2018/03/14 13:44:52
[DEBUG] File doesn't exist, but doesn't need to. Ignoring. 2018/03/14 13:44:52
[INFO] CLI command args: []string{"state", "pull", "-input=false"} 2018/03/14
13:44:52 [DEBUG] command: loading backend config file:
/Users/moi/foo/git-dev/idam/idam-iac/devops-
idam-static-dev 2018/03/14 13:44:52 [WARN] BackendOpts.Config not set, but
config found 2018/03/14 13:44:52 [TRACE] Preserving existing state lineage
"a34ef702-b45c-49ee-ab06-985baa37f7eb" 2018/03/14 13:44:52 [TRACE] Preserving
existing state lineage "a34ef702-b45c-49ee-ab06-985baa37f7eb" 2018/03/14
13:44:52 [INFO] Building AWS region structure 2018/03/14 13:44:52 [INFO]
Building AWS auth structure 2018/03/14 13:44:52 [INFO] Setting AWS metadata API
timeout to 100ms 2018/03/14 13:44:53 [INFO] Ignoring AWS metadata API endpoint
at default location as it doesn't return any instance-id 2018/03/14 13:44:53
[INFO] AWS Auth provider used: "SharedCredentialsProvider" 2018/03/14 13:44:53
[INFO] Initializing DeviceFarm SDK connection 2018/03/14 13:44:53 [DEBUG]
Trying to get account ID via iam:GetUser 2018/03/14 13:44:54 [INFO] command:
backend initialized: *s3.Backend 2018/03/14 13:44:54 [DEBUG] checking for
provider in "." 2018/03/14 13:44:54 [DEBUG] checking for provider in
"/terraform" 2018/03/14 13:44:54 [DEBUG] checking for provider in
".terraform/plugins/darwin_amd64" 2018/03/14 13:44:54 [DEBUG] found provider
"terraform-provider-aws_v1.11.0_x4" 2018/03/14 13:44:54 [DEBUG] found provider
"terraform-provider-null_v1.0.0_x4" 2018/03/14 13:44:54 [DEBUG] checking for
provider in "/Users/moi/.terraform.d/plugins" 2018/03/14 13:44:54
[DEBUG] checking for provider in
"/Users/moi/.terraform.d/plugins/darwin_amd64" 2018/03/14
13:44:54 [DEBUG] found provider "terraform-provider-ansible_v0.0.2" 2018/03/14
13:44:54 [DEBUG] found valid plugin: "aws", "1.11.0",
"/Users/moi/foo/git-dev/idam/idam-iac
/devops-idam-static-dev/.terraform/plugins/darwin_amd64/terraform-provider-
aws_v1.11.0_x4" 2018/03/14 13:44:54 [DEBUG] found valid plugin: "null",
"1.0.0", "/Users/moi/foo/git-dev/idam/idam-
iac/devops-idam-static-dev/.terraform/plugins/darwin_amd64/terraform-provider-
null_v1.0.0_x4" 2018/03/14 13:44:54 [DEBUG] found valid plugin: "ansible",
"0.0.2", "/Users/moi/.terraform.d/plugins/darwin_amd64/terraform-
provider-ansible_v0.0.2" 2018/03/14 13:44:54 [DEBUG] checking for provisioner
in "." 2018/03/14 13:44:54 [DEBUG] checking for provisioner in "/terraform"
2018/03/14 13:44:54 [DEBUG] checking for provisioner in
".terraform/plugins/darwin_amd64" 2018/03/14 13:44:54 [DEBUG] checking for
provisioner in "/Users/moi/.terraform.d/plugins" 2018/03/14
13:44:54 [DEBUG] checking for provisioner in
"/Users/moi/.terraform.d/plugins/darwin_amd64" 2018/03/14
13:44:54 [INFO] command: backend *s3.Backend is not enhanced, wrapping in local
2018/03/14 13:44:54 [TRACE] Preserving existing state lineage "294a1e0c-
cb5d-4365-90e6-0f6a9cecb4fc" 2018/03/14 13:44:54 [TRACE] Preserving existing
state lineage "294a1e0c-cb5d-4365-90e6-0f6a9cecb4fc" 2018/03/14 13:44:54
[TRACE] Preserving existing state lineage "294a1e0c-
cb5d-4365-90e6-0f6a9cecb4fc" 2018/03/14 13:44:54 [DEBUG] plugin: waiting for
all plugin processes to complete...
 [WARNING]:  * Failed to parse /Users/moi/ansible/sandbox-
test/terraform.py with ini plugin: /Users/moi/ansible/sandbox-
test/terraform.py:3: Expected key=value host variable assignment, got: json
 [WARNING]: Unable to parse /Users/moi/ansible/sandbox-
test/terraform.py as an inventory source
 [WARNING]: No inventory was parsed, only implicit localhost is available
 [WARNING]: provided hosts list is empty, only localhost is available. Note
that the implicit localhost does not match 'all'
 [WARNING]: Could not match supplied host pattern, ignoring: jenkins_hosts

when the same command is run in the terminal it works just fine

$ cd /Users/moi/ansible/sandbox-test; ANSIBLE_TF_DIR=/Users/moi/foo/git-dev/idam/idam-iac/devops-idam-static-dev ANSIBLE_HOST_KEY_CHECKING=False ansible-playbook --inventory-file=/Users/moi/ansible/sandbox-test/terraform.py jenkins/master.yml

Should it be used like that within Terraform?

As I am very close to creating a separate application to that is triggered on remote state callback that will then trigger the ansible scripts, and have a feeling it could be a more logical way to separate the concern and domains.

Wondering if anyone else came across the same issue.

[nbering]

If that isn’t the problem, I’d next look at whether you’ve somehow configured Ansible 2.4+ to restrict the Inventory parsing mechanism. There’s a bunch of things to look at there if this still doesn’t work after you run outside the Terraform session.

[dnitsch]

Ok, that could have been part of the problem (the locking, d’oh, I mean, not the perms that was fine) - in fact it was most likely that when ran as part of the null_resource on trigger. I have moved it out to separate callback application so that it’s guaranteed to run after the state has been released.