HCL v0.12 brings incompatible updates.

Steps to reproduce: upgrade terraform to latest version.

cd /tmp/

git clone https://github.com/terraform-providers/terraform-provider-aws.git
Cloning into 'terraform-provider-aws'...
remote: Enumerating objects: 17, done.
remote: Counting objects: 100% (17/17), done.
remote: Compressing objects: 100% (11/11), done.
remote: Total 97677 (delta 7), reused 11 (delta 6), pack-reused 97660
Receiving objects: 100% (97677/97677), 83.75 MiB | 8.58 MiB/s, done.
Resolving deltas: 100% (62884/62884), done.
Checking out files: 100% (6246/6246), done.

cd terraform-provider-aws/examples/two-tier

terraform init

Initializing provider plugins...
- Checking for available provider plugins on https://releases.hashicorp.com...
- Downloading plugin for provider "aws" (2.14.0)...

The following providers do not have any version constraints in configuration,
so the latest version was installed.

To prevent automatic upgrades to new major versions that may contain breaking
changes, it is recommended to add version = "..." constraints to the
corresponding provider blocks in configuration, with the constraint strings
suggested below.

* provider.aws: version = "~> 2.14"

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

terraform 0.12upgrade

This command will rewrite the configuration files in the given directory so
that they use the new syntax features from Terraform v0.12, and will identify
any constructs that may need to be adjusted for correct operation with
Terraform v0.12.

We recommend using this command in a clean version control work tree, so that
you can easily see the proposed changes as a diff against the latest commit.
If you have uncommited changes already present, we recommend aborting this
command and dealing with them before running this command again.

Would you like to upgrade the module in the current directory?
  Only 'yes' will be accepted to confirm.

  Enter a value: yes

-----------------------------------------------------------------------------

Upgrade complete!

The configuration files were upgraded successfully. Use your version control
system to review the proposed changes, make any necessary adjustments, and
then commit.

from glob import iglob
import hcl

for filename in iglob('**.tf'):
    with open(filename, 'r') as f:
        print(filename)
        hcl.loads(f.read())

python3  ~/tmp/testhcl.py
main.tf
Traceback (most recent call last):
  File "/home/evgeniy/tmp/testhcl.py", line 7, in <module>
    hcl.loads(f.read())
  File "/home/evgeniy/.pyenv/versions/3.7.3/lib/python3.7/site-packages/hcl/api.py", line 66, in loads
    return HclParser().parse(s)
  File "/home/evgeniy/.pyenv/versions/3.7.3/lib/python3.7/site-packages/hcl/parser.py", line 326, in parse
    return self.yacc.parse(s, lexer=Lexer())
  File "/home/evgeniy/.pyenv/versions/3.7.3/lib/python3.7/site-packages/ply/yacc.py", line 333, in parse
    return self.parseopt_notrack(input, lexer, debug, tracking, tokenfunc)
  File "/home/evgeniy/.pyenv/versions/3.7.3/lib/python3.7/site-packages/ply/yacc.py", line 1201, in parseopt_notrack
    tok = call_errorfunc(self.errorfunc, errtoken, self)
  File "/home/evgeniy/.pyenv/versions/3.7.3/lib/python3.7/site-packages/ply/yacc.py", line 192, in call_errorfunc
    r = errorfunc(token)
  File "/home/evgeniy/.pyenv/versions/3.7.3/lib/python3.7/site-packages/hcl/parser.py", line 318, in p_error
    raise ValueError(msg)
ValueError: Line 3, column 70: unexpected IDENTIFIER

[pop]

As a user of pyhcl I would expect pyhcl’s API to be something like hcl.load(input_tf_file, hcl_version=1) for loading pre-0.12 Terraform and hcl.load(other_tf_file, hcl_version=2) for loading 0.12-compatible files. (with version defaulting to 2 going forward).

Of course somebody then needs to do the work of figuring out how to parse HCL2 files. Easier said than done. 😓

[sbalbach]

If you don’t want to wait for the pull request, try out the code in my fork: https://github.com/StateFarmIns/pyhcl