Failing on TASK [Read the static blueprint content], variable not accepted

Hello,

I completed an installation of Hortonworks making small changes to the provided configuration. After the completion i exported the blueprint and tried to install another cluster making use of this blueprint.

I am encountering the following issue: fatal: [localhost]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: '**zookeeper_log_max_backup_size**' is undefined\n\nThe error appears to have been in 'my-path/ansible-hortonworks/playbooks/set_variables.yml': line 35, column 11, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n - name: Read the static blueprint content\n ^ here\n"}

The blueprint correctly contains the following: "zookeeper_log_max_backup_size": "10", and log4j.appender.ROLLINGFILE.MaxFileSize={{zookeeper_log_max_backup_size}}MB

If i hard code in the blueprint the value of the variable (10) in all the places where i have the variable the error changes as follow: fatal: [localhost]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: '**zookeeper_log_number_of_backup_files**' is undefined\n\nThe error appears to have been in 'my-path/ansible-hortonworks/playbooks/set_variables.yml': line 35, column 11, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n - name: Read the static blueprint content\n ^ here\n"}

For this reason, my understanding is that it is not able to work with variables. Is it expected? Do i need to remove all the variables from the blueprint?

Thanks

[alexandruanghel]

Ah, damn, this is Ansible way of defining variables ({{ and }}), interfering with the Ambari way of defining variables in the log4j settings: same ({{ and }}).

So Ambari thinks it needs the zookeeper_log_max_backup_size variable, but that’s not true, this is for Ambari use only.

You should probably surround your blueprint in {% raw %} ... {% endraw %} statement: https://docs.ansible.com/ansible/devel/user_guide/playbooks_advanced_syntax.html#unsafe-or-raw-strings

Alternatively, you can change template with file in here: https://github.com/hortonworks/ansible-hortonworks/blob/master/playbooks/roles/ambari-blueprint/tasks/main.yml#L59 if you don’t need to use any Ansible variables in your blueprint

[lhoss]

I found this (closed) issue when searching for any infos around getting TypeError: must be string or buffer on the blueprint upload step, exactly as described in this issue:

5. encountered finally this issue: TASK [ambari-blueprint : Upload the blueprint testHDP3_blueprint to the Ambari server] *********************************************************************************************** An exception occurred during task execution. To see the full traceback, use -vvv. The error was: TypeError: must be string or buffer, not dict fatal: [orto-0]: FAILED! => {"changed": false, "content": "", "msg": "Status code was -1 and not [200, 201, 202, 409]: An unknown error occurred: must be string or buffer, not dict", "redirected": false, "status": -1, "url": "http://orto-0:8080/api/v1/blueprints/testHDP3_blueprint"}

And from Alexandru’s reply:

This is a very annoying bit that I always had when sending JSON or YAML to Ambari API using the the Ansible uri.

Note I just reproduced this issue using a recent repo version (https://github.com/hortonworks/ansible-hortonworks/commit/3020c52ae870fb1475e99445e2c6c57bbb014814) that already contains any fixes mentioned in this issue. Finally I found myself a fix, which surprised myself: Simply upgrading to a new enough ansible (v2.6.3), where as a 1.update to v2.5.3 did not fix it ( from the original redhat installed v2.4.2) (ps: If you like I can still create a separate issue, in case other people run into the same issue… )

---

Details of my TypeError case doing a uri module POST with a dict to_json

My server env (running redhat):

% ansible --version
ansible 2.5.3
...
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.5 (default, May 31 2018, 09:41:32) [GCC 4.8.5 20150623 (Red Hat 4.8.5-28)]

Error msg incl. python stacktrace

TASK [ambari-blueprint : Upload the blueprint x86demo01_blueprint to the Ambari server] ************************
task path: /home/deployer/src/ansible-hortonworks/playbooks/roles/ambari-blueprint/tasks/main.yml:91
Using module file /usr/lib/python2.7/site-packages/ansible/modules/net_tools/basics/uri.py
<a301-8551-4011> ESTABLISH LOCAL CONNECTION FOR USER: root
<a301-8551-4011> EXEC /bin/sh -c '/usr/bin/python && sleep 0'
The full traceback is:
Traceback (most recent call last):
  File "/tmp/ansible_ZXQgDB/ansible_modlib.zip/ansible/module_utils/urls.py", line 1050, in fetch_url
    client_key=client_key, cookies=cookies)
  File "/tmp/ansible_ZXQgDB/ansible_modlib.zip/ansible/module_utils/urls.py", line 957, in open_url
    r = urllib_request.urlopen(*urlopen_args)
  File "/usr/lib64/python2.7/urllib2.py", line 154, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib64/python2.7/urllib2.py", line 431, in open
    response = self._open(req, data)
  File "/usr/lib64/python2.7/urllib2.py", line 449, in _open
    '_open', req)
  File "/usr/lib64/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/usr/lib64/python2.7/urllib2.py", line 1244, in http_open
    return self.do_open(httplib.HTTPConnection, req)
  File "/usr/lib64/python2.7/urllib2.py", line 1211, in do_open
    h.request(req.get_method(), req.get_selector(), req.data, headers)
  File "/usr/lib64/python2.7/httplib.py", line 1041, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib64/python2.7/httplib.py", line 1075, in _send_request
    self.endheaders(body)
  File "/usr/lib64/python2.7/httplib.py", line 1037, in endheaders
    self._send_output(message_body)
  File "/usr/lib64/python2.7/httplib.py", line 885, in _send_output
    self.send(message_body)
  File "/usr/lib64/python2.7/httplib.py", line 857, in send
    self.sock.sendall(data)
  File "/usr/lib64/python2.7/socket.py", line 224, in meth
    return getattr(self._sock,name)(*args)
TypeError: must be string or buffer, not dict

Working fix:

• run with ansible v2.6.3
  • Note; It would be interesting to test intermediary versions, and if there’s already a 2.5.x version that works (then I’ld suggest to clarifiy the role’s minimal ansible version!)

Earlier tried fixes not working:

• run with ansible v2.5.3
• idea (only): try with newer python than v2.7.5 (but could not easily try that)
• try with setting the uri task’s body_format: json, as was proposed in https://github.com/ansible/ansible-modules-core/issues/265
  • but then the request BODY must have changed, because the ambari-server returned with HTTP 500 and following server log stacktrace

10 Dec 2018 13:47:04,547 ERROR [ambari-client-thread-28] ContainerResponse:419 - The RuntimeException could not be mapped to a response, re-throwing to the HTT
P container
com.google.gson.JsonSyntaxException: java.lang.IllegalStateException: Expected a string but was BEGIN_OBJECT at line 1 column 2
        at com.google.gson.Gson.fromJson(Gson.java:806)
        at com.google.gson.Gson.fromJson(Gson.java:761)
        at org.apache.ambari.server.api.GsonJsonProvider.readFrom(GsonJsonProvider.java:70)
        at com.sun.jersey.spi.container.ContainerRequest.getEntity(ContainerRequest.java:490)