Config.includes not picked up when client repacking

I’m trying to let clients add labels via files. This is part of the config for my frontends that are doing the client repacking:

Client Context:
  Platform:Darwin:
    Config.includes:
      - build.yaml
      - "/etc/%(Client.name).labels.yaml"

  Platform:Linux:
    Config.includes:
      - build.yaml
      - "/etc/%(Client.name).labels.yaml"

  Platform:Windows:
    Config.includes:
      - build.yaml
      - "%(Client.install_path)/%(Client.binary_name).labels.yaml"

But when installing the mac .pkg, the Config.includes is never picked up:

$ cat /usr/local/lib/grr/grr_3.2.4.3_amd64/grr.yaml                                                                                                                                                                                                                                                                      
Client.arch: amd64
Client.company_name: GRR Project
Client.description: '%(name) %(platform) %(arch)'
Client.foreman_check_frequency: 1800
Client.install_path: /usr/local/lib/%(Client.name)/%(ClientRepacker.output_basename)
Client.name: grr
Client.platform: darwin
Client.plist_filename: '%(Client.plist_label).plist'
Client.plist_label: '%(Client.plist_label_prefix).google.code.%(Client.name)'
Client.plist_label_prefix: com
Client.plist_path: /Library/LaunchDaemons/%(Client.plist_filename)
Client.poll_max: 600
Client.rekall_profile_cache_path: '%(Client.install_path)/rekall_profiles'
Config.includes:
- build.yaml
Config.writeback: /etc/%(Client.name).local.yaml
Logging.engines: stderr,file,syslog
Logging.path: /var/log
Logging.syslog_path: /var/run/syslog
Logging.verbose: false
Client.deploy_time: '2018-11-01 07:57:29'

The repacking is using the right contexts and is picking up Config.includes, but I have no idea where /tmp/tmpZFFwpY/grr.yaml is coming from. Maybe it comes from the pre-baked OSX template?

Repacking template: /usr/share/grr-server/grr-response-templates/templates/grr_3.2.4.3_amd64.xar.zip
DEBUG:2018-12-28 22:08:18,114 8 MainProcess 140222624716544 MainThread config_lib:682] Applying filter env for CLIENT_INSTALLER_FINGERPRINT.
Using context: [u'ClientBuilder Context', u'ClientBuilder Context', u'Arch:amd64', u'Platform:Darwin', u'Target:Darwin', u'Target:Darwin'] and labels: []
DEBUG:2018-12-28 22:08:18,116 8 MainProcess 140222624716544 MainThread config_lib:1160] Loading configuration from /tmp/tmpZFFwpY/grr.yaml
DEBUG:2018-12-28 22:08:18,117 8 MainProcess 140222624716544 MainThread config_lib:850] Configuration writeback is set to /tmp/tmpZFFwpY/grr.yaml
...
DEBUG:2018-12-28 22:08:18,126 8 MainProcess 140222624716544 MainThread build:321] Copying config option to client: Config.includes
...
INFO:2018-12-28 22:08:18,131 8 MainProcess 140222624716544 MainThread config_lib:501] Writing back configuration to file /tmp/tmpZFFwpY/grr.yaml

[grrrrrrrrr]

Ok I found an somewhat unrelated issue that would overwrite your labels. We have a few options that we always want to persist to the grr.local.yaml: https://github.com/google/grr/blob/36b8a2e5d13c1a464f65b1ce522508ff26fa409e/grr/client/grr_response_client/client_startup.py#L34

The bug is that we assume that those options are always set in the main config (or not at all). In your case, it would take the unset main config and overwrite the custom config with it - hence the null in Yaml which is a Python None.

https://github.com/google/grr/commit/76dfefd0992beb2974ba984de72a321397d262e2 fixes this bug, if you could try with a template after that commit, it should work now.

I also added an optionalfile config filter that will not raise if it can’t find the file given. This is slightly dangerous since it will hide some errors but it should make this labeling process a bit easier.

[OmarDarwish]

Your fixes worked! I can see my labels in /etc/grr.local.yaml after installing a client repacked with the code you pushed.

Thanks for looking into this!