Allow for storing/using metadata about hosts

2018 update: this is an old, old ticket and it is primed to actually be implemented now that 2.0 is available to build features on top of.

The tl;dr is that users need ways to store rich data about their target hosts (and, usually, groups or “roles”), first, and then need a way of addressing that information when doing API or CLI level things, second.

Fabric 2 is built on Invoke which has a powerful configuration system, which is then exposed to tasks via a ‘context’ object. It seems likely that we will build this feature on those, something like (but not necessarily limited to):

• Standardize on some relatively generic config-style format for representing hosts; basically Connection’s parameters (user, hostname, port, connect_kwargs, timeout, etc etc) and probably with a bit more on top
  • or at least the opportunity for users to put arbitrary data on top and have that show up in the objects exposed to the user at runtime
• Update Context so it has a lightweight link to some other class or classes which turn the ‘raw’ config into usable API objects, based on some query or lookup
  • Open question is whether these show up as actual Connections or if there’s an intermediate representational class like Host

For example (again: just an off the cuff example!) perhaps we’ll set it up so host data gets its own config-style file that lives alongside the regular config files - say, $PROJECT/hosts.(yml|json|py|etc):

web1:
  host: web1
  # Implicit local user, as with Connection
web2:
  host: web2
  user: admin2
  port: 2223
db:
  # Implicit dict-key-is-the-host-value, i.e. implicit "host: db"
  user: dbadmin

Then perhaps there’s something like this (using pure Invoke style tasks for now, though certainly this would want the ability to use -H or decorators to select target hosts to ‘wrap’ the task, as in v1):

@task
def deploy_webs(c):
  # Assuming auto-creation of Connection objects...
  for cxn in [c.find_host('web1'), c.find_host('web2')]:
    cxn.run("hostname")

There are a whole lot of different ways we could slice and dice this, and a lot of directions that it could be extended in; the emphasis should be on giving users as much power and control as reasonably possible and then getting out of their way. Ideally all we’ll do is standardize on some very basic way of shoveling data into Connection objects, and add support to the core CLI exec framework to arrive at something resembling Fabric 1 re: selecting execution targets.

All while exposing these mechanisms publicly so advanced users can take matters into their own hands - again I expect anybody beyond the most basic use cases to be highly likely to fall back on a “regular Invoke style tasks + making the needful API calls within those task bodies” approach.

---

Original description

Right now a “host” is solely limited to user/hostname/port. Would be nice, even just for user fabfiles, to store additional information such as operating system, per-host settings like env.shell, and so forth.

Note that this may (or may not) be a good time to reconsider changing the default value of shell to /bin/sh.

---

Originally submitted by Jeff Forcier (bitprophet) on 2009-07-20 at 05:02pm EDT

Relations

• Duplicated by #43: /bin/bash is not always available – per host shell configuration?
• Related to #97: In some situations, pressing Enter does not reuse the previous password
• Related to #138: env.port not honored if host string lacks port specification
• Related to #3: Make use of ssh_config where possible
• Related to #76: Use decorator to define tasks

[SharkFourSix]

Hello!

After a lot of reading I still don’t understand how this really works. For example If I have a hosts.yml configuration file like this:

hosts.yml:

server1:
  host: serverip
  user: username

How should I use this to create a Connection? I had to rename the hosts.yml to fabric.yml to get access these data, through the context variable, e.g:

@task
def do(ctx):
    ctx['server1']

And it will give back a DataProxy, that I can’t use for create connection, or I just didn’t find in the documentation

My another problem: how is it possible to specify these hosts declared in the hosts.yml file with -H toggle? It only works if I create an alias in the ~/.ssh/config file which is not so great at all.

Off-topic: prompt was removed from the api? I didn’t find a corresponding method.

I have the same question. I haven’t found a single example or tutorial on how to create and use configuration files. The API documentation is very lacking in that regard.

[char101]

We can create a custom task decorator that replaces the context argument:

import fabric
from functools import wraps

def task(f, *args, **kwargs):
    @wraps(f)
    def wrapper(c, *args, **kwargs):
        c = fabric.Connection('host')
        f(c, *args, **kwargs)

    return fabric.task(wrapper, *args, **kwargs)

@task
def deploy(c):
    with c.forward_remote(9418):
        c.run('git pull http://localhost/app')

or put it in an external module

fabutil.py

import inspect
import fabric
from functools import wraps


class Connection(fabric.Connection):
  # add helper methods
  def exists(self, path):
     return not self.run(f'test -e "{path}").failed

def task(f, *args, **kwargs):
    caller = inspect.getmodule(inspect.currentframe().f_back)
    host = caller.HOST
    user = caller.USER

    @wraps(f)
    def wrapper(c, *args, **kwargs):
        c = Connection(host, user=user, connect_kwargs={'key_filename': os.path.expanduser('~/.ssh/id_rsa.pub')})
        f(c, *args, **kwargs)

    return fabric.task(wrapper, *args, **kwargs)

fabfile.py

from fabutil import task

HOST = 'host'
USER = 'user'

@task
def deploy(c):
    with c.forward_remote(9418):
        c.run('git pull http://localhost/app')