Security-Gateway Action remove-permissions deletes valid SG rules

When run a policy to remove any SG rule that has a source IP of 0.0.0.0/0 and is not http or https the filter appears to match okay, but the action removes ALL SG rules where the source is 0.0.0.0/0, including any with HTTP or HTTPS. Use of ingress: matched does not appear to make any difference. Policy:

policies:
  - name: sg-test
    resource: security-group
    filters:
       - and:  
         - type: ingress
           OnlyPorts: [22, 80, 443]
#
         - type: ingress
           Cidr:
              value_type: cidr
              op: eq
              value: 0.0.0.0/0
#
    actions:
      - type: remove-permissions
        ingress: matched 

[jantman]

A colleague, @marcelluseasley, and I have been looking into this and we’ve identified the bug. It’s an unhandled edge case (and untested as far as I can tell) when OnlyPorts and Cidr are specified together.

It’s a very subtle issue in c7n.resources.vpc.SGPermission.process_ports:

    def process_ports(self, perm):
        found = None
        if 'FromPort' in perm and 'ToPort' in perm:
            for port in self.ports:
                if port >= perm['FromPort'] and port <= perm['ToPort']:
                    found = True
                    break
                found = False
            only_found = False
            for port in self.only_ports:
                if port == perm['FromPort'] and port == perm['ToPort']:
                    only_found = True
            if self.only_ports and not only_found:
                found = found is None or found and True or False
        return found

In the above method, the return value (found) defaults to None. If only_ports is specified and ports is not, AND the permission we’re checking matches a port in only_ports, (i.e. only_found becomes True)… the method ends up never setting found to False, it’s left at the default value of None.

So given the above, when only_ports is set and ports is not set, process_ports() returns True for ports that aren’t in only_ports but None for ports that are in only_ports.

In the __call__ method of c7n.resources.vpc.SGPermission, we have the following block of code for evaluating each permission on the SG:

            perm_matches = {}
            for idx, f in enumerate(self.vfilters):
                perm_matches[idx] = bool(f(perm))
            perm_matches['ports'] = self.process_ports(perm)
            perm_matches['cidrs'] = self.process_cidrs(perm)
            perm_matches['self-refs'] = self.process_self_reference(perm, sg_id)
            perm_match_values = list(filter(
                lambda x: x is not None, perm_matches.values()))

            # account for one python behavior any([]) == False, all([]) == True
            if match_op == all and not perm_match_values:
                continue

            match = match_op(perm_match_values)
            if match:
                matched.append(perm)

This code checks for permission matches for ports, cidrs, and self-references, and then builds a list of all of those values that are not None.

The end result of this is that when a port matches one of the OnlyPorts values, process_ports() returns None, which gets ignored when building perm_match_values. In this case, the port check is effectively ignored and if all other checks (CIDR or self_reference) are True, the permission is considered a match.

The fix for this is adding a simple branch at the end of process_ports to return False if only_ports is present and only_found is True - i.e. stop ignoring the negated match for only_ports and return False.

We’ll be trying to work up a PR for this that includes the relevant tests; it appears that OnlyPorts is not currently tested in combination with cidr or self-reference.

[kapilt]

@JoshRosen thanks for the example that seems definitely out of alignment to the intent of how that should work.