ec2 Instance object doesn't handle None reservations gracefully

When you call the DescribeInstances AWS API call and give it a list of instances, if that instance was terminated some time ago, AWS in some cases may not return a InvalidInstanceID.NotFound; instead, it will return a valid response but with an empty reservations list. When this happens, boto3 does the object mapping as expected, but accessing any attribute on an ec2 instance in this state results in:

  File "/Library/Python/2.7/site-packages/boto3/resources/factory.py", line 345, in property_loader
    return self.meta.data.get(name)
AttributeError: 'NoneType' object has no attribute 'get'

This can be reproduced by executing the following on an instance ID that once existed, but has been terminated for a day or so:

import boto3
bad_iid = "i-1234567"
ec2 = boto3.Session().resource("ec2")
instance = ec2.Instance(bad_iid)
print i.launch_time

I see 2 ways of fixing this problem:

1. Catching the fact that reservations are empty and issuing spontaneous InvalidInstanceID.NotFound ourselves.
2. Expose the reservation ID as part of the Instance object so that callers can check the reservation for themselves before attempting to access other Instance attributes.

I would much prefer solution #1 as empty reservations and InvalidInstanceID.NotFound are really the same thing, and I can’t think of any possible scenario where a developer would ever need to distinguish between the 2 not found scenarios (and if they needed that detail they should probably be calling describe_instances directly and examining the contents of the dict.)

[Quentin01]

I also encountered this error recently, the workaround could be to check instance.meta.data is None just after the call to .load().

I checked the source code and the load is generated from the data in data/ec2/{date}/resources-1.json:

      "load": {
        "request": {
          "operation": "DescribeInstances",
          "params": [
            { "target": "InstanceIds[0]", "source": "identifier", "name": "Id" }
          ]
        },
        "path": "Reservations[0].Instances[0]"
      },

The code looking for the path (Reservations[0].Instances[0]) in the response is resources/response.py:153 (boto3.resources.response.RawHandler.__call__) is using jmespath which returns None if the path isn’t found in the response.

As I don’t know the source code of boto3 I can’t do a fix without knowing if I will break something or if it’s the right way but a possible fix would be to update the JSON to add the name of the exception to launch if the response is None and raise it in the response handler.

Another solution would be to detect if object.meta.data is None after a load (like in resources/factory.py:506 which is creating the load action) and raise an exception in this case.

Edit: You can also monkey patch the factory:

from botocore.exceptions import ClientError

from boto3.resources.factory import ResourceFactory

original_load_from_definition = ResourceFactory.load_from_definition

def patched_load_from_definition(*args, **kwargs):
    cls = original_load_from_definition(*args, **kwargs)

    if cls.__name__ == 'ec2.Instance':
        original_instance_load = cls.load

        def patched_instance_load(self):
            original_instance_load(self)

            if self.meta.data is None:
                raise ClientError({
                    'Error': {
                        'Code': 'InvalidInstanceID.NotFound',
                        'Message': "The instance ID '%s' does not exist" % self.id,
                    },
                }, "DescribeInstances")

        cls.load = patched_instance_load

    return cls

ResourceFactory.load_from_definition = patched_load_from_definition

[elbertwang]

+1 for seeking workaround way