Provide a method to convert ndb Key's urlsafe format to/from datastore.Key

In GAE standard, ndb has the concept of the urlsafe string version of a Key. e.g. url_string = sandy_key.urlsafe() produces a result like agVoZWxsb3IPCxIHQWNjb3VudBiZiwIM.

However, I cannot find a way to use this format with the google.cloud.datastore library for code running outside GAE standard. Please add a method to the Key class to create urlsafe strings and a method to generate Keys from urlsafe strings. Alternatively, provide some documentation on how to decode/encode them with the existing library.

NB, the console support page says “The encoding method is available in any Cloud Datastore client library.”

[dhermes]

So I did some more digging and it turns out the Reference protobuf in App Engine is proto1 (i.e. first generation of protobuf), so the modern proto2/proto3 tooling can’t convert it. I got very close to converting using standard tools, but some small differences in the protocols prevent it.

You can definitely include the ProtocolBuffer module in your application (in any location):

google-cloud-sdk/platform/google_appengine/google/net/proto/ProtocolBuffer.py

and then just define your own local Reference and Path classes from

google-cloud-sdk/platform/google_appengine/google/appengine/datastore/entity_pb.py

I.e. you can use these files outside of App Engine. I have done so in the gist I linked to and it works just fine.

[michaelenglo]

Go to the source and you will find this:

def _DecodeUrlSafe(urlsafe):
  """Decode a url-safe base64-encoded string.
  This returns the decoded string.
  """
  if not isinstance(urlsafe, basestring):
    raise TypeError('urlsafe must be a string; received %r' % urlsafe)
  if isinstance(urlsafe, unicode):
    urlsafe = urlsafe.encode('utf8')
  mod = len(urlsafe) % 4
  if mod:
    urlsafe += '=' * (4 - mod)
  #This is 3-4x faster than urlsafe_b64decode()
  return base64.b64decode(urlsafe.replace('-', '+').replace('_', '/'))

I just copied and pasted this snippet in my code and it works now. Hope this helps.