question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

datastore: No way to exclude_from_indexes new properties on updated entities

See original GitHub issue

When loading an Entity from the datastore and updating it, there is no way to exclude new attributes from being indexed, since the only public API for setting exclude_from_indexes is on the Entity constructor. To work around this, I am currently setting entity._exclude_from_indexes = (...) in my code, which is not a “public” API.

Example:

client = google.cloud.datastore.Client()
key = client.key('SomeEntityKey')
entity = google.cloud.datastore.Entity(key, exclude_from_indexes=('foo', 'bar'))
entity['foo'] = 'foo'
print 'entity.exclude_from_indexes:', entity.exclude_from_indexes
client.put(entity)

entity2 = client.get(entity.key)
print 'entity2.exclude_from_indexes:', entity2.exclude_from_indexes
entity2['bar'] = 'bar'
client.put(entity2)

Output:

entity.exclude_from_indexes: frozenset(['foo', 'bar'])
entity2.exclude_from_indexes: frozenset([u'foo'])

This is actually “expected” based on how the Datastore works, however, in the code sample above, there should be some way for me to put the entity back, without copying it into a brand new Entity.

Tested using:

  • Mac OS X 10.11.6
  • Python 2.7.10
  • The following Google packages:
gapic-google-cloud-datastore-v1==0.15.3
google-auth==1.0.1
google-auth-httplib2==0.0.2
google-cloud-core==0.24.1
google-cloud-datastore==1.0.0
google-cloud-storage==1.1.1
google-gax==0.15.13
google-resumable-media==0.0.2
googleapis-common-protos==1.5.2
proto-google-cloud-datastore-v1==0.90.4

Issue Analytics

  • State:closed
  • Created 6 years ago
  • Comments:10 (7 by maintainers)

github_iconTop GitHub Comments

2reactions
evanjcommented, Aug 7, 2017

I would like a public API to read an entity from the datastore, then add new attributes that are “excluded from indexes.” There doesn’t seem to be a way to do it. I’d like some way to do the following using a publicly documented api:

existing_entity = client.get(entity.key)
existing_entity['some_new_attribute'] = 'bar'
existing_entity.add_exclude_from_indexes('some_new_attribute')
client.put(existing_entity)

The way the API currently works, the attribute 'some_new_attribute' will be indexed, unless I create a new entity using the Entity constructor.

1reaction
lukesneeringercommented, Aug 8, 2017

I am going to sleep on it, but rather than add a bunch of add and remove methods, I am inclined to just change exclude_from_indexes to a set rather than a frozenset.

Read more comments on GitHub >

github_iconTop Results From Across the Web

[datastore] new excludeFromIndexes syntax should allow for a ...
I would just like a way to define that data stored at a certain property of an entity should not be indexed at...
Read more >
Indexes | Cloud Datastore Documentation
Similarly, changing a property from indexed to excluded only affects entities subsequently written to your database. The index entries for any existing entities...
Read more >
What's the right way to use excludeFromIndexes in an ...
Here I share a small piece of code (which is an MCV from the piece of code that you shared) that works and...
Read more >
excludeFromIndexes() - gstore-node - GitBook
excludeFromIndexes (property) passing a string property or an Array of properties. ... If you save this entity, the text won't be saved in...
Read more >
Datastore: Problem updating entities - Google Groups
Hi, I am currently trying to update a kind in my database and add a field (indexed=0), the table has more than 10M...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found