question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Excluded indexes from nested Array

See original GitHub issue

Hi team, Currently I couldn’t exclude indexs of properties inside array when using google-cloud-python

My model is quite complex,1 flow contains many flow childs and each child could contains other flow childs as well

Flow model:

{
  title: 'name',
  description: 'LONG_TEXT_HERE',
  flow_children: [ 
      {
        title: 'name',
        description: 'LONG_TEXT_HERE',
        flow_children: []
    }
   ] 
}

Im trying to follow document to exclude indexs by several way but it didnt work.

task = datastore.Entity(key=key, exclude_from_indexes=['consideration','description','flow_children','flow_children[].description'])

Still get below error google.api_core.exceptions.BadRequest: 400 The value of property "description" is longer than 1500 bytes.

Then I try to use google cloud console to update entity with value and index directly, it works.

{
  "values": [
    {
      "entityValue": {
        "properties": {
          "flow_children": {
            "arrayValue": {
              "values": [
                {
                  "entityValue": {
                    "properties": {
                      "description": {
                        "stringValue":"LONG_TEXT_HERE",
                       "excludeFromIndexes": true
          },
          "type": {
            "stringValue": "flowstep"
          },
          "title": {
            "stringValue": "Go to back yard"
          }
        }
      }
    }
  ]
}

Anyone could help me with this case please 😦

Thanks My envs: python 3.6.4 google-cloud-datastore 1.4.0 Macosx High Serria

Issue Analytics

  • State:closed
  • Created 5 years ago
  • Comments:9 (6 by maintainers)

github_iconTop GitHub Comments

2reactions
paddiecommented, Nov 13, 2019

For anyone else struggling with this one and landing on this page, much like myself, here’s a snippet that is a bit more comprehensible than the one above IMO:

key = client.key("MyKind", f"{account_id}_{integration_id}")
# this is not enough to not index the secret
integration = datastore.Entity(key=key, exclude_from_indexes=["secret"])

# ensure that nested fields are not indexed - *le sigh*
keys = secret.keys()
secret_entity = datastore.Entity(exclude_from_indexes=list(keys))
secret_entity.update(secret)


record = {
    "integration_id": integration_id,
    "account_id": account_id,
    "secret": secret_entity,
    "created_at": datetime.utcnow(),
    "updated_at": datetime.utcnow(),
    "visible_fields": visible_fields or [],
}
integration.update(record)

client.put(integration)
1reaction
tseavercommented, Jul 25, 2018

@duyphuong13 For nested values to be excluded, they must be created as Entity instances, rather than bare dicts. So, for your example, I wrote this gist.

Read more comments on GitHub >

github_iconTop Results From Across the Web

How can I exclude an Array field from indexes in Google ...
To exclude an Array type property from indexes, each element within the array must have excludeFromIndexes set to true .
Read more >
Class: Array (Ruby 2.6.2)
Arrays are ordered, integer-indexed collections of any object. Array indexing starts at 0, as in C or Java. A negative index is assumed...
Read more >
Indexing on ndarrays — NumPy v1.25.dev0 Manual
Integer array indexing allows selection of arbitrary items in the array based on their N-dimensional index. Each integer array represents a number of...
Read more >
Understand JavaScript Arrays and Common ... - Launch School
We talk about what the JavaScript Array is, nested arrays and array comparison, ... You can refer to any element of an array...
Read more >
Indexes | Cloud Datastore Documentation
If you exclude priority from indexes and create an index for priority and done, Datastore mode will not create index entries for the...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found