Editing annotation access through API (or viewing them through UI) sometimes deletes annotation elements [HIGH PRIORITY]
See original GitHub issueThe is a bug that causes the dangerous side effect of deleting annotation elements when the access permissions are edited!
This behavior happens CONSISTENTLY when we APPEND to the existing users or groups, and OCCASIONALLY when we replace the users or groups with new ones. Until this bug is fixed, try to avoid changing annotation access permissions, and if you really do have to do it, make SURE you replace existing users/groups as opposed to appending to them.
To help track down the bug here are some details:
- Here is a video showing this behavior in real-time: https://youtu.be/8jcH_9uX5m8
- This only happens when the annotation access is edited through the girder client API, but not through the UI.
- One particular folder and all its subfolders show the consistent behavior of deleting the annotation elements even when we fully replace the existing list of users or groups. The other folders only show this behavior when we append to the list of users as opposed to fully replacing them.
- The particular DSA instance we are using may not be the last version (or it may be, @dgutman can comment on that). I know for sure that the DSA instance version we are using uses https, and therefore
/gider
needs to be added in the middle of the link when a DSA slide is opened in the HistomicsTK UI. See this video capture: https://youtu.be/iyX2MXCYMEk - This is the code snippet I am using to edit the annotation access:
def update_permissions_for_annotation(
gc, annotation_id=None, annotation=None,
groups_to_add=[], replace_original_groups=True,
users_to_add=[], replace_original_users=True):
"""Update permissions for a single annotation.
Parameters
----------
gc : gider_client.GirderClient
authenticated girder client instance
annotation_id : str
girder id of annotation
annotation : dict
overrides annotation_id if given
groups_to_add : list
each entry is a dict containing the information about user groups
to add and their permission levels. A sample entry must have the
following keys
- level, int -> 0 (view), 1 (edit) or 2 (owner)
- name, str -> name of user group
- id, st -> girder id of user group
replace_original_groups : bool
whether to replace original groups or append to them
users_to_add : list
each entry is a dict containing the information about user
to add and their permission levels. A sample entry must have the
following keys
- level, int -> 0 (view), 1 (edit) or 2 (owner)
- login, str -> username of user
- id, st -> girder id of user
replace_original_users
whether to replace original users or append to them
Returns
-------
dict
server response
"""
# TODO -- this is an especially damaging bug! Fix me!
GIRDERBUG = """
We've discovered a bug that causes the dangerous side effect of
deleting annotation elements whent he access permissions are edited!
This behavior happens CONSISTENTLY when we APPEND to the existing
users of groups, and OCCASIONALLY when we replace the users or
groups with new ones. Until this bug is fixed, try to avoid
changing annotation access permissions, and if you really do
have to do it, make SURE you replace existing users/groups
as opposed to appending to them."""
warnings.warn(GIRDERBUG, RuntimeWarning)
global ASKTOCONTINUE
if ASKTOCONTINUE:
warnings.warn("""
> Press any key to continue at your own risk, or..
> Press Control+C to stop.""")
input("")
ASKTOCONTINUE = False
if annotation is not None:
annotation_id = annotation['_id']
elif annotation_id is None:
raise Exception(
"You must provide either the annotation or its girder id.")
# get current permissions
current = gc.get('/annotation/%s/access' % annotation_id)
# add or replace as needed
if replace_original_groups:
current['groups'] = []
else:
raise Exception(GIRDERBUG)
if replace_original_users:
current['users'] = []
else:
raise Exception(GIRDERBUG)
for group in groups_to_add:
current['groups'].append(group)
for user in users_to_add:
current['users'].append(user)
# now update accordingly
# OLD WAY (BAD!!)
# return gc.put('/annotation/%s/access?access=%s' % (
# annotation_id, json.dumps(current)))
# PROPER WAY
return gc.put('/annotation/%s/access' % annotation_id, data={
'access': json.dumps(current)})
- This is how I managed to revert the annotation and get back the elements:
def revert_annotation(
gc, annotation_id=None, annotation=None, version=None,
revert_to_nonempty_elements=False, only_revert_if_empty=True):
"""Revert an annotation to a previous version.
Parameters
----------
gc : girder_client.GirderClient
authenticated girder client
annotation_id : str
girder id of annotation
annotation : dict
overrides annotation_id if given
version : int
versoin number for annotation. If None, and
not revert_to_nonempty_elements
the default behavior of the endpoint is evoked, which reverts the
annotation if it was deleted and if not, reverts to the last version.
revert_to_nonempty_elements : bool
if true, reverts to the most recent version of the annotation
with non-empty elements.
Returns
-------
dict
server response
"""
if annotation is not None:
annotation_id = annotation['_id']
elif annotation_id is None:
raise Exception(
"You must provide either the annotation or its girder id.")
history = gc.get("/annotation/%s/history" % annotation_id)
# no need to revert if empty
if only_revert_if_empty and len(history[0]["groups"]) > 0:
return dict()
if (version is None) and revert_to_nonempty_elements:
# NOTE: even though the "history" may show
# the elements as empty, the "groups" attribute is really the
# indication if the annotation version actually has some elements.
# TODO -- This is likely a bug (?); fix me!!!
for ver in history:
if len(ver["groups"]) > 0:
version = ver['_version']
break
ver = "" if version is None else "?version=%d" % version
if version is None:
print(" Reverting ...")
else:
print(" Reverting to version %d" % version)
return gc.put("/annotation/%s/history/revert%s" % (annotation_id, ver))
-
Even though the “history” may show the elements as empty for a particular annotation version, the “groups” attribute is really the indication if the annotation version actually has some elements. Is this a bug??
-
We’ve experience some of these things before and always were baffled as they weren’t so reproducible. Now, it seems this is a reproducible bug, especially on that particular server and collection. Annotations are showing up in the wrong place and observing an annotation causes it to disappear! I recorded a video of this behavior: https://youtu.be/Q1Xj7yBuZFQ March 12, 2020 meeting update: Here is the history of the moved anntoation …
[
{
"_id": "5d9cc27c46929b04713bfdb8",
"_version": 331490,
"_versionId": "5e65255c7632342dac6a7a32",
"access": {
"groups": [
{
"flags": [],
"id": "5d9624c946929b0471d03ae3",
"level": 1
},
{
"flags": [],
"id": "5e44a2e0ddda5f8398785304",
"level": 0
}
],
"users": []
},
"annotation": {},
"created": "2019-10-08T17:08:12.243000+00:00",
"creatorId": "5ccbaefde629140075e2ba00",
"groups": [
"(+)tumor",
"(-)tumor",
"roi",
"unknown"
],
"itemId": "5d94313746929b0471cf4d81",
"public": false,
"updated": "2020-03-12T19:27:08.188000+00:00",
"updatedId": "5cc9bbd2e629140075e2b3fa"
},
{
"_active": false,
"_annotationId": "5d9cc27c46929b04713bfdb8",
"_id": "5e6a8d0c5ba2ecde741a6110",
"_version": 331478,
"_versionId": "5e620c7634679044bdaa77ac",
"access": {
"groups": [
{
"flags": [],
"id": "5d9624c946929b0471d03ae3",
"level": 1
}
],
"users": []
},
"annotation": {},
"created": "2019-10-08T17:08:12.243000+00:00",
"creatorId": "5ccbaefde629140075e2ba00",
"groups": [],
"itemId": "5d94313746929b0471cf4d81",
"public": false,
"updated": "2020-03-06T09:01:02.763000+00:00",
"updatedId": "5cc9bbd2e629140075e2b3fa"
},
{
"_active": false,
"_annotationId": "5d9cc27c46929b04713bfdb8",
"_id": "5e65255c7632342dac6a7a32",
"_version": 331036,
"_versionId": "5e620c7634679044bdaa77ac",
"access": {
"groups": [
{
"flags": [],
"id": "5d9624c946929b0471d03ae3",
"level": 1
},
{
"flags": [],
"id": "5e44a2e0ddda5f8398785304",
"level": 0
}
],
"users": []
},
"annotation": {},
"created": "2019-10-08T17:08:12.243000+00:00",
"creatorId": "5ccbaefde629140075e2ba00",
"groups": [
"(+)tumor",
"(-)tumor",
"roi",
"unknown"
],
"itemId": "5d94313746929b0471cf4d81",
"public": false,
"updated": "2020-03-06T09:01:02.763000+00:00",
"updatedId": "5cc9bbd2e629140075e2b3fa"
},
{
"_active": false,
"_annotationId": "5d9cc27c46929b04713bfdb8",
"_id": "5e62114e34679044bdaadac4",
"_version": 330821,
"_versionId": "5e62096a34679044bda9a06e",
"access": {
"groups": [
{
"flags": [],
"id": "5e61330034679044bda81c85",
"level": 1
}
],
"users": [
{
"flags": [],
"id": "5956b83b92ca9a000d9c0488",
"level": 2
},
{
"flags": [],
"id": "59bc677892ca9a0017c2e855",
"level": 2
},
{
"flags": [],
"id": "5ccbadf2e629140075e2b9f4",
"level": 2
},
{
"flags": [],
"id": "5ccbae92e629140075e2b9fa",
"level": 2
},
{
"flags": [],
"id": "5ccbaed5e629140075e2b9fd",
"level": 2
},
{
"flags": [],
"id": "5ccbaefde629140075e2ba00",
"level": 2
}
]
},
"annotation": {},
"created": "2019-10-08T17:08:12.243000+00:00",
"creatorId": "5ccbaefde629140075e2ba00",
"groups": [],
"itemId": "5d94313746929b0471cf4d81",
"public": false,
"updated": "2020-03-06T08:39:40.078000+00:00",
"updatedId": "5cc9bbd2e629140075e2b3fa"
},
{
"_active": false,
"_annotationId": "5d9cc27c46929b04713bfdb8",
"_id": "5e620c7634679044bdaa77ac",
"_version": 330757,
"_versionId": "5e62096a34679044bda9a06e",
"access": {
"groups": [
{
"flags": [],
"id": "5d9624c946929b0471d03ae3",
"level": 1
},
{
"flags": [],
"id": "5e44a2e0ddda5f8398785304",
"level": 0
}
],
"users": []
},
"annotation": {},
"created": "2019-10-08T17:08:12.243000+00:00",
"creatorId": "5ccbaefde629140075e2ba00",
"groups": [
"(+)tumor",
"(-)tumor",
"roi",
"unknown"
],
"itemId": "5d94313746929b0471cf4d81",
"public": false,
"updated": "2020-03-06T08:39:40.078000+00:00",
"updatedId": "5cc9bbd2e629140075e2b3fa"
},
{
"_active": false,
"_annotationId": "5d9cc27c46929b04713bfdb8",
"_id": "5e620c4c34679044bdaa60cd",
"_version": 330106,
"_versionId": "5e62014b34679044bda8d6d0",
"access": {
"groups": [
{
"flags": [],
"id": "5e61330034679044bda81c85",
"level": 1
}
],
"users": [
{
"flags": [],
"id": "5956b83b92ca9a000d9c0488",
"level": 2
},
{
"flags": [],
"id": "59bc677892ca9a0017c2e855",
"level": 2
},
{
"flags": [],
"id": "5ccbadf2e629140075e2b9f4",
"level": 2
},
{
"flags": [],
"id": "5ccbae92e629140075e2b9fa",
"level": 2
},
{
"flags": [],
"id": "5ccbaed5e629140075e2b9fd",
"level": 2
},
{
"flags": [],
"id": "5ccbaefde629140075e2ba00",
"level": 2
}
]
},
"annotation": {},
"created": "2019-10-08T17:08:12.243000+00:00",
"creatorId": "5ccbaefde629140075e2ba00",
"groups": [],
"itemId": "5d94313746929b0471cf4d81",
"public": false,
"updated": "2020-03-06T08:26:39.178000+00:00",
"updatedId": "5cc9bbd2e629140075e2b3fa"
},
{
"_active": false,
"_annotationId": "5d9cc27c46929b04713bfdb8",
"_id": "5e62096a34679044bda9a06e",
"_version": 330044,
"_versionId": "5e62014b34679044bda8d6d0",
"access": {
"groups": [
{
"flags": [],
"id": "5d9624c946929b0471d03ae3",
"level": 1
},
{
"flags": [],
"id": "5e44a2e0ddda5f8398785304",
"level": 0
}
],
"users": []
},
"annotation": {},
"created": "2019-10-08T17:08:12.243000+00:00",
"creatorId": "5ccbaefde629140075e2ba00",
"groups": [
"(+)tumor",
"(-)tumor",
"roi",
"unknown"
],
"itemId": "5d94313746929b0471cf4d81",
"public": false,
"updated": "2020-03-06T08:26:39.178000+00:00",
"updatedId": "5cc9bbd2e629140075e2b3fa"
},
{
"_active": false,
"_annotationId": "5d9cc27c46929b04713bfdb8",
"_id": "5e62093f34679044bda98b81",
"_version": 329343,
"_versionId": "5e6135ed34679044bda81df4",
"access": {
"groups": [
{
"flags": [],
"id": "5e61330034679044bda81c85",
"level": 1
}
],
"users": [
{
"flags": [],
"id": "5956b83b92ca9a000d9c0488",
"level": 2
},
{
"flags": [],
"id": "59bc677892ca9a0017c2e855",
"level": 2
},
{
"flags": [],
"id": "5ccbadf2e629140075e2b9f4",
"level": 2
},
{
"flags": [],
"id": "5ccbae92e629140075e2b9fa",
"level": 2
},
{
"flags": [],
"id": "5ccbaed5e629140075e2b9fd",
"level": 2
},
{
"flags": [],
"id": "5ccbaefde629140075e2ba00",
"level": 2
}
]
},
"annotation": {},
"created": "2019-10-08T17:08:12.243000+00:00",
"creatorId": "5ccbaefde629140075e2ba00",
"groups": [],
"itemId": "5d94313746929b0471cf4d81",
"public": false,
"updated": "2020-03-06T07:51:53.819000+00:00",
"updatedId": "5cc9bbd2e629140075e2b3fa"
},
{
"_active": false,
"_annotationId": "5d9cc27c46929b04713bfdb8",
"_id": "5e62014b34679044bda8d6d0",
"_version": 329281,
"_versionId": "5e6135ed34679044bda81df4",
"access": {
"groups": [
{
"flags": [],
"id": "5d9624c946929b0471d03ae3",
"level": 1
},
{
"flags": [],
"id": "5e44a2e0ddda5f8398785304",
"level": 0
}
],
"users": []
},
"annotation": {},
"created": "2019-10-08T17:08:12.243000+00:00",
"creatorId": "5ccbaefde629140075e2ba00",
"groups": [
"(+)tumor",
"(-)tumor",
"roi",
"unknown"
],
"itemId": "5d94313746929b0471cf4d81",
"public": false,
"updated": "2020-03-06T07:51:53.819000+00:00",
"updatedId": "5cc9bbd2e629140075e2b3fa"
}
]
-
March 12, 2020 meeting update: This works even while the database is stale and is not being simultaneously backed up, so ignore the point below.
-
On a related note, when a database is being backed up or if the admin is doing anything on the database, it seems any activity corrupts the data. It may be the case that this weird behavior I recorded is precisely due to the fact that a database backup operation is happening while I am querying it. This should absolutely not be the case. Background operations should either: A. Block everyone from accessing the database, or B. Not impact things at all. Note that this may or may not explain the previous times we’ve seen this happen in the past and were perplexed about what happened. -
March 12, 2020 meeting update:
- Any single annotation permission viewed through the Histomics UI causes its elements to disappear.
- The bug is replicable through the DSA interface (as well as HistomicsUI where it was first discovered)
- Sending a JSON should be in the BODY of the request, as opposed to being encoded in the URL string itself (see comment from David M below).
Some discussion can also be found here: https://github.com/DigitalSlideArchive/HistomicsTK/pull/821
Issue Analytics
- State:
- Created 4 years ago
- Comments:6 (3 by maintainers)
This should be fixed by https://github.com/girder/large_image/pull/428
@manthey OK I checked and the PR does fix this bug. Fantastic!