[BUG] Unable to convert COCO to TensorFlow dataset

Calling .to_tensorflow() on any of the COCO datasets yields TypeError. It happens for both TensorFlow 1.x and TensorFlow 2.x.

The problematic code:

import hub
ds = hub.Dataset('activeloop/coco_train')
ds= ds.to_tensorflow()

Error message:

TypeError                                 Traceback (most recent call last)
<ipython-input-6-6674a34504fe> in <module>()
----> 1 gen = ds.to_tensorflow()

6 frames
/usr/local/lib/python3.6/dist-packages/hub/api/dataset.py in to_tensorflow(self, indexes)
    651 
    652         return tf.data.Dataset.from_generator(
--> 653             tf_gen, output_types=output_types, output_shapes=output_shapes
    654         )
    655 

/usr/local/lib/python3.6/dist-packages/tensorflow/python/util/deprecation.py in new_func(*args, **kwargs)
    536                 'in a future version' if date is None else ('after %s' % date),
    537                 instructions)
--> 538       return func(*args, **kwargs)
    539 
    540     doc = _add_deprecated_arg_notice_to_docstring(

/usr/local/lib/python3.6/dist-packages/tensorflow/python/data/ops/dataset_ops.py in from_generator(generator, output_types, output_shapes, args, output_signature)
    830         output_shapes = nest.map_structure_up_to(output_types,
    831                                                  tensor_shape.as_shape,
--> 832                                                  output_shapes)
    833       output_signature = nest.map_structure_up_to(output_types,
    834                                                   tensor_spec.TensorSpec,

/usr/local/lib/python3.6/dist-packages/tensorflow/python/data/util/nest.py in map_structure_up_to(shallow_tree, func, *inputs)
    462     raise ValueError("Cannot map over no sequences")
    463   for input_tree in inputs:
--> 464     assert_shallow_structure(shallow_tree, input_tree)
    465 
    466   # Flatten each input separately, apply the function to corresponding elements,

/usr/local/lib/python3.6/dist-packages/tensorflow/python/data/util/nest.py in assert_shallow_structure(shallow_tree, input_tree, check_types)
    322     for shallow_branch, input_branch in zip(shallow_tree, input_tree):
    323       assert_shallow_structure(shallow_branch, input_branch,
--> 324                                check_types=check_types)
    325 
    326 

/usr/local/lib/python3.6/dist-packages/tensorflow/python/data/util/nest.py in assert_shallow_structure(shallow_tree, input_tree, check_types)
    322     for shallow_branch, input_branch in zip(shallow_tree, input_tree):
    323       assert_shallow_structure(shallow_branch, input_branch,
--> 324                                check_types=check_types)
    325 
    326 

/usr/local/lib/python3.6/dist-packages/tensorflow/python/data/util/nest.py in assert_shallow_structure(shallow_tree, input_tree, check_types)
    303           "The two structures don't have the same sequence type. Input "
    304           "structure has type %s, while shallow structure has type %s."
--> 305           % (type(input_tree), type(shallow_tree)))
    306 
    307     if len(input_tree) != len(shallow_tree):

TypeError: The two structures don't have the same sequence type. Input structure has type <class 'tuple'>, while shallow structure has type <class 'dict'>.

Here is a GoogleColab notebook that reproduces the error: https://colab.research.google.com/drive/1tnlnqivwfgfL38U1hrCMJEDYLfhXA96-?usp=sharing

[haiyangdeperci]

@oKatanaaa the dataset without the sequence is ready for you. Just run:

ds = hub.Dataset('hydp/coco_train_workaround')
ds = ds.to_tensorflow()

[mynameisvinn]

@haiyangdeperci yes, the dataset technical debt is accruing 😦