TFX StatisticsGen failing to decode image data

I’ve been trying to adapt the TFX Components.ipynb to work with image data, but when I run the StatisticsGen I get the following warning for each example:

WARNING:root:Feature "image/raw" has bytes value "None" which cannot be decoded as a UTF-8 string. This is causing me two problems.

1. It floods my Colab cell output causing the notebook to become unresponsive.
2. I guess it’s not providing accurate statistics for the image data in my TFExamples.

After some digging, I found where the warning was coming from _get_unicode_value() in tensorflow/data-validation.

I thought maybe I was encoding the image data incorrectly so I tried an existing cifar-10 TFRecord from the tfx repo but encountered the same issue.

To reproduce: I’ve added the heading ☹ ☹ ☹ ☹ WARNINGS OCCUR HERE ☹ ☹ ☹ ☹ above the offending cell.

[embr]

Hi Folks,

The warnings you are seeing indicate that StatisticsGen is trying to treat your raw image features like a categorical string feature. The image bytes are being decoded just fine. The issue is that when the stats (including top K examples) are being written, the output proto is expecting a UTF-8 valid string, but instead gets the raw image bytes. Nothing is wrong with your setups from what I can tell, but this is just an unintended side-effect of a well-intentioned warning in the event that you have a categorical string feature which can’t be serialized. We’ll look into finding a better default that handles image data more elegantly.

In the meantime, to tell StatisticsGen that this feature is really an opaque blob, you can pass in a user-modified schema as described in the StatsGen docs. To generate this schema, you can run StatisticsGen and SchemaGen once (on a sample of data) and then modify the inferred schema to annotate that image features. Here is a modified version of the colab from @tall-josh:

The additional steps are a bit verbose, but having a curated schema is often a good practice for other reasons. Here is the cell that I added to the notebook:

from google.protobuf import text_format
from tensorflow.python.lib.io import file_io
from tensorflow_metadata.proto.v0 import schema_pb2

# Load autogenerated schema (using stats from small batch)

schema = tfx.utils.io_utils.SchemaReader().read(
    tfx.utils.io_utils.get_only_uri_in_dir(
        tfx.types.artifact_utils.get_single_uri(schema_gen.outputs['schema'].get())))

# Modify schema to indicate which string features are images.
# Ideally you would persist a golden version of this schema somewhere rather
# than regenerating it on every run.
for feature in schema.feature:
  if feature.name == 'image/raw':
    feature.image_domain.SetInParent()

# Write modified schema to local file
user_schema_dir ='/tmp/user-schema/'
tfx.utils.io_utils.write_pbtxt_file(
    os.path.join(user_schema_dir, 'schema.pbtxt'), schema)

# Create ImportNode to make modified schema available to other components
user_schema_importer = tfx.components.ImporterNode(
    instance_name='import_user_schema',
    source_uri=user_schema_dir,
    artifact_type=tfx.types.standard_artifacts.Schema)

# Run the user schema ImportNode
context.run(user_schema_importer

Hopefully you find this workaround is useful. In the meantime, we’ll take a look at a better default experience for image-valued features.

[nikelite]

Thank you for reaching out to us. This issue is reported to the TFX team, and we will take a look.