Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Help using InfraValidator locally on OS X

See original GitHub issue

We are trying to use InfraValidator, running locally on OS X (mac machine specs, etc.) with BeamDagRunner, like so:

    infra_validator = InfraValidator(
        model=trainer.outputs["model"],
        examples=example_gen.outputs["examples"],
        serving_spec=infra_validator_pb2.ServingSpec(
            tensorflow_serving=infra_validator_pb2.TensorFlowServing(tags=["latest"]),
            local_docker=infra_validator_pb2.LocalDockerConfig(),
        ),
        request_spec=infra_validator_pb2.RequestSpec(
            tensorflow_serving=infra_validator_pb2.TensorFlowServingRequestSpec()
        ),
    )
    components.append(infra_validator)

The docker container seems to start, but then stops, fails, and gives the following error log:

INFO:absl:Starting infra validation (attempt 1/5).
INFO:absl:Starting LocalDockerRunner(image: tensorflow/serving:latest).
INFO:absl:Running container with parameter {'auto_remove': True, 'detach': True, 'publish_all_ports': True, 'image': 'tensorflow/serving:latest', 'environment': {'MODEL_NAME': 'infra-validation-model', 'MODEL_BASE_PATH': '/model'}, 'mounts': [{'Target': '/model/infra-validation-model/1', 'Source': '/var/folders/l4/pwtkmm_11xd9lnrq5t0yghdh0000gn/T/beam_testt6n8xs2i/tfx_pipeline_output/autotagging-de-tfx-full-run/.temp/6/infra-validation-model/1606931336', 'Type': 'bind', 'ReadOnly': True}]}
INFO:absl:Stopping LocalDockerRunner(image: tensorflow/serving:latest).
ERROR:absl:Infra validation (attempt 1/5) failed.
Traceback (most recent call last):
  File "/Users/adu27/.local/share/virtualenvs/training-vsRZzzSA/lib/python3.6/site-packages/docker/api/client.py", line 268, in _raise_for_status
    response.raise_for_status()
  File "/Users/adu27/.local/share/virtualenvs/training-vsRZzzSA/lib/python3.6/site-packages/requests/models.py", line 943, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: http+docker://localhost/v1.40/containers/create

We assumed that maybe the LocalDockerConfig needed more set up, so we tried to set client_base_url, but couldn’t really figure out what this was supposed to be. We’ve tried client_base_url as unix://var/run/docker.sock, unix:///var/run/docker.sock (note three /s), and http://host.docker.internal:8500.

Any guidance on what client_base_url should be / anything else we should be doing?