Creating workflows in sandbox projects return 500

I’ve set up Flyte sandbox using the instructions in the documentation. The set up works for the default projects flytesnacks and flytetester: registering the workflow and running them works fine.

I then create a new project

flyte-cli register-project --identifier flyteproj -n flyteporj -d "A cool ML proj to test Flyte" -h '<IP>:30081' -i

which works. I then register a workflow:

docker run --network host -e FLYTE_PLATFORM_URL='127.0.0.1:30081' myimage:<git hash> pyflyte -p flyteproj -d development -c sandbox.config register workflows

which also works.

However, when I launch the newly created workflow I only get a 500 error in the “Launch workflow” window.

The flyteadmin log looks like this:

{"json":{"src":"queues.go:76"},"level":"debug","msg":"refreshing execution queues","ts":"2020-01-24T11:24:20Z"}
{"json":{"src":"queues.go:116"},"level":"debug","msg":"refreshing workflow configs","ts":"2020-01-24T11:24:20Z"}
{"json":{"src":"queues.go:221"},"level":"debug","msg":"Evaluating execution queue for [{ResourceType:WORKFLOW Project:flyteproj Domain:development Name:single_step.edges.EdgeDetectorWf Version:772b42066c8a738ba3a06f165d131f3589a1533b XXX_NoUnkeyedLiteral:{} XXX_unrecognized:[] XXX_sizecache:0}] with available queues [[]] and available workflow configs [[]]","ts":"2020-01-24T11:24:20Z"}
{"json":{"src":"namespace_config_provider.go:34"},"level":"warning","msg":"Unsupported value for namespace_mapping in config, defaulting to \u003cproject\u003e-\u003cdomain\u003e","ts":"2020-01-24T11:24:20Z"}
{"json":{"src":"propeller_executor.go:130"},"level":"debug","msg":"failed to create workflow [resource_type:WORKFLOW project:\"flyteproj\" domain:\"development\" name:\"single_step.edges.EdgeDetectorWf\" version:\"772b42066c8a738ba3a06f165d131f3589a1533b\" [ in propeller namespaces \"flyteproj-development\" not found","ts":"2020-01-24T11:24:20Z"}
{"json":{"src":"execution_manager.go:259"},"level":"info","msg":"Failed to execute workflow {Project:flyteproj Domain:development Name: Spec:launch_plan:\u003cresource_type:LAUNCH_PLAN project:\"flyteproj\" domain:\"development\" name:\"single_step.edges.EdgeDetectorWf\" version:\"772b42066c8a738ba3a06f165d131f3589a1533b\" \u003e inputs:\u003cliterals:\u003ckey:\"image_input\" value:\u003cscalar:\u003cprimitive:\u003cstring_value:\"tttt\" \u003e \u003e \u003e \u003e \u003e metadata:\u003cprincipal:\"flyteconsole\" \u003e  Inputs:literals:\u003ckey:\"image_input\" value:\u003cscalar:\u003cprimitive:\u003cstring_value:\"tttt\" \u003e \u003e \u003e \u003e  XXX_NoUnkeyedLiteral:{} XXX_unrecognized:[] XXX_sizecache:0} with execution id {Project:flyteproj Domain:development Name:rizu575do9 XXX_NoUnkeyedLiteral:{} XXX_unrecognized:[] XXX_sizecache:0} and inputs literals:\u003ckey:\"image_input\" value:\u003cscalar:\u003cprimitive:\u003cstring_value:\"tttt\" \u003e \u003e \u003e \u003e  with err failed to create workflow in propeller namespaces \"flyteproj-development\" not found","ts":"2020-01-24T11:24:20Z"}

Am I missing something?

Edit: Running on Minikube.

I noticed that syncresources is not running periodically. Shouldn’t it?

[kumare3]

cc @mayitbeegh / @katrogan lets root cause this one and close it out? @katrogan can we just convert the cluster resource controller to be a golang daemon, instead of a cron job?

[EngHabu]

Fixed in https://github.com/flyteorg/flyte/releases/tag/v0.16.1