bug: Telemetry Invalid Context JSON

I’m seeing 3 events this morning that are causing failures in the Snowflake staging of the telemetry data because theres invalid json in the contexts column. I was able to narrow it down to this snippet of the contexts field. It almost looks like the user ran something like meltano invoke dbt-snowflake:run\ with the backslash in their plugin command, and we just passed it through. So it ends up acting as an escape character for the context json thats generated.

{
    "schema": "iglu:com.snowplowanalytics.snowplow/contexts/jsonschema/1-0-1",
    "data": [
        {
            "schema": "iglu:com.meltano/plugins_context/jsonschema/1-0-0",
            "data": {
                "context_uuid": "c03c812e-d912-4996-aad8-ee6367f6de90",
                "plugins": [
                    {
                        "category": "transformers",
                        "name_hash": "5ce17935e3051fb03c316f7a01e66fad78c8f3d799c12673f39d5cece817df26",
                        "namespace_hash": "33e3de565605b33042aa3ff5953cfba66bc7c5212d8caf0775acb9523fcb84da",
                        "executable_hash": "de371adefe4de7fe43b17dd1d87d8d47eab2f506eb5dfed54560638e894018f7",
                        "variant_name_hash": "c39e17256a61578d1c3bb160f9c73ae64f81e9f9541f9874670e840c6d9db66d",
                        "pip_url_hash": "96a16e0c3fff2ed8facc9a4d89a5364a8704afcd8a853d53a67e9ab60606ab45",
                        "parent_name_hash": "5ce17935e3051fb03c316f7a01e66fad78c8f3d799c12673f39d5cece817df26",
                        "command": "run\"
                    }
                ]
            }
        }

I’m not sure exactly if this is a meltano bug, a bug with snowplow (allowing invalid json through), or something I should handle in Snowflake. I could definitely handle it in Snowflake but I would expect that our pipeline of schema validation wouldnt allow invalid json to get through so wanted to flag it.

cc @WillDaSilva @pandemicsyn @tayloramurphy

[pnadolny13]

@WillDaSilva I appreciate you looking into this. I agree that its safe to ignore it for now if we can’t reproduce it. It was only a single event in our entire data set so who knows what happened. I’ll keep you posted if I see it again.

[pnadolny13]

@WillDaSilva I’m also not able to re-produce it locally but I did notice that it was on Windows so maybe that has something to do with it. Here the full context object I get with the message. This is also from Snowflake. Snowplow/Snowcat Cloud drops the data into our S3 bucket as TSV files, then we COPY it into Snowflake, I cant see how that would make a difference but wanted to put it all on the table.

{
    "schema": "iglu:com.snowplowanalytics.snowplow/contexts/jsonschema/1-0-1",
    "data": [
        {
            "schema": "iglu:com.meltano/environment_context/jsonschema/1-0-0",
            "data": {
                "context_uuid": "2208ce40-9893-41ab-9fb6-0f0b1b4c890f",
                "meltano_version": "2.3.0",
                "is_dev_build": true,
                "is_ci_environment": false,
                "python_version": "3.8.10",
                "python_implementation": "CPython",
                "system_name": "Windows",
                "system_release": "10",
                "system_version": "10.0.19044",
                "machine": "AMD64",
                "windows_edition": "Professional",
                "freedesktop_id": null,
                "freedesktop_id_like": null,
                "freedesktop_version_id": null,
                "num_cpu_cores": 8,
                "num_cpu_cores_available": 8,
                "process_hierarchy": [
                    {
                        "process_name_hash": "f3b3ff1fb4ece1584b3065be7380721b0ac6ad459696f997c24581b1693fcfe3",
                        "process_creation_timestamp": "2022-08-02T08:32:37.151024Z"
                    },
                    {
                        "process_name_hash": "731a2bf17e147ce99e846f7fe0c196c5c66603af99e409551e73ed9c2f7d4f5b",
                        "process_creation_timestamp": "2022-08-02T08:32:37.095738Z"
                    },
                    {
                        "process_name_hash": "f307e73a3447b10444e67233700c1aa5ca8e3673dd2f292084bf485583caad6f",
                        "process_creation_timestamp": "2022-08-02T08:23:14.996283Z"
                    },
                    {
                        "process_name_hash": "8860d259e8c6688fffad4a3b928fda578bb42e7034d09a8bf1410d4ccf7993e3",
                        "process_creation_timestamp": "2022-08-02T08:23:09.231678Z"
                    },
                    {
                        "process_name_hash": "8860d259e8c6688fffad4a3b928fda578bb42e7034d09a8bf1410d4ccf7993e3",
                        "process_creation_timestamp": "2022-08-02T08:23:08.698266Z"
                    },
                    {
                        "process_name_hash": "8860d259e8c6688fffad4a3b928fda578bb42e7034d09a8bf1410d4ccf7993e3",
                        "process_creation_timestamp": "2022-08-02T08:23:06.581065Z"
                    },
                    {
                        "process_name_hash": "7592a3326e8f8297547f8c170b96b8aa8f5234027fd76593841a6574f098759c",
                        "process_creation_timestamp": "2022-08-02T07:24:11.170278Z"
                    }
                ]
            }
        },
        {
            "schema": "iglu:com.meltano/project_context/jsonschema/1-0-0",
            "data": {
                "context_uuid": "ddcf40b3-f470-4a53-8d71-82bedce80f41",
                "project_uuid": "21d78c2d-56c5-4753-8dfa-02dba2b21298",
                "project_uuid_source": "explicit",
                "client_uuid": "94a247c2-54cc-49b8-a044-6ab5ec018c25",
                "environment_name_hash": "ef260e9aa3c673af240d17a2660480361a8e081d1ffeca2a5ed0e3219fc18567"
            }
        },
        {
            "schema": "iglu:com.meltano/cli_context/jsonschema/1-1-0",
            "data": {
                "command": "cli",
                "parent_command_hint": null,
                "options": {
                    "log_level": null,
                    "log_config": null,
                    "verbose": 0,
                    "environment": null,
                    "no_environment": false
                }
            }
        },
        {
            "schema": "iglu:com.meltano/cli_context/jsonschema/1-1-0",
            "data": {
                "command": "invoke",
                "parent_command_hint": "cli",
                "options": {
                    "plugin_name": "1cd1dfc78caaaa301649f199bc174ad4195c88429fa551d408b83c1f684ca00c",
                    "plugin_args": "2e38e77b22c314a449e91fafed92a43826ac6aa403ae6a8acb6cf58239fbaf5d",
                    "print_var": "2e38e77b22c314a449e91fafed92a43826ac6aa403ae6a8acb6cf58239fbaf5d",
                    "plugin_type": null,
                    "dump": null,
                    "list_commands": false,
                    "containers": false,
                    "database_uri": null
                }
            }
        },
        {
            "schema": "iglu:com.meltano/plugins_context/jsonschema/1-0-0",
            "data": {
                "context_uuid": "c03c812e-d912-4996-aad8-ee6367f6de90",
                "plugins": [
                    {
                        "category": "transformers",
                        "name_hash": "5ce17935e3051fb03c316f7a01e66fad78c8f3d799c12673f39d5cece817df26",
                        "namespace_hash": "33e3de565605b33042aa3ff5953cfba66bc7c5212d8caf0775acb9523fcb84da",
                        "executable_hash": "de371adefe4de7fe43b17dd1d87d8d47eab2f506eb5dfed54560638e894018f7",
                        "variant_name_hash": "c39e17256a61578d1c3bb160f9c73ae64f81e9f9541f9874670e840c6d9db66d",
                        "pip_url_hash": "96a16e0c3fff2ed8facc9a4d89a5364a8704afcd8a853d53a67e9ab60606ab45",
                        "parent_name_hash": "5ce17935e3051fb03c316f7a01e66fad78c8f3d799c12673f39d5cece817df26",
                        "command": "run\"}]}},{"schema":"iglu:com.meltano/exception_context/jsonschema/1-0-0","data":{"context_uuid":"8c3f551b-df98-46b3-bbac-9a57d6255b83","exception":{"type":"UnknownCommandError","str_hash":"953a866ff7c1068b79c9187b64d1c9264b3adabac87e05065f4b4d8e643e4f7b","repr_hash":"2c3b4c6b7445577de4d8fb6ef996209be1eae6c11b7a23222e442014398ebc41","traceback":[{"file":"lib/site-packages/meltano/cli/invoke.py","line_number":112},{"file":"lib/asyncio/runners.py","line_number":44},{"file":"lib/asyncio/base_events.py","line_number":616},{"file":"lib/site-packages/meltano/cli/invoke.py","line_number":150},{"file":"lib/site-packages/meltano/core/plugin_invoker.py","line_number":316}],"cause":{"type":"KeyError","str_hash":"3f9748565383dc355ee7f5c4852d52f97df428919e77247a82a23391f7e0b8bb","repr_hash":"c13906568e917b2e108d788a91c6df84582e1d34ef9752d69af026582e66d948","traceback":[{"file":"lib/site-packages/meltano/core/plugin_invoker.py","line_number":314}],"cause":null,"context":null},"context":{"type":"KeyError","str_hash":"3f9748565383dc355ee7f5c4852d52f97df428919e77247a82a23391f7e0b8bb","repr_hash":"c13906568e917b2e108d788a91c6df84582e1d34ef9752d69af026582e66d948","traceback":[{"file":"lib/site-packages/meltano/core/plugin_invoker.py","line_number":314}],"cause":null,"context":null}}}}]}