"ERROR: Failed to fetch schemas allowed for csv upload in this database!" when trying to upload CSV

“ERROR: Failed to fetch schemas allowed for csv upload in this database!” when trying to upload CSV

Expected results

No error should happen when uploading CSV to the default docker-compose database.

Actual results

I receive an error stating ERROR: Failed to fetch schemas allowed for csv upload in this database! Please contact your Superset Admin!

Terminal error:

192.168.165.1 - - [14/Jul/2021:19:51:12 +0000] "GET /csvtodatabaseview/form HTTP/1.1" 200 27340 "http://192.168.165.128:8088/databaseview/list/?pageIndex=0&sortColumn=changed_on_delta_humanized&sortOrder=desc" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:89.0) Gecko/20100101 Firefox/89.0"
superset_app            | can only concatenate str (not "list") to str
superset_app            | Traceback (most recent call last):
superset_app            |   File "/app/superset/views/core.py", line 3121, in schemas_access_for_csv_upload
superset_app            |     schemas_allowed = database.get_schema_access_for_csv_upload()
superset_app            |   File "/app/superset/models/core.py", line 675, in get_schema_access_for_csv_upload
superset_app            |     allowed_databases += extra_allowed_databases
superset_app            | TypeError: can only concatenate str (not "list") to str
superset_app            | 2021-07-14 19:51:12,677:ERROR:superset.views.core:can only concatenate str (not "list") to str
superset_app            | Traceback (most recent call last):
superset_app            |   File "/app/superset/views/core.py", line 3121, in schemas_access_for_csv_upload
superset_app            |     schemas_allowed = database.get_schema_access_for_csv_upload()
superset_app            |   File "/app/superset/models/core.py", line 675, in get_schema_access_for_csv_upload
superset_app            |     allowed_databases += extra_allowed_databases
superset_app            | TypeError: can only concatenate str (not "list") to str

Screenshots

How to reproduce the bug

1. Enable data upload for the default database
2. Go to http://IP:8088/csvtodatabaseview/form

Environment

(please complete the following information):

• superset version: superset version
• python version: python --version
• node.js version: node -v

See https://github.com/apache/superset/blob/070f0b6cb2a1c3385bc2a7ed2448aa7eb1251666/docker-compose-non-dev.yml for the used versions since I ran this with docker-compose

Checklist

Make sure to follow these steps before submitting your issue - thank you!

• I have checked the superset logs for python stacktraces and included it here as text if there are any.
• I have reproduced the issue with at least the latest released version of superset.
• I have checked the issue tracker for the same issue and I haven’t found one similar.

[myheartsgoon]

I also encounter the same problem when using latest 1.2.0. Below is my setting:

Below is the error in the superset logs when opening the “Upload CSV” page:

I checked the put request that sent to the backend for enabling csv upload: https://{superset_url}/api/v1/database/13 In the request payload, the parameter “schemas_allowed_for_csv_upload” in extra was incorrectly set. It will cause schemas_allowed_for_csv_upload gets json parsed into string type instead of list type (source code in superset.models.core.Database.get_schema_access_for_csv_upload function).

extra in the request payload in my case: "{\"metadata_params\":{},\"engine_params\":{},\"schemas_allowed_for_csv_upload\":\"[\\\"superset_upload\\\"]\"}"

and after json parsing :

>>> json.loads("{\"metadata_params\":{},\"engine_params\":{},\"schemas_allowed_for_csv_upload\":\"[\\\"superset_upload\\\"]\"}") {'metadata_params': {}, 'engine_params': {}, 'schemas_allowed_for_csv_upload': '["superset_upload"]'}

You can see schemas_allowed_for_csv_upload became a string instead of list. And it will raise the error TypeError: can only concatenate str (not "list") to str

    def get_schema_access_for_csv_upload(  # pylint: disable=invalid-name
        self,
    ) -> List[str]:
        allowed_databases = self.get_extra().get("schemas_allowed_for_csv_upload", [])
        if hasattr(g, "user"):
            extra_allowed_databases = config["ALLOWED_USER_CSV_SCHEMA_FUNC"](
                self, g.user
            )
            allowed_databases += extra_allowed_databases
        return sorted(set(allowed_databases))

[myheartsgoon]

Currently I fixed this issue temporarily by adding below code to the funtion get_schema_access_for_csv_upload on backend and rebuilt the image.

from ast import literal_eval
allowed_databases = literal_eval(allowed_databases)

full code of function get_schema_access_for_csv_upload now.

    def get_schema_access_for_csv_upload(  # pylint: disable=invalid-name
        self,
    ) -> List[str]:
        allowed_databases = self.get_extra().get("schemas_allowed_for_csv_upload", [])
        from ast import literal_eval
        allowed_databases = literal_eval(allowed_databases)
        if hasattr(g, "user"):
            extra_allowed_databases = config["ALLOWED_USER_CSV_SCHEMA_FUNC"](
                self, g.user
            )
            allowed_databases += extra_allowed_databases
        return sorted(set(allowed_databases))