Building a debian package with Python 3.7 deps

Describe the bug:

Hi! This is more a request to figure out what is the best procedure to build Hue from source, and then package it (in my case, in a deb package). From the documentation I see that make apps is needed to populate the build directory, and I though that the Debian package needed only to copy that directory on the target system, but then I realized that the procedure is more complex.

I also tried with PREFIX=/some/path PYTHON_VER=python3.7 make install and ended up in:

  File "/home/vagrant/hue-release-4.7.1/desktop/core/ext-py/MySQL-python-1.2.5/setup_posix.py", line 2, in <module>
    from ConfigParser import SafeConfigParser
ModuleNotFoundError: No module named 'ConfigParser'

I checked the ext-py directories and some of them seem not ready for Python3, so I am wondering if I am doing the right steps.

If I follow https://github.com/cloudera/hue#building-from-source everything works fine (so the dev server comes up without any issue).

Steps to reproduce it?

Hue source version 4.7.1, then:

PYTHON_VER=python3.7 make apps
PYTHON_VER=python3.7 PREFIX=/some/local/path make install

Followed https://docs.gethue.com/administrator/installation/install/

Ideally what I want to achieve is something similar to the debian package that Cloudera releases with CDH, but built for python 3.7 and some other dependencies (like Mariadb dev instead of Mysql, etc…).

Any help would be really appreciated 😃

[elukey]

@romainr thanks a lot for the suggestion, sorry for the lag I was afk 😃

Very interesting: I tried PYTHONPATH=/usr/lib/hue/desktop/libs/:/usr/lib/hue/desktop/core/src/:/usr/lib/hue/build/env/lib/python3.7/site-packages/ /usr/lib/hue/build/env/bin/python3 /usr/lib/hue/build/env/bin/hue rungunicornserver and it works fine, but the same PYTHONPATH with /usr/lib/hue/desktop/core/src/desktop/management/commands/rungunicornserver.py leads to the Config error.

Is there any special PYTHONPATH or anything else to pass to rungunicornserver.py to make it work properly?

[elukey]

Also this happens when I try to submit a hive query:

[10/Sep/2020 12:30:37 +0000] decorators   ERROR    Error running execute
Traceback (most recent call last):
  File "/usr/lib/hue/apps/beeswax/src/beeswax/server/hive_server2_lib.py", line 299, in fields
    return [HiveServerTColumnValue2(field).val.pop(0) for field in self.cols]
  File "/usr/lib/hue/apps/beeswax/src/beeswax/server/hive_server2_lib.py", line 299, in <listcomp>
    return [HiveServerTColumnValue2(field).val.pop(0) for field in self.cols]
IndexError: pop from empty list

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/hue/apps/beeswax/src/beeswax/server/hive_server2_lib.py", line 395, in rows
    yield row.fields()
  File "/usr/lib/hue/apps/beeswax/src/beeswax/server/hive_server2_lib.py", line 301, in fields
    raise StopIteration
StopIteration

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib/hue/desktop/libs/notebook/src/notebook/decorators.py", line 114, in wrapper
    return f(*args, **kwargs)
  File "/usr/lib/hue/desktop/libs/notebook/src/notebook/api.py", line 225, in execute
    response = _execute_notebook(request, notebook, snippet)
  File "/usr/lib/hue/desktop/libs/notebook/src/notebook/api.py", line 153, in _execute_notebook
    response['handle'] = interpreter.execute(notebook, snippet)
  File "/usr/lib/hue/desktop/libs/notebook/src/notebook/connectors/hiveserver2.py", line 98, in decorator
    return func(*args, **kwargs)
  File "/usr/lib/hue/desktop/libs/notebook/src/notebook/connectors/hiveserver2.py", line 301, in execute
    result = db.use(query.database, session=_session)
  File "/usr/lib/hue/apps/beeswax/src/beeswax/server/dbms.py", line 895, in use
    return self.client.use(query, session=session)
  File "/usr/lib/hue/apps/beeswax/src/beeswax/server/hive_server2_lib.py", line 1301, in use
    data = self._client.execute_query(query, session=session)
  File "/usr/lib/hue/apps/beeswax/src/beeswax/server/hive_server2_lib.py", line 937, in execute_query
    return self.execute_query_statement(statement=query.query['query'], max_rows=max_rows, configuration=configuration, session=session)
  File "/usr/lib/hue/apps/beeswax/src/beeswax/server/hive_server2_lib.py", line 944, in execute_query_statement
    results, schema, operation_handle, session = self.execute_statement(statement=statement, max_rows=max_rows, configuration=configuration, orientation=orientation, session=session)
  File "/usr/lib/hue/apps/beeswax/src/beeswax/server/hive_server2_lib.py", line 981, in execute_statement
    (res, session) = self.call(self._client.ExecuteStatement, req, session=session)
  File "/usr/lib/hue/apps/beeswax/src/beeswax/server/hive_server2_lib.py", line 717, in call
    return self.call_return_result_and_session(fn, req, status, session=session)
  File "/usr/lib/hue/apps/beeswax/src/beeswax/server/hive_server2_lib.py", line 754, in call_return_result_and_session
    session = self.open_session(self.user)
  File "/usr/lib/hue/apps/beeswax/src/beeswax/server/hive_server2_lib.py", line 708, in open_session
    configuration = self.get_configuration(session=session)
  File "/usr/lib/hue/apps/beeswax/src/beeswax/server/hive_server2_lib.py", line 1162, in get_configuration
    properties = [(row[0].split('=')[0], row[0].split('=')[1]) for row in results.rows() if '=' in row[0]]
  File "/usr/lib/hue/apps/beeswax/src/beeswax/server/hive_server2_lib.py", line 1162, in <listcomp>
    properties = [(row[0].split('=')[0], row[0].split('=')[1]) for row in results.rows() if '=' in row[0]]
RuntimeError: generator raised StopIteration