RobotFramework LSP for Pycharm is not taking into account Pythonpath settings during libspec creation step

Description of the defect I’m using RoboFramework LSP for Pycharm and it’s failing during the libspec generation for my custom libraries. I have a directory structure as follows:

    |-tests
    |-resources
    |    |-my_first_module
    |    |    |-FirstRobotCustomLib.py
    |    |    |-SecondRobotCustomLib.py
    |    |    |-...
    |    |-my_second_module
    |    |    |-ThirdRobotCustomLib.py
    |    |    |-...
    |    |...
    |...

Inside the “resources” directory i have all the python modules that conform a set of customized robotframework libraries. As I don’t have these modules instaled, but they’re directly imported in the robot tests, I have update the Pythonpath setting for robotframework-lsp plugin as follows:

But, looking at the errors from robot_err log file it seems that this path is not applied:

lsp: 2021-04-16 17:02:43 UTC pid: 32689 - ThreadPoolExecutor-1_0 - EXCEPTION - robotframework_ls.impl.libspec_manager
Error creating libspec: AssetsApi.
Return code: 252
Output:
Importing test library 'AssetsApi' failed: ModuleNotFoundError: No module named 'resources'
Traceback (most recent call last):
  File "/home/albertm/workspace/automated-tests-firedrill/resources/objects_layer/api/assets/AssetsApi.py", line 6, in <module>
    from resources.objects_layer.api.AiqApi import AiqApi
PYTHONPATH:
  /home/albertm/workspace/automated-tests-firedrill/resources/objects_layer/api/assets
  /home/albertm/workspace/automated-tests-firedrill/resources/objects_layer/api/assets
  /usr/lib/python38.zip
  /usr/lib/python3.8
  /usr/lib/python3.8/lib-dynload
  /home/albertm/workspace/automated-tests-firedrill/venv/lib/python3.8/site-packages

Try --help for usage information.

To Reproduce Steps to reproduce the behavior:

1. Open Pycharm
2. Go to File -> Settings -> Languages & Frameworks -> Robot Framework (Project)
3. Add the path to python executable for the used venv within the “Language Server Python” field.
4. Add the path to the root python custom library module within the “Pythonpath” field usin with JSON array format.
5. Apply the configuration and restart the IDE.
6. Analyze the error logs from lsp/robot_err_<timestamp>.log

Expected Behavior During the libspec generation for the custom robot libraries, the directory added in the “Pythonpath” setting should be added to the Pythonpath in order to be able to find Python dependencies that are not installed as actual libraries in the Python environment.

Actual Behavior During the libspec generation the robot.libdoc process fails because no custom dependences are found in the Pythonpath. I’ve tried to execute the process manually adding the same directory configured in the “Pythonpath” settings for robotframework-lsp plugin using the robot.libdoc -P argument and its works correctly, so it seems that settings are not being correctly applied.

Versions:

• OS: Ubuntu 18.04
• Robot Framework 3.2.2
• Robot Framework Language Server 0.14.0
• Client PyCharm 2021.1 Professional Edition

Logs Full log file: robot_err_20210416.log

[himperion92]

@fabioz After solving all the constructors, it works like a charm.

Thank you for helping me through these errors!

[fabioz]

Humm, I’m taking a look at the logs here… it shows:

Generating libspec for: NetworkAssessmentResultsSummary.
Cwd:/home/albertm/workspace/automated-tests-firedrill/resources/./objects_layer/page_objects/assessments
Command line:
/home/albertm/workspace/automated-tests-firedrill/venv/bin/python3.8 -m robot.libdoc --format XML:HTML -P /home/albertm/workspace/automated-tests-firedrill/resources/./objects_layer/page_objects/assessments -P /home/albertm/workspace/automated-tests-firedrill NetworkAssessmentResultsSummary /home/albertm/.robotframework-ls/specs/v2/b420254f_3.2.2/user/e06a6213.libspec

lsp: 2021-04-19 13:27:34 UTC pid: 11612 - ThreadPoolExecutor-1_0 - EXCEPTION - robotframework_ls.impl.libspec_manager
Error creating libspec: NetworkAssessmentResultsSummary.
Return code: 252
Output:
Importing test library 'NetworkAssessmentResultsSummary' failed: RobotNotRunningError: Cannot access execution context
Traceback (most recent call last):
  File "/home/albertm/workspace/automated-tests-firedrill/resources/objects_layer/page_objects/assessments/NetworkAssessmentResultsSummary.py", line 8, in <module>
    class NetworkAssessmentResultsSummary(BasicPageObject):
  File "/home/albertm/workspace/automated-tests-firedrill/resources/objects_layer/page_objects/BasicPageObject.py", line 21, in decorate
    setattr(cls, attr, decorator(cls(), getattr(cls, attr)))
  File "/home/albertm/workspace/automated-tests-firedrill/resources/objects_layer/page_objects/BasicPageObject.py", line 34, in __init__
    self._webdriver = BuiltIn().get_library_instance('CustomExtendedSeleniumLibrary')
  File "/home/albertm/workspace/automated-tests-firedrill/venv/lib/python3.8/site-packages/robot/libraries/BuiltIn.py", line 3291, in get_library_instance
    return self._namespace.get_library_instance(name)
  File "/home/albertm/workspace/automated-tests-firedrill/venv/lib/python3.8/site-packages/robot/libraries/BuiltIn.py", line 73, in _namespace
    return self._get_context().namespace
  File "/home/albertm/workspace/automated-tests-firedrill/venv/lib/python3.8/site-packages/robot/libraries/BuiltIn.py", line 68, in _get_context
    raise RobotNotRunningError('Cannot access execution context')
PYTHONPATH:
  /home/albertm/workspace/automated-tests-firedrill/resources/objects_layer/page_objects/assessments
  /home/albertm/workspace/automated-tests-firedrill
  /home/albertm/workspace/automated-tests-firedrill/resources/objects_layer/page_objects/assessments
  /usr/lib/python38.zip
  /usr/lib/python3.8
  /usr/lib/python3.8/lib-dynload
  /home/albertm/workspace/automated-tests-firedrill/venv/lib/python3.8/site-packages

Try --help for usage information.

So, the PYTHONPATH seems to be making it properly, but the BasicPageObject constructor is not ok.

Also, there’s:

Generating libspec for: FdapiClient.
Cwd:/home/albertm/workspace/automated-tests-firedrill/resources/./service_layer/api/fdapi
Command line:
/home/albertm/workspace/automated-tests-firedrill/venv/bin/python3.8 -m robot.libdoc --format XML:HTML -P /home/albertm/workspace/automated-tests-firedrill/resources/./service_layer/api/fdapi -P /home/albertm/workspace/automated-tests-firedrill FdapiClient /home/albertm/.robotframework-ls/specs/v2/b420254f_3.2.2/user/411a5df0.libspec

lsp: 2021-04-19 13:27:35 UTC pid: 11612 - ThreadPoolExecutor-1_0 - EXCEPTION - robotframework_ls.impl.libspec_manager
Error creating libspec: FdapiClient.
Return code: 252
Output:
Initializing test library 'FdapiClient' with no arguments failed: AttributeError: 'NoneType' object has no attribute 'lower'
Traceback (most recent call last):
  File "/home/albertm/workspace/automated-tests-firedrill/resources/service_layer/api/fdapi/FdapiClient.py", line 24, in __init__
    self._username = os.getenv('DEFAULT_LOGIN').lower()

Where the default constructor is also not ok…

So, in this case, what needs to be done is fix those so that the libspec can be properly generated for them (reference: https://github.com/robocorp/robotframework-lsp/blob/master/robotframework-ls/docs/faq.md#how-to-proceed-if-keywords-from-a-library-are-not-being-resolved).

One thing that just occurred to me is that the language server caches that a library wasn’t generated and won’t try to recreate it until it’s restarted (due to performance considerations as attempting to generate a libspec can be slow), so, you have to fix the library and then use the Reload Window command so that the language server is restarted to retest…

You can inspect the robotframework_ls.log.lint.lsp.<pid>.api.log to check if there are further related errors.

As the PYTHONPATH is being properly set and the errors seem to be all on the library, I’m closing the issue, since I don’t think anything needs to be done in the language server.

I created https://github.com/robocorp/robotframework-lsp/issues/324 so that in the future the language server makes the error more apparent.