Lightrun
question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

self.modules is not picking up full module path

See original GitHub issue

At NERSC we have a couple of paths that we add to MODULEPATH for interactive sessions. When I invoke self.modules for one of these modules it fails because it isn’t in the MODULEPATH:

login01> ../reframe/bin/reframe -C nersc-config.py -c python/ --system=muller -r -R -n test_scipy -p cpe-gnu --system muller:cpu
[2021-04-01T10:23:54-07:00] [ReFrame Setup]
[2021-04-01T10:23:54-07:00]   version:           3.3-nersc (rev: ff858afe)
[2021-04-01T10:23:54-07:00]   command:           '../reframe/bin/reframe -C nersc-config.py -c python/ --system=muller -r -R -n test_scipy -p cpe-gnu --system muller:cpu'
[2021-04-01T10:23:54-07:00]   launched by:       lgerhard@login01
[2021-04-01T10:23:54-07:00]   working directory: '/global/cfs/cdirs/mpccc/lgerhard/nersc/reframe-nersc-tests'
[2021-04-01T10:23:54-07:00]   settings file:     'nersc-config.py'
[2021-04-01T10:23:54-07:00]   check search path: (R) '/global/cfs/cdirs/mpccc/lgerhard/nersc/reframe-nersc-tests/python'
[2021-04-01T10:23:54-07:00]   stage directory:   '/global/cfs/cdirs/reframe/lgerhard/stage'
[2021-04-01T10:23:54-07:00]   output directory:  '/global/cfs/cdirs/reframe/lgerhard/output'
[2021-04-01T10:23:54-07:00] 
[2021-04-01T10:23:54-07:00] [==========] Running 1 check(s)
[2021-04-01T10:23:54-07:00] [==========] Started on Thu Apr  1 10:23:54 2021 
[2021-04-01T10:23:54-07:00] 
[2021-04-01T10:23:54-07:00] [----------] started processing test_scipy (Does scipy work?)
[2021-04-01T10:23:54-07:00] [ RUN      ] test_scipy on muller:cpu using cpe-gnu
[2021-04-01T10:23:55-07:00] [     FAIL ] (1/1) test_scipy on muller:cpu using cpe-gnu [compile: 0.003s run: 0.633s total: 0.643s]
[2021-04-01T10:23:55-07:00] ==> test failed during 'run': test staged in '/global/cfs/cdirs/reframe/lgerhard/stage/muller/cpu/cpe-gnu/test_scipy'
[2021-04-01T10:23:55-07:00] [----------] finished processing test_scipy (Does scipy work?)
[2021-04-01T10:23:55-07:00] 
[2021-04-01T10:23:55-07:00] [----------] waiting for spawned checks to finish
[2021-04-01T10:23:55-07:00] [----------] all spawned checks have finished

[2021-04-01T10:23:55-07:00] [  FAILED  ] Ran 1 test case(s) from 1 check(s) (1 failure(s))
[2021-04-01T10:23:55-07:00] [==========] Finished on Thu Apr  1 10:23:55 2021 
[2021-04-01T10:23:55-07:00] 
[2021-04-01T10:23:55-07:00] ==============================================================================
[2021-04-01T10:23:55-07:00] SUMMARY OF FAILURES
[2021-04-01T10:23:55-07:00] ------------------------------------------------------------------------------
[2021-04-01T10:23:55-07:00] FAILURE INFO for test_scipy 
[2021-04-01T10:23:55-07:00]   * Test Description: Does scipy work?
[2021-04-01T10:23:55-07:00]   * System partition: muller:cpu
[2021-04-01T10:23:55-07:00]   * Environment: cpe-gnu
[2021-04-01T10:23:55-07:00]   * Stage directory: /global/cfs/cdirs/reframe/lgerhard/stage/muller/cpu/cpe-gnu/test_scipy
[2021-04-01T10:23:55-07:00]   * Node list: None
[2021-04-01T10:23:55-07:00]   * Job type: batch job (id=None)
[2021-04-01T10:23:55-07:00]   * Dependencies (conceptual): []
[2021-04-01T10:23:55-07:00]   * Dependencies (actual): []
[2021-04-01T10:23:55-07:00]   * Maintainers: ['lastephey@lbl.gov']
[2021-04-01T10:23:55-07:00]   * Failing phase: run
[2021-04-01T10:23:55-07:00]   * Rerun with '-n test_scipy -p cpe-gnu --system muller:cpu'
[2021-04-01T10:23:55-07:00]   * Reason: environ error: could not execute module operation: command '/usr/share/lmod/lmod/libexec/lmod python show python/3.8-anaconda-2020.11' failed with exit code 1:
--- stdout ---
import os
os.environ["MODULEPATH"] = "/opt/cray/pe/lmod/modulefiles/cncm/gnu/8.0/ofi/1.0/x86-rome/1.0/cray-mpich/8.0:/opt/cray/pe/lmod/modulefiles/mpi/gnu/8.0/ofi/1.0/cray-mpich/8.0:/opt/cray/pe/lmod/modulefiles/perftools/21.02.0:/opt/cray/pe/lmod/modulefiles/comnet/gnu/8.0/ofi/1.0:/opt/cray/pe/lmod/modulefiles/net/ofi/1.0:/opt/cray/pe/lmod/modulefiles/comcpu/gnu/8.0/x86-rome/1.0:/opt/cray/pe/lmod/modulefiles/cpu/x86-rome/1.0:/opt/cray/pe/lmod/modulefiles/compiler/gnu/8.0:/opt/cray/pe/cpe-prgenv/7.0.0/lmod:/opt/cray/pe/lmod/modulefiles/core:/opt/cray/pe/lmod/modulefiles/craype-targets/default:/opt/modulefiles:/opt/cray/modulefiles";
os.environ["_ModuleTable001_"] = "X01vZHVsZVRhYmxlXz17WyJNVHZlcnNpb24iXT0zLFsiY19yZWJ1aWxkVGltZSJdPWZhbHNlLFsiY19zaG9ydFRpbWUiXT1mYWxzZSxkZXB0aFQ9e30sZmFtaWx5PXtbIlByZ0VudiJdPSJjcGUtZ251IixbImNvbXBpbGVyIl09ImdjYyIsWyJjcmF5cGUiXT0iY3JheXBlIixbImNyYXlwZV9jcHUiXT0iY3JheXBlLXg4Ni1yb21lIixbImNyYXlwZV9uZXR3b3JrIl09ImNyYXlwZS1uZXR3b3JrLW9maSIsfSxtVD17WyJjcGUtZ251Il09e1siZm4iXT0iL29wdC9jcmF5L3BlL2NwZS1wcmdlbnYvNy4wLjAvbG1vZC9jcGUtZ251Lmx1YSIsWyJmdWxsTmFtZSJdPSJjcGUtZ251IixbImxvYWRPcmRlciJdPTEscHJvcFQ9e30sWyJzdGFja0RlcHRoIl09MCxbInN0YXR1cyJdPSJhY3Rp";
os.environ["_ModuleTable002_"] = "dmUiLFsidXNlck5hbWUiXT0iY3BlLWdudSIsfSxbImNyYXktZHNtbWwiXT17WyJmbiJdPSIvb3B0L2NyYXkvcGUvbG1vZC9tb2R1bGVmaWxlcy9jb3JlL2NyYXktZHNtbWwvMC4xLjMubHVhIixbImZ1bGxOYW1lIl09ImNyYXktZHNtbWwvMC4xLjMiLFsibG9hZE9yZGVyIl09Nyxwcm9wVD17fSxbInN0YWNrRGVwdGgiXT0wLFsic3RhdHVzIl09ImFjdGl2ZSIsWyJ1c2VyTmFtZSJdPSJjcmF5LWRzbW1sIix9LFsiY3JheS1saWJzY2kiXT17WyJmbiJdPSIvb3B0L2NyYXkvcGUvbG1vZC9tb2R1bGVmaWxlcy9jb3JlL2NyYXktbGlic2NpLzIwLjEyLjEuMi5sdWEiLFsiZnVsbE5hbWUiXT0iY3JheS1saWJzY2kvMjAuMTIuMS4yIixbImxvYWRPcmRlciJdPTExLHByb3BUPXt9LFsi";
os.environ["_ModuleTable003_"] = "c3RhY2tEZXB0aCJdPTAsWyJzdGF0dXMiXT0iYWN0aXZlIixbInVzZXJOYW1lIl09ImNyYXktbGlic2NpIix9LFsiY3JheS1tcGljaCJdPXtbImZuIl09Ii9vcHQvY3JheS9wZS9sbW9kL21vZHVsZWZpbGVzL2NvbW5ldC9nbnUvOC4wL29maS8xLjAvY3JheS1tcGljaC84LjEuMi5sdWEiLFsiZnVsbE5hbWUiXT0iY3JheS1tcGljaC84LjEuMiIsWyJsb2FkT3JkZXIiXT0xMCxwcm9wVD17fSxbInN0YWNrRGVwdGgiXT0wLFsic3RhdHVzIl09ImFjdGl2ZSIsWyJ1c2VyTmFtZSJdPSJjcmF5LW1waWNoIix9LGNyYXlwZT17WyJmbiJdPSIvb3B0L2NyYXkvcGUvbG1vZC9tb2R1bGVmaWxlcy9jb3JlL2NyYXlwZS8yLjcuNS5sdWEiLFsiZnVsbE5hbWUiXT0iY3JheXBlLzIuNy41Iixb";
os.environ["_ModuleTable004_"] = "ImxvYWRPcmRlciJdPTMscHJvcFQ9e30sWyJzdGFja0RlcHRoIl09MCxbInN0YXR1cyJdPSJhY3RpdmUiLFsidXNlck5hbWUiXT0iY3JheXBlIix9LFsiY3JheXBlLW5ldHdvcmstb2ZpIl09e1siZm4iXT0iL29wdC9jcmF5L3BlL2xtb2QvbW9kdWxlZmlsZXMvY3JheXBlLXRhcmdldHMvZGVmYXVsdC9jcmF5cGUtbmV0d29yay1vZmkubHVhIixbImZ1bGxOYW1lIl09ImNyYXlwZS1uZXR3b3JrLW9maSIsWyJsb2FkT3JkZXIiXT02LHByb3BUPXt9LFsic3RhY2tEZXB0aCJdPTAsWyJzdGF0dXMiXT0iYWN0aXZlIixbInVzZXJOYW1lIl09ImNyYXlwZS1uZXR3b3JrLW9maSIsfSxbImNyYXlwZS14ODYtcm9tZSJdPXtbImZuIl09Ii9vcHQvY3JheS9wZS9sbW9kL21vZHVsZWZpbGVz";
os.environ["_ModuleTable005_"] = "L2NyYXlwZS10YXJnZXRzL2RlZmF1bHQvY3JheXBlLXg4Ni1yb21lLmx1YSIsWyJmdWxsTmFtZSJdPSJjcmF5cGUteDg2LXJvbWUiLFsibG9hZE9yZGVyIl09NCxwcm9wVD17fSxbInN0YWNrRGVwdGgiXT0wLFsic3RhdHVzIl09ImFjdGl2ZSIsWyJ1c2VyTmFtZSJdPSJjcmF5cGUteDg2LXJvbWUiLH0sZ2NjPXtbImZuIl09Ii9vcHQvY3JheS9wZS9sbW9kL21vZHVsZWZpbGVzL2NvcmUvZ2NjLzEwLjIuMC5sdWEiLFsiZnVsbE5hbWUiXT0iZ2NjLzEwLjIuMCIsWyJsb2FkT3JkZXIiXT0yLHByb3BUPXt9LFsic3RhY2tEZXB0aCJdPTAsWyJzdGF0dXMiXT0iYWN0aXZlIixbInVzZXJOYW1lIl09ImdjYyIsfSxsaWJmYWJyaWM9e1siZm4iXT0iL29wdC9jcmF5L21vZHVsZWZpbGVz";
os.environ["_ModuleTable006_"] = "L2xpYmZhYnJpYy8xLjExLjAuMC4yMzMiLFsiZnVsbE5hbWUiXT0ibGliZmFicmljLzEuMTEuMC4wLjIzMyIsWyJsb2FkT3JkZXIiXT01LHByb3BUPXt9LFsic3RhY2tEZXB0aCJdPTEsWyJzdGF0dXMiXT0iYWN0aXZlIixbInVzZXJOYW1lIl09ImxpYmZhYnJpYyIsfSxbInBlcmZ0b29scy1iYXNlIl09e1siZm4iXT0iL29wdC9jcmF5L3BlL2xtb2QvbW9kdWxlZmlsZXMvY29yZS9wZXJmdG9vbHMtYmFzZS8yMS4wMi4wLmx1YSIsWyJmdWxsTmFtZSJdPSJwZXJmdG9vbHMtYmFzZS8yMS4wMi4wIixbImxvYWRPcmRlciJdPTgscHJvcFQ9e30sWyJzdGFja0RlcHRoIl09MCxbInN0YXR1cyJdPSJhY3RpdmUiLFsidXNlck5hbWUiXT0icGVyZnRvb2xzLWJhc2UiLH0seHBtZW09e1si";
os.environ["_ModuleTable007_"] = "Zm4iXT0iL29wdC9jcmF5L21vZHVsZWZpbGVzL3hwbWVtLzIuMi4zNS03LjAuMS4wXzEuOV9fZ2Q1MGZhYmYuc2hhc3RhIixbImZ1bGxOYW1lIl09InhwbWVtLzIuMi4zNS03LjAuMS4wXzEuOV9fZ2Q1MGZhYmYuc2hhc3RhIixbImxvYWRPcmRlciJdPTkscHJvcFQ9e30sWyJzdGFja0RlcHRoIl09MCxbInN0YXR1cyJdPSJhY3RpdmUiLFsidXNlck5hbWUiXT0ieHBtZW0iLH0sfSxtcGF0aEE9eyIvb3B0L2NyYXkvcGUvbG1vZC9tb2R1bGVmaWxlcy9jbmNtL2dudS84LjAvb2ZpLzEuMC94ODYtcm9tZS8xLjAvY3JheS1tcGljaC84LjAiLCIvb3B0L2NyYXkvcGUvbG1vZC9tb2R1bGVmaWxlcy9tcGkvZ251LzguMC9vZmkvMS4wL2NyYXktbXBpY2gvOC4wIiwiL29wdC9jcmF5L3Bl";
os.environ["_ModuleTable008_"] = "L2xtb2QvbW9kdWxlZmlsZXMvcGVyZnRvb2xzLzIxLjAyLjAiLCIvb3B0L2NyYXkvcGUvbG1vZC9tb2R1bGVmaWxlcy9jb21uZXQvZ251LzguMC9vZmkvMS4wIiwiL29wdC9jcmF5L3BlL2xtb2QvbW9kdWxlZmlsZXMvbmV0L29maS8xLjAiLCIvb3B0L2NyYXkvcGUvbG1vZC9tb2R1bGVmaWxlcy9jb21jcHUvZ251LzguMC94ODYtcm9tZS8xLjAiLCIvb3B0L2NyYXkvcGUvbG1vZC9tb2R1bGVmaWxlcy9jcHUveDg2LXJvbWUvMS4wIiwiL29wdC9jcmF5L3BlL2xtb2QvbW9kdWxlZmlsZXMvY29tcGlsZXIvZ251LzguMCIsIi9vcHQvY3JheS9wZS9jcGUtcHJnZW52LzcuMC4wL2xtb2QiLCIvb3B0L2NyYXkvcGUvbG1vZC9tb2R1bGVmaWxlcy9jb3JlIiwiL29wdC9jcmF5L3BlL2xt";
os.environ["_ModuleTable009_"] = "b2QvbW9kdWxlZmlsZXMvY3JheXBlLXRhcmdldHMvZGVmYXVsdCIsIi9vcHQvbW9kdWxlZmlsZXMiLCIvb3B0L2NyYXkvbW9kdWxlZmlsZXMiLH0sfQ==";
os.environ["_ModuleTable_Sz_"] = "9";
_mlstatus = True
_mlstatus = False

--- stdout ---
--- stderr ---
Lmod Warning: Failed to find the following module(s): "python/3.8-anaconda-2020.11" in your MODULEPATH
Try:

    $ module spider python/3.8-anaconda-2020.11

to see if the module(s) are available across all compilers and MPI implementations.




--- stderr ---
[2021-04-01T10:23:55-07:00] ------------------------------------------------------------------------------
[2021-04-01T10:23:55-07:00] Log file(s) saved in: '/global/cfs/cdirs/mpccc/lgerhard/nersc/reframe-nersc-tests/reframe.log', '/global/cfs/cdirs/mpccc/lgerhard/nersc/reframe-nersc-tests/reframe.out'
login01> echo $MODULEPATH
/opt/cray/pe/lmod/modulefiles/cncm/crayclang/10.0/ofi/1.0/x86-rome/1.0/cray-mpich/8.0:/opt/cray/pe/lmod/modulefiles/mpi/crayclang/10.0/ofi/1.0/cray-mpich/8.0:/opt/cray/pe/lmod/modulefiles/perftools/21.02.0:/opt/cray/pe/lmod/modulefiles/comnet/crayclang/10.0/ofi/1.0:/opt/cray/pe/lmod/modulefiles/net/ofi/1.0:/opt/cray/pe/lmod/modulefiles/comcpu/crayclang/10.0/x86-rome/1.0:/opt/cray/pe/lmod/modulefiles/cpu/x86-rome/1.0:/opt/cray/pe/lmod/modulefiles/compiler/crayclang/10.0:/opt/cray/pe/cpe-prgenv/7.0.0/lmod:/opt/cray/pe/lmod/modulefiles/core:/opt/cray/pe/lmod/modulefiles/craype-targets/default:/opt/modulefiles:/opt/cray/modulefiles:/global/common/software/nersc/cos1.3/modulefiles:/global/common/software/nersc/cos1.3/extra_modulefiles

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:15 (9 by maintainers)

github_iconTop GitHub Comments

1reaction
lagerhardtcommented, May 17, 2021

Apologies, it turns out our private fork disabled the -l flag a looong time ago. If we take that section of code out, this environment variable works as expected. So we can add this to our site config for perlmutter and that should take care of this particular case.

0reactions
vkarakcommented, May 19, 2021

I’m closing this issue, since it’s not a ReFrame problem.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Problems in importing self-defined packages and modules?
(Spyder dev here) This is a bug in Spyder. We are not adding the project's path to PYTHONPATH and that's why you can't...
Read more >
Why Can't Python Find My Modules? - Real Python
A common error that new Pythonistas will come across is that the packages they think they've installed are not actually being recognized by...
Read more >
How to import a Python module given the full path?
The path variable contains the directories Python interpreter looks in for finding modules that were imported in the source files. Syntax :
Read more >
sys — System-specific parameters and functions — Python ...
This module provides access to some variables used or maintained by the interpreter and to functions that interact strongly with the interpreter.
Read more >
Bicep modules - Azure Resource Manager - Microsoft Learn
Definition syntax; Path to module; Parameters; Set module scope; Output; Next steps ... If you are not yet using Bicep, use template specs....
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found
Previous page

Support SGE scheduler

Next page

error running tutorial: decorated test ('MyTest') is an abstract test