Not detecting imports in python file and questions

Hi, I run pydeps on my file with this following command line (with different parameters) : pydeps JobScriptTemplate.py --max-bacon 6 --noise-level 2 --pylib

There is some imports in my file but it detects nothing, the graph is empty. This file comes from a package and it imports a class from another directory in this package. It imports :

import os
import sys
import time
import shutil
from commons.core.checker.RepetException import RepetException
from commons.core.sql.TableJobAdaptator import TableJobAdaptator
from commons.core.sql.DbFactory import DbFactory
from commons.core.sql.Job import Job

The directory ‘commons’ is like :

.
├── core
│   ├── checker
│   ├── launcher
│   └── sql
├── launcher
└── tools

and my file is on ‘/core/launcher’ (each directory has a init.py file).

Morever I don’t really understand the difference between --max-bacon and --noise-level. What is the effect of noise-level on max-bacon ? I’m wondering if I can attribute a color for a folder level. How can I do this ? Because I want to run pydeps on different folders and I want to keep the same color for each folder. And can I import and use pydeps in a python script ? I don’t find any constructor in pydeps sources.

I thank you in advance for your help. Mariene

[thebjorn]

Which version (pydeps --version) are you using?

I don’t really understand the difference between --max-bacon and --noise-level.

--max-bacon determines how many “hops” from the root that should be included in the graph (max bacon of 6 will likely include the entire universe… – the name comes from the game “6 degrees of Kevin Bacon”), while --noise-level=n excludes nodes that have more than n incoming/outgoing arrows. --max-bacon increases the size of the graph, --noise-level decreases the size of the graph.

If I remember correctly, then graphviz/dot has a limitation of 1000 nodes (or maybe it was 10 000), and if you hit that limit there will be no output.

I’m wondering if I can attribute a color for a folder level.

The color assignment is done here https://github.com/thebjorn/pydeps/blob/master/pydeps/colors.py#L47 (it uses the part before the first dot in the module name and adjusts saturation depending on how many arrows go in/out of the node). Early versions of pydeps tried to use different colors for different modules, but my implementation(s) didn’t work that well. Maybe you’ll have better luck…?

can I import and use pydeps in a python script ?

Yes, check e.g. https://github.com/thebjorn/pydeps/blob/master/tests/test_cli.py#L25

I run pydeps on my file with this following command line (with different parameters) : pydeps JobScriptTemplate.py --max-bacon 6 --noise-level 2 --pylib

I would suggest starting without any flags, i.e. pydeps JobScriptTemplate.py, although the preferred method is to call it on the root of the module: pydeps commons. Then you can increase --max-bacon if you don’t reach all the modules you want to find. Changing --noise-level is almost never needed (setting it to 2 will remove most nodes - the default is 200).

PS: Python is not designed to support running scripts from sub-folders (mostly because Guido doesn’t like it), so you’re fighting against the system when you try to run a script that is two levels deep. In particular, you need to use absolute imports (like you’re doing: from commons.core.sql.TableJobAdaptator import TableJobAdaptator) instead of relative imports (from ..sql.TableAdapter import ..). The implications for pydeps is that from a.b.c import d loads a.__init__, b.__init__, c.__init__, and d. If you try to import from sub-packages you’ll easily get circular imports.

The solution is to create a myproj/setup.py file with entry points to the file:function you want to use as a script (like pydeps does: https://github.com/thebjorn/pydeps/blob/master/setup.py#L47). After you run pip install -e myproj the command is available directly from the shell, and you’re free to use relative imports again.

[mariene]

You’re absolutely right, it’s @@cmdStart@@ and @@cmdFinish@@ that causes the error, I assume they exist because this script is used to generate python script that is submitted as a job (for a scheduler).Pydeps worked after removing them.

Thank you for your time.