question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

ModuleNotFoundError: No module named 'dagster.utils.test'

See original GitHub issue

Summary

I have installed the 0.9.22.post0 version of dagster, dagit and dagster-cron, and all was runing good. Then I make some changes in the Dockerfile, install some python libraries and start to give this error:

scrappers-web-tasks    | Traceback (most recent call last):
scrappers-web-tasks    |   File "/usr/local/bin/dagit", line 5, in <module>
scrappers-web-tasks    |     from dagit.cli import main
scrappers-web-tasks    |   File "/usr/local/lib/python3.8/site-packages/dagit/__init__.py", line 1, in <module>
scrappers-web-tasks    |     from dagster.core.utils import check_dagster_package_version
scrappers-web-tasks    |   File "/usr/local/lib/python3.8/site-packages/dagster/__init__.py", line 153, in <module>
scrappers-web-tasks    |     from dagster.utils.test import (
scrappers-web-tasks    | ModuleNotFoundError: No module named 'dagster.utils.test'
scrappers-web-tasks exited with code 1

I update the version to 0.10.5 and the error persist.

Reproduction

This error goes when I start the container with the folowing command:

dagit -f /src/tasks/schedule_definition.py -h 0.0.0.0 -p 3000

I dont understunt why could be this and dont find nothing in internet with an answer.
So I hope you could help me find the bug.

Additional Info about Your Environment

Dockerfile

FROM python:3.8-slim
ENV PYTHONUNBUFFERED=1

RUN apt-get update

# Set general dependencies
RUN apt-get install -y --no-install-recommends \
    build-essential \
    libgflags-dev \
    libsnappy-dev \
    zlib1g-dev \
    libbz2-dev \
    liblz4-dev \
    librocksdb-dev \
    libzstd-dev \
    cron \
    wget \
    xvfb \
    unzip \
    gcc \
    g++ \
    gnupg \
    gnupg1 \
    gnupg2 \
    unixodbc-dev \
    python3-dev

# Set up chrome dependencies
RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add -
RUN echo "deb http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list
RUN apt-get update -y
RUN apt-get install -y google-chrome-stable

# Create the workdir
WORKDIR scrape_web_tasks

CMD mkdir scrape_web_tasks/src
COPY requirements.txt src/
RUN pip install -r src/requirements.txt

# Clear dependencies
RUN apt-get remove -y --purge \
    gcc \
    g++ \
    unixodbc-dev \
# clear apt-get cache & remove unnecessary files
    && apt-get -y autoremove \
    && apt-get clean && rm -rf /var/lib/apt/lists/* \
    && find /usr/local -depth \
    	\( \
    		\( -type d -a \( -name test -o -name tests -o -name idle_test \) \) \
    		-o \( -type f -a \( -name '*.pyc' -o -name '*.pyo' -o -name '*.a' \) \) \
    		-o \( -type f -a -name 'wininst-*.exe' \) \
    	\) -exec rm -rf '{}' + \
    && rm -f get-pip.py \
    && rm -rf /usr/src/python

ENV SRCDIR "/scrape_web_tasks/src/"
ENV DAGSTER_HOME "${SRCDIR}"
ENV PYTHONPATH "${SRCDIR}"

requirements.txt

click==7.1.2
coverage==5.3.1
awscli==1.18.212
flake8==3.8.4
python-dotenv>=0.5.1
Sphinx==3.4.3

# Scrappers
requests==2.24.0
selenium==3.141.0
webdriver-manager==3.2.2
beautifulsoup4==4.9.3
pandas==1.1.3
fs==2.4.12
six==1.15.0
pysmb==1.2.6
property-cached==1.6.4
fs.smbfs==1.0.0

# TEST
pytest==5.4.2
pytest-sugar==0.9.4
freezegun==1.0.0
ipdb==0.13.3

# Repositories
pymongo==3.11.0
SharePlum==0.5.1
pyodbc==4.0.30

# Dagster
dagster==0.10.5
dagit==0.10.5
dagster-cron==0.10.5

# Extras
rarfile==4.0

Lastes changes:

  • pyodbc==4.0.30
  • fs==2.4.12
  • six==1.15.0
  • pysmb==1.2.6
  • property-cached==1.6.4
  • fs.smbfs==1.0.0

Message from the maintainers:

Impacted by this bug? Give it a 👍. We factor engagement into prioritization.

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Reactions:1
  • Comments:6 (4 by maintainers)

github_iconTop GitHub Comments

2reactions
gibsondancommented, Feb 12, 2021

I believe it’s the part of the dependency clear step where it wipes out any files with “-name test”. There are some modules included in dagster that have “test” in the name (to support users writing their own tests).

0reactions
galloramirocommented, Feb 12, 2021

I believe it’s the part of the dependency clear step where it wipes out any files with “-name test”. There are some modules included in dagster that have “test” in the name (to support users writing their own tests).

That was!! Thnks!!

Read more comments on GitHub >

github_iconTop Results From Across the Web

Root repo does not load · Issue #3220 · dagster-io ... - GitHub
DagsterUserCodeProcessError: (ModuleNotFoundError) - ModuleNotFoundError: No module named 'dagster_test.toys.fan_in_fan_out' Stack Trace: ...
Read more >
ModuleNotFoundError: No module named 'dagster-postgres'
Hi, My Python program is throwing following error: ModuleNotFoundError: No module named 'dagster-postgres' How to remove the M.
Read more >
Source code for dagster._core.definitions.utils
Source code for dagster._core.definitions.utils. import keyword import os import re from glob import glob from typing import Any, Dict, Mapping, Optional, ...
Read more >
No module named ..." when trying to make unit tests in Python
I am trying to make unit tests for my project, but I am having issues figuring out where my error is in regards...
Read more >
No module named pkg_resources - python - Stack Overflow
EDIT: This only happens inside the virtualenv. If I open a console outside the virtualenv then pkg_resources is present, but I am still...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found