question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Pandas import error

See original GitHub issue

A Dash user is seeing an error from pandas without ever directly importing pandas, just dash, plotly, and numpy. https://community.plotly.com/t/callback-error-when-plotting-multiple-graph-objects/38756

The error occurs inside plotly when the Dash app tries to render one of the plots in a callback:

Traceback (most recent call last):
  File "/Users/alex/plotly/fiddle/f.py", line 74, in update_dist_plot
    "data": [go.Bar(x=bins, y=counts)],
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/plotly/graph_objs/__init__.py", line 93149, in __init__
    self["x"] = x if x is not None else _v
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/plotly/basedatatypes.py", line 3490, in __setitem__
    self._set_prop(prop, value)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/plotly/basedatatypes.py", line 3772, in _set_prop
    val = validator.validate_coerce(val)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/_plotly_utils/basevalidators.py", line 385, in validate_coerce
    v = copy_to_readonly_numpy_array(v)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/_plotly_utils/basevalidators.py", line 93, in copy_to_readonly_numpy_array
    if pd and isinstance(v, (pd.Series, pd.Index)):
AttributeError: module 'pandas' has no attribute 'Series'

That’s the error I see - the OP’s error message is a little more extensive, which is a little funny because we both report pandas v1.0.3

AttributeError: partially initialized module ‘pandas’ has no attribute ‘Series’ (most likely due to a circular import)

I can reproduce locally with this app:

import dash_core_components as dcc
import dash_html_components as html
from dash import Dash
from dash.dependencies import Input, Output
import plotly.graph_objs as go
import numpy as np

app = Dash(__name__)

app.layout = html.Div(
    [
        html.Div(
            [
                html.Br(),
                html.Label("Plotting Options"),
                dcc.RadioItems(
                    id="trunk-angle-radio",
                    options=[
                        {"label": "Sagittal", "value": "Sagittal"},
                        {"label": "Lateral", "value": "Lateral"},
                        {"label": "Twist", "value": "Twist"},
                    ],
                    value="Sagittal",
                ),
            ]
        ),
        html.Div(
            [
                html.Div(
                    [dcc.Graph(id="trunk-angle-plot")],
                    style={"width": "48%", "display": "inline-block"},
                ),
                html.Div(
                    [dcc.Graph(id="trunk-angle-dist")],
                    style={"width": "48%", "display": "inline-block", "float": "right"},
                ),
            ]
        ),
        html.Div(
            [
                html.Label("Data Statistics"),
                html.Div(id="data-stats-div", style={"padding": 10}),
            ]
        ),
    ]
)


@app.callback(
    Output("trunk-angle-plot", "figure"), [Input("trunk-angle-radio", "value")]
)
def update_angle_plot(radio_option):
    (x, y) = get_trunk_angles(radio_option)
    fig = {
        "data": [go.Scatter(x=x, y=y, mode="lines+markers")],
        "layout": go.Layout(
            title="Trunk Angle Time Series Plot",
            xaxis={"title": "Time (sec)"},
            yaxis={"title": "Degrees"},
        ),
    }
    return fig


@app.callback(
    Output("trunk-angle-dist", "figure"), [Input("trunk-angle-radio", "value")]
)
def update_dist_plot(radio_option):
    (x, y) = get_trunk_angles(radio_option)
    counts, bins = np.histogram(y, bins=range(-90, 91, 30))
    bins = bins + (bins[1] - bins[0]) / 2
    # print(counts, bins)
    fig = {
        "data": [go.Bar(x=bins, y=counts)],
        "layout": go.Layout(
            title="Trunk Angle Distributions",
            xaxis={
                "title": "Bin midpoint (degrees)",
                "tickmode": "array",
                "tickvals": bins,
                "ticktext": [str(int(bin)) for bin in bins],
            },
            yaxis={"title": "Percentage of time"},
        ),
    }
    return fig


@app.callback(
    Output("data-stats-div", "children"), [Input("trunk-angle-radio", "value")]
)
def update_stats(radio_option):
    (x, y) = get_trunk_angles(radio_option)
    stats_div = [
        html.Div("Minimum: {}".format(np.min(y)), id="trunk-angle-dist-min"),
        html.Div("Maximum: {}".format(np.max(y)), id="trunk-angle-dist-max"),
        html.Div("Mean: {:.2f}".format(np.mean(y)), id="trunk-angle-dist-mean"),
        html.Div(
            "Standard Deviation: {:.2f}".format(np.std(y)), id="trunk-angle-dist-std"
        ),
        html.Div(
            "Range: {}".format(np.max(y) - np.min(y)), id="trunk-angle-dist-range"
        ),
    ]
    return stats_div


def get_trunk_angles(radio_option):

    dummy_x = np.linspace(0, 50, 101)

    if radio_option == "Sagittal":
        dummy_y = np.random.randint(-90, 90, 101)
    elif radio_option == "Lateral":
        dummy_y = np.random.randint(-90, 90, 101)
    elif radio_option == "Twist":
        dummy_y = np.random.randint(-90, 90, 101)

    return (dummy_x, dummy_y)


if __name__ == "__main__":
    app.run_server(debug=True)

Issue Analytics

  • State:open
  • Created 3 years ago
  • Reactions:2
  • Comments:10 (7 by maintainers)

github_iconTop GitHub Comments

1reaction
alexcjohnsoncommented, May 4, 2020

The error appears for me when first loading the app in the browser: Screen Shot 2020-05-04 at 5 33 42 PM

If I reload the page everything is fine thereafter - so it’s just the first time these callbacks are called. after starting the app Maybe the two callbacks are being called in parallel threads and this is a race condition? One callback starts importing pandas but doesn’t finish… the second one sees pandas as already loaded and tries to use it prematurely?

0reactions
mhwaagecommented, Oct 29, 2021

I have a scenario that seems to reproduce this type of error fairly consistently:

from uuid import uuid4

import flask
import dash
from dash import html, dcc
from dash.dependencies import Output, Input

import plotly.graph_objects as go


server = flask.Flask('app')
app = dash.Dash('app', server=server)


n_comps = 10
buttton_id = "click_me"
n_clicks_counter_id = str(uuid4())
figids = [str(uuid4()) for _ in range(n_comps)]

@app.callback(
    Output(n_clicks_counter_id, "children"),
    Input(buttton_id, "n_clicks")
    )
def update_count(n_clicks):
    return str(n_clicks)

for figid in figids:
    @app.callback(
        Output(figid, "figure"),
        Input(buttton_id, "n_clicks")
        )
    def plot(val):
        if val is None:
            val = 0
        figure = go.Figure(
            data=[go.Bar(x=[1+val, 2, 3], y=[1, 3, 2])],
            layout=go.Layout(
                title=go.layout.Title(text="A Figure Specified By A Graph Object")
            )
        )
        return figure
            

    app.layout = html.Div(
            id="main-title",
            children=[html.Button(id=buttton_id, children="click me"), html.Div(children="0", id=n_clicks_counter_id)] + 
            [
                dcc.Graph(id=figid) for figid in figids
             ]
        )

if __name__ == '__main__':
    app.run_server()

Instead of running the script directly, start the server with gunicorn set to do semi-frequent worker reboots; e.g. gunicorn --bind 0.0.0.0:5000 --keep-alive 120 --max-requests 40 --preload --workers 2 --worker-class gthread --threads 2 --timeout 1000000 "reproducing_module:server".

Then, click the button repeatedly; it should trigger an internal error fairly frequently. With dash 2.0 + orjson installed, it seems to first trigger a bad import of orjson:

Exception on /_dash-update-component [POST]
Traceback (most recent call last):
  File "/home/mhwa/.cache/pypoetry/virtualenvs/temp-BbzQGPLE-py3.8/lib/python3.8/site-packages/flask/app.py", line 2073, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/mhwa/.cache/pypoetry/virtualenvs/temp-BbzQGPLE-py3.8/lib/python3.8/site-packages/flask/app.py", line 1518, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/mhwa/.cache/pypoetry/virtualenvs/temp-BbzQGPLE-py3.8/lib/python3.8/site-packages/flask/app.py", line 1516, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/mhwa/.cache/pypoetry/virtualenvs/temp-BbzQGPLE-py3.8/lib/python3.8/site-packages/flask/app.py", line 1502, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
  File "/home/mhwa/.cache/pypoetry/virtualenvs/temp-BbzQGPLE-py3.8/lib/python3.8/site-packages/dash/dash.py", line 1336, in dispatch
    response.set_data(func(*args, outputs_list=outputs_list))
  File "/home/mhwa/.cache/pypoetry/virtualenvs/temp-BbzQGPLE-py3.8/lib/python3.8/site-packages/dash/_callback.py", line 191, in add_context
    jsonResponse = to_json(response)
  File "/home/mhwa/.cache/pypoetry/virtualenvs/temp-BbzQGPLE-py3.8/lib/python3.8/site-packages/dash/_utils.py", line 21, in to_json
    return to_json_plotly(value)
  File "/home/mhwa/.cache/pypoetry/virtualenvs/temp-BbzQGPLE-py3.8/lib/python3.8/site-packages/plotly/io/_json.py", line 127, in to_json_plotly
    opts = orjson.OPT_SORT_KEYS | orjson.OPT_SERIALIZE_NUMPY
AttributeError: partially initialized module 'orjson' has no attribute 'OPT_SORT_KEYS' (most likely due to a circular import)

if orjson is pre-emptively imported, it will instead break on a numpy import:

Exception on /_dash-update-component [POST]
Traceback (most recent call last):
  File "/home/mhwa/.cache/pypoetry/virtualenvs/temp-BbzQGPLE-py3.8/lib/python3.8/site-packages/flask/app.py", line 2073, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/mhwa/.cache/pypoetry/virtualenvs/temp-BbzQGPLE-py3.8/lib/python3.8/site-packages/flask/app.py", line 1518, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/mhwa/.cache/pypoetry/virtualenvs/temp-BbzQGPLE-py3.8/lib/python3.8/site-packages/flask/app.py", line 1516, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/mhwa/.cache/pypoetry/virtualenvs/temp-BbzQGPLE-py3.8/lib/python3.8/site-packages/flask/app.py", line 1502, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
  File "/home/mhwa/.cache/pypoetry/virtualenvs/temp-BbzQGPLE-py3.8/lib/python3.8/site-packages/dash/dash.py", line 1336, in dispatch
    response.set_data(func(*args, outputs_list=outputs_list))
  File "/home/mhwa/.cache/pypoetry/virtualenvs/temp-BbzQGPLE-py3.8/lib/python3.8/site-packages/dash/_callback.py", line 151, in add_context
    output_value = func(*func_args, **func_kwargs)  # %% callback invoked %%
  File "/home/mhwa/local/temp/temp/try_to_break.py", line 36, in plot
    data=[go.Bar(x=[1+val, 2, 3], y=[1, 3, 2])],
  File "/home/mhwa/.cache/pypoetry/virtualenvs/temp-BbzQGPLE-py3.8/lib/python3.8/site-packages/plotly/graph_objs/_bar.py", line 3174, in __init__
    self["x"] = _v
  File "/home/mhwa/.cache/pypoetry/virtualenvs/temp-BbzQGPLE-py3.8/lib/python3.8/site-packages/plotly/basedatatypes.py", line 4819, in __setitem__
    self._set_prop(prop, value)
  File "/home/mhwa/.cache/pypoetry/virtualenvs/temp-BbzQGPLE-py3.8/lib/python3.8/site-packages/plotly/basedatatypes.py", line 5158, in _set_prop
    val = validator.validate_coerce(val)
  File "/home/mhwa/.cache/pypoetry/virtualenvs/temp-BbzQGPLE-py3.8/lib/python3.8/site-packages/_plotly_utils/basevalidators.py", line 400, in validate_coerce
    elif is_homogeneous_array(v):
  File "/home/mhwa/.cache/pypoetry/virtualenvs/temp-BbzQGPLE-py3.8/lib/python3.8/site-packages/_plotly_utils/basevalidators.py", line 187, in is_homogeneous_array
    and isinstance(v, np.ndarray)
AttributeError: partially initialized module 'numpy' has no attribute 'ndarray' (most likely due to a circular import)

I tried this on a couple of machines and it seems to reproduce consistently.

Read more comments on GitHub >

github_iconTop Results From Across the Web

import pandas as pd ImportError: No module named pandas
Go to file > settings > project interpreter and see if pandas is available in the list of packages. if not simply click...
Read more >
No module named pandas [Mac/Linux/Windows/PyCharm]
Open File > Settings > Project from the PyCharm menu. · Select your current project. · Click the Python Interpreter tab within your...
Read more >
ImportError: No module named pandas
In most cases this error in Python generally raised: ... So, before being able to import the Pandas module , you need to...
Read more >
How to Fix: No module named pandas
Step 1: pip install pandas ... Since pandas doesn't come installed automatically with Python, you'll need to install it yourself. The easiest way ......
Read more >
How to Fix: No module named pandas
The error “No module named pandas ” will occur when there is no pandas library in your environment IE the pandas module is...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found