question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Must stream updates to all existing columns

See original GitHub issue

Streaming a data frame with same columns and index raises ‘Must stream updates to all existing columns’

from bokeh.plotting import figure
from bokeh.models import ColumnDataSource
import pandas as pd

df = pd.DataFrame(index= pd.date_range('20160701', '20160710', freq='1T', name='time'), 
                                      columns=['X', 'Y'])
df['X'] = df.reset_index().index
df['Y'] = df.X * df.X

source = ColumnDataSource(df)
source.stream(df) 

Issue Analytics

  • State:closed
  • Created 7 years ago
  • Comments:9 (4 by maintainers)

github_iconTop GitHub Comments

3reactions
satyan-gcommented, Jul 13, 2016

Hi, point taken about the process. That said, this is still an issue, so not sure why this was closed. Note that the ColumnDataSource was created and stream’d using the same dataframe with no structural changes to the dataframe in between (there is no addition or deletion of columns going on here), and it still fails. It seems both the constructor and stream() are converting ‘time’, the index, to a regular column, but the check in stream() is not taking index in to account.

In fact, this also fails with a simple dataframe with default index. For example:

In [13]: df = pd.DataFrame({‘X’ : range(100)}) In [14]: source = ColumnDataSource(df) In [15]: source.stream(df)


ValueError Traceback (most recent call last) <ipython-input-15-fdf154a68872> in <module>() ----> 1 source.stream(df)

e:\3rdParty\Enthought1.5.5b\lib\site-packages\bokeh\models\sources.pyc in stream(self, new_data, roll over) 231 raise ValueError(“Must stream updates to all existing columns (missing: %s, e xtra: %s)” % (", ".join(sorted(missing)), ", ".join(sorted(extra)))) 232 elif missing: –> 233 raise ValueError(“Must stream updates to all existing columns (missing: %s)” % ", ".join(sorted(missing))) 234 else: 235 raise ValueError(“Must stream updates to all existing columns (extra: %s)” % ", ".join(sorted(extra)))

ValueError: Must stream updates to all existing columns (missing: index)

0reactions
bryevdvcommented, Apr 23, 2018

was this fixed?

it would be easier to answer your question if you had included version information. (Please always include version information, in any issue, anywhere) It would be even better if you had included a complete code sample. I would have been able to just run it on master and answer definitively immediately. As it is, all I can say is there were previous PRs merged that were possibly related:

https://github.com/bokeh/bokeh/pull/6738 https://github.com/bokeh/bokeh/pull/7379

If either of those is later than your installed version, try upgrading.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Must stream updates to all existing columns (missing: index ...
Live data streaming gives this error. And the new data has exactly same columns as the original source. And the code seems pretty...
Read more >
Error thrown from periodic callback: ValueError('Must stream ...
Error thrown from periodic callback: ValueError('Must stream updates to all existing columns (missing: index)',) · Subscribe to RSS.
Read more >
Table streaming reads and writes | Databricks on AWS
All table changes starting from this version (inclusive) will be read by the streaming source. You can obtain the commit versions from the...
Read more >
Change data capture with Delta Live Tables - Azure Databricks
Note. This article describes how to update tables in your Delta Live Tables pipeline based on changes in source data.
Read more >
Introduction to Streams - Snowflake Documentation
Supported for streams on tables, directory tables, or views. A standard (i.e. delta) stream tracks all DML changes to the source object, including...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found