question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Error occurred when exporting database to other formats

See original GitHub issue

Bug description [A clear and concise description of what the bug is.] An error occurred when I exported the database to CSV

abc-export --db test.db --out exported.csv --format csv
/home/xieduo/.local/lib/python3.8/site-packages/pandas/io/sql.py:1402: SAWarning: TypeDecorator BytesStorage() will not produce a cache key because the ``cache_ok`` attribute is not set to True.  This can have significant performance implications including some performance degradations in comparison to prior SQLAlchemy versions.  Set this attribute to True if this type object's state is safe to use in a cache key, or False to disable this warning. (Background on this error at: https://sqlalche.me/e/14/cprf)
  return self.connectable.execution_options().execute(*args, **kwargs)

The output file:

$ cat exported.csv
,index,epsilon,samples,m,model_name,p_model,w,particle_id,distance,par_name,par_val,sumstat_name,sumstat_val

Expected behavior [A clear and concise description of what you expected to happen.] Output the CSV file successfully To reproduce [A minimal example to reproduce this error, ideally a python script.] First run:

import os
import tempfile

import matplotlib.pyplot as plt
import numpy as np

import pyabc

pyabc.settings.set_figure_params('pyabc')  # for beautified plots

def model(parameter):
    return {"data": parameter["mu"] + 0.5 * np.random.randn()}

prior = pyabc.Distribution(mu=pyabc.RV("uniform", 0, 5))

def distance(x, x0):
    return abs(x["data"] - x0["data"])

abc = pyabc.ABCSMC(model, prior, distance, population_size=1000)

observation = 2.5
db_path = pyabc.storage.create_sqlite_db_id(file_="/data/xieduo/ABC_model/ABC_SMC/pyabc/test.db")
abc.new(db_path, {"data": observation})

history = abc.run(minimum_epsilon=0.1, max_nr_populations=10)

fig, ax = plt.subplots()
for t in range(history.max_t + 1):
    df, w = history.get_distribution(m=0, t=t)
    pyabc.visualization.plot_kde_1d(
        df,
        w,
        xmin=0,
        xmax=5,
        x="mu",
        xname=r"$\mu$",
        ax=ax,
        label=f"PDF t={t}",
    )
ax.axvline(observation, color="k", linestyle="dashed")
ax.legend();

And then:

abc-export --db test.db --out exported.csv --format csv

Environment [Report at least the pyABC version (pip show pyabc),

pip show pyabc
Name: pyabc
Version: 0.12.3
Summary: Distributed, likelihood-free ABC-SMC inference
Home-page: https://github.com/icb-dcm/pyabc
Author: The pyABC developers
Author-email: yannik.schaelte@gmail.com
License: BSD-3-Clause
Location: /home/xieduo/software/miniconda3/envs/jupyter-env/lib/python3.8/site-packages
Requires: scikit-learn, jabbar, redis, pandas, matplotlib, cloudpickle, click, sqlalchemy, scipy, distributed, numpy, gitpython
Required-by:

python version (which python; python --version),

Python 3.8.13

and system (Ubuntu/Mac/…).]

Linux pbs-master 3.10.0-1160.41.1.el7.x86_64 #1 SMP Tue Aug 31 14:52:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

And sqlalchemy version:

$ pip show sqlalchemy
Name: SQLAlchemy
Version: 1.4.32
Summary: Database Abstraction Library
Home-page: https://www.sqlalchemy.org
Author: Mike Bayer
Author-email: mike_mp@zzzcomputing.com
License: MIT
Location: /home/xieduo/.local/lib/python3.8/site-packages
Requires: greenlet
Required-by: pyabc

Thank you.

Best, Duo

Issue Analytics

  • State:closed
  • Created a year ago
  • Comments:6 (4 by maintainers)

github_iconTop GitHub Comments

1reaction
xieduo7commented, Nov 17, 2022

Hi @yannikschaelte , I tried it again and it works. Thank you!

0reactions
yannikschaeltecommented, Nov 16, 2022
Read more comments on GitHub >

github_iconTop Results From Across the Web

Error: An error occurred exporting the table - Esri Support
Navigate to Annotation Attribute Table > Options > Export > Browse button, select the desired output location, export the format required, and ...
Read more >
Database export fails when I try to export this format '2020-04 ...
I get an error: The conversion of a varchar data type to a datetime data type resulted in an out-of-range value. The process...
Read more >
Export a database object to another Access database
Open the source database, if it is not already open. The file format can be either MDB or ACCDB. If the file is...
Read more >
If you're unable to share an item in Final Cut Pro - Apple Support
This can happen if the storage device isn't formatted properly. This can also occur when sharing to destinations other than the Export File ......
Read more >
Error when exporting Module Data as XLS or PDF
1) Module contains at least 2 line items, one line item has TEXT format "Text Type of Link" WITHOUT value and the other...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found