question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Data is not loaded in Amundsen from Mysql database

See original GitHub issue

This is the code I replaced from file and using it to load data from mysql database (films) into amundsen

def connection_string():
    user = 'root'
    password = '747747747Yash$'
    host = 'localhost'
    port = '3306'
    db = 'films'
    return "mysql+pymysql://%s:%s@%s:%s/%s" % (user, password, host, port, db)



def run_mysql_job(): # set where clause according to your need
    where_clause_suffix = textwrap.dedent("""
        where c.table_schema = 'inforamtion_schema'
    """)

    tmp_folder = '/var/tmp/amundsen/table_metadata'
    node_files_folder = '{tmp_folder}/nodes/'.format(tmp_folder=tmp_folder)
    relationship_files_folder = '{tmp_folder}/relationships/'.format(tmp_folder=tmp_folder)

    job_config = ConfigFactory.from_dict({
        'extractor.mysql_metadata.{}'.format(MysqlMetadataExtractor.WHERE_CLAUSE_SUFFIX_KEY):
            where_clause_suffix,
        'extractor.mysql_metadata.{}'.format(MysqlMetadataExtractor.USE_CATALOG_AS_CLUSTER_NAME):
            True,
        'extractor.mysql_metadata.extractor.sqlalchemy.{}'.format(SQLAlchemyExtractor.CONN_STRING):
            connection_string(),

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:5 (4 by maintainers)

github_iconTop GitHub Comments

1reaction
feng-taocommented, Aug 8, 2020

close it for now, feel free to reopen if the issue persists.

1reaction
jornhcommented, Jul 13, 2020

Looks like you have a typo in this line:

        where c.table_schema = 'inforamtion_schema'

Should probably be information_schema

Read more comments on GitHub >

github_iconTop Results From Across the Web

Data is not loaded in Amundsen from Mysql database #549
This is the code I replaced from file and using it to load data from mysql database (films) into amundsen def connection_string(): user ......
Read more >
Amundsen Databuilder
An extractor that extracts table and column metadata including database, schema, table name, table description, column name and column description from a MYSQL...
Read more >
How can I connect mysql to Amundsen Lyft? - Stack Overflow
I'm working on the PoC about Amundsen Lyft for introducing data catalog. So, I use the file 'sample_mysql_loader.py' and insert database ...
Read more >
MySQL 8.0 Reference Manual :: 13.2.9 LOAD DATA Statement
Input File Location. These rules determine the LOAD DATA input file location: If LOCAL is not specified, the file must be located on...
Read more >
Amundsen Set Up Tutorial: A Step-By-Step Installation Guide
Metadata — can choose to store and retrieve data from a host of database backends, including neo4j, MySQL on RDS, Apache Atlas, etc....
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found