question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Would you consider support for more object storage providers?

See original GitHub issue

Hi there,

I just discovered this project today, and it looks fantastic - thank you for making it!

Would you consider supporting more providers than just AWS if they are already boto/boto3 compatible ?

I see this code in the repo here which I think is responsible for uploading snapshots to an object storage service like AWS S3:

def get_s3_bucket():
    from boto.s3.connection import S3Connection

    conn = S3Connection(app_settings.S3_ACCESS_KEY,
                        app_settings.S3_SECRET_KEY)
    return conn.get_bucket(app_settings.S3_BUCKET)


def s3_upload(key, data):
    from boto.s3.key import Key
    bucket = get_s3_bucket()
    k = Key(bucket)
    k.key = key
    k.set_contents_from_file(data, rewind=True)
    k.set_acl('public-read')
    k.set_metadata('Content-Type', 'text/csv')
    return k.generate_url(expires_in=0, query_auth=False)

In a project I work on we use the boto library to talk to s3 compatible services from other providers, and I think to add this support to django-sql-explorer, it would mainly require a adding two new parameters to the code that connects to S3.

We have a comparable method on a project I work on where we fetch an S3-compatible bucket, which does a somewhat similar job to get_s3_bucket above.

def object_storage_bucket(bucket_name: str):
    """
    Return the bucket identified by `bucket_name` for uploading
    and downloading files.
    """
    session = boto3.Session(region_name=settings.OBJECT_STORAGE_REGION)

    object_storage = session.resource(
        "s3",
        endpoint_url=settings.OBJECT_STORAGE_ENDPOINT,
        aws_access_key_id=settings.OBJECT_STORAGE_ACCESS_KEY_ID,
        aws_secret_access_key=settings.OBJECT_STORAGE_SECRET_ACCESS_KEY,
    )
    return object_storage.Bucket(bucket_name)

While the example I have shared uses boto3, instead of boto, I think it would not be too complicated to add the extra params, the region_name, and endpoint_url to the code connecting to an object storage service.

Doing so would then make this library compatible with a range of S3-compatible providers (i.e. Cloudflare, Digital ocean, Scaleway, etc), as well as open source S3-compatible object storage software like Minio, Garage and so on.

I’m currently adding django-sql-explorer to a project I’m working on but once I’m a bit more familiar with it, would you accept a PR to add this functionality, and assuming it meets contribution standards, giving some pointers on how to have this gracefully integrated into the project?

Issue Analytics

  • State:closed
  • Created 10 months ago
  • Comments:8 (4 by maintainers)

github_iconTop GitHub Comments

1reaction
mrchrisadamscommented, Dec 5, 2022

You’re referring to this PR, right?

https://github.com/groveco/django-sql-explorer/pull/505

1reaction
mrchrisadamscommented, Dec 2, 2022

@marksweb - I found this project via Simon Willison’s blog below:

https://simonwillison.net/2021/May/10/django-sql-dashboard/

We use MariaDB where I work, and this project’s use of Django’s database connection code made it easy to understand and see how we could use this.

Thank you again for working on it 👍

Read more comments on GitHub >

github_iconTop Results From Across the Web

What is Object Storage and Why Should You Care?
Object storage is still relatively new, but it has key advantages. Learn more about what object storage is and why you should pay...
Read more >
Object Storage: An Introduction
With object-based storage, you can store and manage data volumes on the order of terabytes (TBs), petabytes (PBs), and even greater.
Read more >
What is Object Storage? Use cases & benefits
What is cloud object storage? Learn more about common use cases, benefits, how object storage works, and how it differs from file storage...
Read more >
Best Object Storage Software in 2022
Top Object Storage Software. Choose the right Object Storage Software using real-time, up-to-date product reviews from 1876 verified user ...
Read more >
11 Cloud Object Storage to Store Public & Private Data
Alibaba Cloud OSS (Object storage service) offers a 99.9% availability guarantee. Costing is based on what you use, without upfront or long-term commitment....
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found