question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. ItĀ collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

python flag for `--output` to export chats to json?

See original GitHub issue

is there a python flag for --output so I can export chats to json?

I see this in the CLI documentation (you included -o flag in your cmd) but not for the ChatDownloader().get_chat() … I need this json so I can pd.read_json for ETL of chat data

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-106-472d1877d159> in <module>
----> 1 chat = ChatDownloader().get_chat(url=url,message_types=["text_message", "paid_message", "membership_item"],output=outfile)

TypeError: get_chat() got an unexpected keyword argument 'output'

__Originally posted by @atnjqt in https://github.com/xenova/chat-downloader/issues/61#issuecomment-778847756__

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:5 (3 by maintainers)

github_iconTop GitHub Comments

1reaction
xenovacommented, Feb 15, 2021

Hi there!

This functionality has been included in the code, but not in the get_chat method. The get_chat method was only responsible for returning a an iterable Chat object (hence the get in the name). This Chat object has the ability to generate more messages (using http requests) when needed.

File output and other more advanced processing should be the responsibility of the user implementing the code. That being said, there are ways to use the module’s outputting functionality as I will describe later.

Early into development, users wanted a way to continuously write to file. Since this is not standard functionality for file IO in python, the continuous_write.py module was created.

Now, there are many ways to use this, but I will describe 2:

  1. Using ContinuousWriter
from chat_downloader import ChatDownloader
from chat_downloader.output.continuous_write import ContinuousWriter

# We now create a ContinuousWriter object and set options. Different
# options are allowed for different output formats:
#
# - JSON: file_name, overwrite=False, indent=None, separator=', ', indent_character=' ', sort_keys=True
# - CSV: file_name, overwrite=False, sort_keys=True
# - TXT: file_name, overwrite=False
#
# All values except file_name are optional, and assume the default values listed above


file_name = 'output.json' # or 'output.csv' or 'output.txt'

writer = ContinuousWriter(file_name, overwrite=True)

url = 'https://www.youtube.com/watch?v=5qap5aO4i9A'
chat = ChatDownloader().get_chat(url)   # create a generator

try:
    for message in chat:                    # iterate over messages
        writer.write(message)
        print(chat.format(message))         # print the formatted message
finally:
    writer.close()

  1. Using run. Alternatively, if your program’s functionality matches that of the command line interface, you can ā€œsimulateā€ running it from the command line using the run(**kwargs) method:
from chat_downloader import run

url = 'https://www.youtube.com/watch?v=5qap5aO4i9A'
file_name = 'output.json'  # or 'output.csv' or 'output.txt'
run(
    url=url,
    output=file_name,
    overwrite=True
)

This option is less suitable if you need more control.

0reactions
xenovacommented, Feb 16, 2021

of course a quiet arg makes sense, thanks for this! I missed that when skimming through the documentation …

No worries! Yeah, the documentation still needs a lot of work… šŸ˜…

Read more comments on GitHub >

github_iconTop Results From Across the Web

Python conversion from JSON to JSONL - Stack Overflow
5 Answers 5 ; import json with open ; 'output.jsonl', 'w') as ; for entry in JSON_file: json.dump(entry, outfile) outfile.write('\n').
Read more >
Working With JSON Data in Python
In this tutorial you'll learn how to read and write JSON-encoded data ... Well, us sneaky Pythoneers use the -i interactive flag when...
Read more >
Chatistics | Python scripts to parse Messenger, Hangouts ...
Python scripts to parse Messenger, Hangouts, WhatsApp and Telegram chat logs into DataFrames.
Read more >
bigquery-schema-generator - PyPI
BigQuery schema generator from JSON or CSV data. ... Command Line; Schema Output; Command Line Flag Options. Help ( --help ); Input Format...
Read more >
CoffeeScript
-t, --transpile, Pipe the CoffeeScript compiler's output through Babel before saving or running ... Use this flag to forward options directly to Node.js....
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found