question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Might be nice to always call `exporter.finish_exporting`

See original GitHub issue

Description

When an exception happens inside Exporter (for whatever reason) - itemcount never gets increased (that’s OK). But then during closing, the code doesn’t get chance to act

This is the place in the code:

https://github.com/scrapy/scrapy/blob/master/scrapy/extensions/feedexport.py#L357

Perhaps the lines 357 and 353 should have been swapped?

Additional context

I’m using slot.storage to open a database; and an exporter to start a session – store() gets called (to close connection to the database) but session didn’t have chance to commit – yes, there is nothing to commit, but still - given the semantics (also visible from the code – when file is supposed to be a file descriptor it might be worth consideration to always call finish_exporting to give the file descriptor chance to react before it is closed by the parent object)

necessary caveat: I might be mis-using the API and it is a minor detail, so please forgive if this looks silly (and feature not a bug)

Issue Analytics

  • State:open
  • Created a year ago
  • Comments:9 (4 by maintainers)

github_iconTop GitHub Comments

1reaction
wRARcommented, Oct 28, 2022

If you mean a test case for making sure it’s ohly called when start_exporting() was called it would just be a test spider with some exporter enabled and some different edge cases like zero items, maybe an exception happening at some early point etc.

0reactions
wRARcommented, Oct 31, 2022

Even if slot.finish_exporting() was called without the start being called, it wouldn’t do anything because it checks for the flag/boolean set by start. Hence even if it gets called , it wont do any harm. Am I right?

Yes, this sounds correct.

if test will be required can you point me to an already written test case, which I can use as an example.

I don’t have one handy, but it looks like tests/test_exporters.py is a place with a lot of tests of the related code.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Reprepro export could not find signing key - Server Fault
ERROR: Could not finish exporting 'unstable'!. I Googled and found a couple of old threads that indicated a possible problem with reprepro ...
Read more >
How to export Outlook contacts to CSV, PST or vCard
Now, choose the folder you want to export. By default, your contacts will be in the Contacts folder. Mind that there is no...
Read more >
Import Outlook contacts to Gmail and export Google ... - Ablebits
Click Finish and Outlook will start exporting your contacts immediately. ... The wizard only exports the contacts in your personal contact ...
Read more >
Screen your calls before you answer them - Phone app Help
You can use Call Screen to find out who's calling and why before you pick up a call. Call Screen works on your...
Read more >
Convert PDF to Word - Online PDF to Word Converter | pdfFiller
To finish exporting, select Save As at the bottom of the screen and find a sample in the chosen destination. You can also...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found