Add a command-line option for overwriting exported file

What do you think about adding an option to overwrite/recreate exported file? Something like

$ scrapy crawl myspider -o data.jl --overwrite

or

$ scrapy crawl myspider -O data.jl

This is useful during development where old data is not needed. I usually run

$ rm data.jl; scrapy crawl myspider -o data.jl

multiple times. This is not DRY: if I want to change file name for the next iteration and preserve existing file then I must be careful and update both names (of course, the command comes from shell autocompletion).

[kmike]

If we have -o and -O then how can you tell which one is default? Both must be written explicitly.

But @darkrho raised an interesting question because -o doesn’t play well with csv, xml and pickle currently: csv adds an extra header, xml starts a new xml instead of adding to previous, and pickle does the same.

As a side note: scrapy crawl myspider -o combined.jl -O last.jl can also be useful, and users will try writing this.

[robkorv]

In the meantime one could subclass FileFeedStorage. I just typed an answer over at http://stackoverflow.com/a/40823149/2546958 before I spotted this issue.