Request: EIA 861 data 1990-2000

Is your feature request related to a problem? Please describe.

I’m using EIA-861 for a research project, and it would be convenient if I could use the PUDL-compiled version. I’d like data that goes as far back as possible. Currently the 861 files from 1990 through 2000 are not included.

I realize this old dataset might be a bit niche, and therefore not a priority. As the docs mention, only operational data and sales are available that far back (tables operational_data_eia861 and sales_eia861).

Describe the solution you’d like

Ideally, I’d like to have the full history of eia861 data included in the database.

Describe alternatives you’ve considered

I could also work with the data myself, outside of PUDL.

Additional context

EIA added a new, reformatted version of the old EIA 861 files (1990 through 2011; they kept the old format too). The new version of the old files more closely matches the formatting of the files from 2012 and later, so extraction might be easier. The change happened sometime between April 18 and May 9, 2020. As far as I can tell, the change did not affect the pudl-scraper code because it takes the first available ZIP. The reformatted version is listed second.

[karldw]

Just a quick update: I won’t be able to work on this for a few weeks, but I’ll take a look in April.

[aesharpe]

Ah ok, thanks for clarifying. This is indeed strange! I’m hopeful that EIA can provide some clarification here, and then we can make a more informed decision about which file format to use for the 2000-2011 data.