question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

UnicodeDecodeError during fits.Header.fromfile()

See original GitHub issue

MWE:

import numpy as np
from astropy.io import fits

fn = 'test.fits'
data = np.random.randn(1, 1)
hdu = fits.HDUList(fits.PrimaryHDU(data))
hdu.writeto(fn, overwrite=True)
header = fits.Header.fromfile(fn)

produces:

---------------------------------------------------------------------------
UnicodeDecodeError                        Traceback (most recent call last)
<ipython-input-2-32a656c8e40e> in <module>
      6 hdu = fits.HDUList(fits.PrimaryHDU(data))
      7 hdu.writeto(fn, overwrite=True)
----> 8 header = fits.Header.fromfile(fn)

/usr/local/lib/python3.6/dist-packages/astropy/io/fits/header.py in fromfile(cls, fileobj, sep, endcard, padding)
    445 
    446             return cls._from_blocks(block_iter, is_binary, sep, endcard,
--> 447                                     padding)[1]
    448         finally:
    449             if close_file:

/usr/local/lib/python3.6/dist-packages/astropy/io/fits/header.py in _from_blocks(cls, block_iter, is_binary, sep, endcard, padding)
    474         # Read the first header block.
    475         try:
--> 476             block = next(blocks)
    477         except StopIteration:
    478             raise EOFError()

/usr/local/lib/python3.6/dist-packages/astropy/io/fits/header.py in block_iter(nbytes)
    437             def block_iter(nbytes):
    438                 while True:
--> 439                     data = fileobj.read(nbytes)
    440 
    441                     if data:

/usr/lib/python3.6/codecs.py in decode(self, input, final)
    319         # decode input (taking the buffer into account)
    320         data = self.buffer + input
--> 321         (result, consumed) = self._buffer_decode(data, self.errors, final)
    322         # keep undecoded input until the next call
    323         self.buffer = data[consumed:]

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa4 in position 2883: invalid start byte

However, opening the file in binary mode seems to work:

with open(fn, 'rb') as fp:
    header = fits.Header.fromfile(fp)

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:13 (13 by maintainers)

github_iconTop GitHub Comments

1reaction
astromancercommented, May 20, 2019

I closed this before realising @embray had done some additional work on the PR. Keeping open for now.

1reaction
bsipoczcommented, May 15, 2019

@astromancer - Please open a PR if you think you have the fix.

Read more comments on GitHub >

github_iconTop Results From Across the Web

UnicodeDecodeError when reading CSV file in Pandas with ...
A real world example is an UTF8 file that has been edited with a non utf8 editor and which contains some lines with...
Read more >
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x81 in ...
Already Solved at error UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte. Source: flutterq.com.
Read more >
UnicodeDecodeError when reading CSV file in Pandas with Python ...
I'm running a program which is processing 30,000 similar files. A random number of them are stopping and producing this error.
Read more >
Headers — Astropy v5.2
This class exposes both a dict-like interface and a list-like interface to FITS headers. The header may be indexed by keyword and, like...
Read more >
While reading file on Python, I got a UnicodeDecodeError ...
Coding example for the question While reading file on Python, I got a UnicodeDecodeError. What can I do to resolve this?
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found