Fail to index table with iloc[] after dump/load table into pickle file

EDIT: Temporary Workaround

See https://github.com/astropy/astropy/issues/11332#issuecomment-781561392

Description

I am saving and loading a Table object into a pickle file, but after loading the table cannot be indexed using the iloc method. I found that problem is because after loading the table doesn’t the attribute Table.primary_key

Steps to Reproduce

from astropy.table import Table
import pickle

t = Table([(1, 2, 3, 4), (10, 1, 9, 9)], names=('a', 'b'), dtype=['i8', 'i8'])
t.add_index('a')
print(t.iloc[2])   #--> [3, 9]

with open("t_test.pkl" , "wb") as f:
    pickle.dump(t, f, protocol=0)
    
t_ = pickle.load(open("t_test.pkl", "rb"))

print(t_.iloc[2])

leads to the following error:

-----------------------------
TypeErrorTraceback (most recent call last)
<ipython-input-173-7e3638d00d00> in <module>
     11 t_ = pickle.load(open("t_test.pkl", "rb"))
     12 
---> 13 print(t_.iloc[2])

~/.pyenv/versions/adap/lib/python3.8/site-packages/astropy/table/index.py in __getitem__(self, item)
    953         else:
    954             key = self.table.primary_key
--> 955         index = self.indices[key]
    956         rows = index.sorted_data()[item]
    957         table_slice = self.table[rows]

~/.pyenv/versions/adap/lib/python3.8/site-packages/astropy/table/index.py in __getitem__(self, item)
    809             raise IndexError(f"No index found for {item}")
    810 
--> 811         return super().__getitem__(item)
    812 
    813 

TypeError: list indices must be integers or slices, not NoneType

and here when accessing the primary_key attribute of the original and post-pickle table:

t.primary_key, t_.primary_key
# (('a',), None)

System Details

macOS-10.15.7-x86_64-i386-64bit Python 3.8.6 (default, Jan 5 2021, 15:15:33) [Clang 12.0.0 (clang-1200.0.32.28)] Numpy 1.19.5 astropy 4.2 Scipy 1.6.0 Matplotlib 3.3.3

[taldcroft]

@pllim - I must have made a mistake in my test because I am reproducing the failure you reported. The oddity is that it only fails if t has had an index created. Based on my recollection of how pickling works for tables this makes no sense, but obviously my recollection is not serving me well. 😄

Here is my minimum failing example:

In [20]: from astropy.table import Table                                                            
In [21]: import pickle                                                                              
In [22]: t = Table([[1,2], [3,4]])                                                                  
In [23]: t.add_index('col0')                                                                        
In [24]: t2 = pickle.loads(pickle.dumps(t))                                                         
In [25]: t2.loc[1]                                                                                  
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
...
TypeError: list indices must be integers or slices, not NoneType

[Meistermagier]

Im not sure if this has something todo with it but this problem also happens when you create a new TimeSeries out of an old one like so ts2 = ts["time","Test"] ts2.iloc[:] this leads to the same error. I have figured out that for some reason when doing this it doesnt assign a new primary key so if you set that by hand things work but before its problematic. Hope this helps if its not helpfull to this conversation then i apologize it just seemed like a closely related problem.