Cannot load pickle file when containing instance of custom class

I cannot load a pickle file when containing instance of custom class. The error message is: ModuleNotFoundError: No module named ‘imgres’ (imgres is the module) A remark, the module (at least the class) is loaded because I can create an instance of this class (this works

I use

• Python 3.7.4
• Windows 10

I zipped everything into the attached file AZ12_FuncPickleProblem.zip

Essential folders and files in the zip

• TerminalOutput.txt contains the terminal output with the error message.
• other: folder containing script to create pickle file and example of loading it
  • imgres.py contains my class ImgRes
  • dump_pickle_file.py create pickle file
  • myimgres.pkl pickle file
  • load_pickle_file.py test loading pickle file myimgres.pkl (astonishingly there is no need for import of module, it only have to be present)
• azurefunc: folder of azure function project
  • myimgres.pkl pickle file to be read
  • HttpTriggerTest: folder with function named HttpTriggerTest
    • __init__.py source, error in myimgresread = pickle.load(f)
    • imgres.py module

What am I doing wrong? How can I solve this problem? It would be great if anybody could help!

[Hazhzeng]

Hi @HoLue,

Sorry for making you wait so long, please see the following code that should work in your scenario.

import logging

import azure.functions as func

import os
import sys
import pickle

# [Fix] Create a wrapper for importing imgres
from .imgres import *
from . import imgres

# [Fix] Register imgres into system modules
sys.modules['imgres'] = imgres

def main(req: func.HttpRequest) -> func.HttpResponse:
    logging.info('Python HTTP trigger function processed a request.')

    ir = imgres.ImgRes()
    ir.error = 'this works'
    logging.info('Can create instance of ImgRes - so module seems to be available, test: ' + ir.error)

    fn = os.path.join(os.path.dirname(__file__), 'myimgres.pkl')
    if os.path.exists(fn):
        logging.info('File ' + fn + ' does exist')
        with open(fn, 'rb') as f:
            logging.info('Reading pickle file...')
            myimgresread = pickle.load(f)
            # Terminal output after error here save in 'TerminalOutput.txt'
        logging.info('Pickle file read!')
        # show
        logging.info(myimgresread.img_fn)
        logging.info(myimgresread.error)
        # Output:
        # myfilename
        # no error
    else:
        logging.info('File does not exist')

    return func.HttpResponse(f"Success!")

Basically, the issue you’re facing is not Azure Functions specific. The pickle serializer will also record the module import path in the .pkl file. The reason why it keeps throwing Module Not Found error during unpickling is because the imgres is never registered in the system modules.

There’s a comprehensive solution for this kind of issue from the stack overflow. Unpickling python objects with a changed module path

If there’s any more question, feel free to reopen the thread. Thanks for your patience.

[teno-justos]

@Hazhzeng are you a god coming from heaven?