Can't load model estimater after training

I was trying to follow the Sagemaker instructions here to load the model I just trained and test an estimation. I get the error message: NotImplementedError: Creating model with HuggingFace training job is not supported. Can someone share some sample code to run to do this? Here is the basic thing I am trying to do:

from sagemaker.estimator import Estimator

# job which is going to be attached to the estimator
old_training_job_name='huggingface-sdk-extension-2021-04-02-19-10-00-242'

# attach old training job
huggingface_estimator_loaded = Estimator.attach(old_training_job_name)

# get model output s3 from training job
testModel = huggingface_estimator_loaded.model_data

ner_classifier = huggingface_estimator.deploy(initial_instance_count=1, instance_type='ml.m4.xlarge')

I also tried some things with .deploy() and endpoints but didn’t have any luck there either.

[C24IO]

Hey @gwc4github you would have to implement a model_loading and inference handler for this to get setup within a SageMaker Endpoint. Would you please mind sharing the Framework (TF/PyTorch), version, CPU/GPU for your usecase. I can send you the recipe for writing a model_handler post that.

Here is how it will look like, from within SageMaker endpoint:

    self.manifest = ctx.manifest
    properties = ctx.system_properties
    self.device = 'cpu'
    model_dir = properties.get('model_dir')
    
    #print('model_dir ' + model_dir)
    self.model = RobertaModel.from_pretrained(model_dir)
    self.tokenizer = RobertaTokenizerFast.from_pretrained(model_dir)

[gwc4github]

Thanks for the update Philipp! I’ll take a look!