add `ast.literal.eval()` to documentation

I have created my custom handler, taking the default base_handler.py and adding just few lines, for my signal vs background discrimination use-case. So in my use-case I don’t want to make inference on images but simply on lists of floats. From what I understood the entry point for the default handler is the handle function, which takes as argument data defined as data (list): The input data that needs to be made a prediction request on. Because my input data is a json file where is stored a specific event in the form {"keys":["key_1", "key_2"], "values":[value_1, value_2]}, how can I properly make inference? Firstly I added file=json.load(open(data)) at the beginning of the handle function and I launched in the terminal a command like this curl http://localhost:8080/predictions/pytorch_physics_3 -T predict_signal.json and I got this error TypeError: expected str, bytes or os.PathLike object, not list. I tried multiple ways (e.g. use data[0], passing directly the list of floats of the event) but it seems that I didn’t properly understand what is data and how I can use the json file passed during the request of inference. In any case, if I use the local path of the json file instead of using data in file=json.load(open(data)) all the pipeline works well giving back the correct result of prediction.

Thanks.

[vkuznet]

@msaroufim what is a data-type of data object passed to preprocess method of BaseHandler class. In other words can someone clearly describe the data flow and data-format from HTTP POST request (with JSON payload) to BaseHandler class and define what data means. From the code I read the following definition:

    def preprocess(self, data):
        """
        Preprocess function to convert the request input to a tensor(Torchserve supported format).
        The user needs to override to customize the pre-processing
        Args :
            data (list): List of the data from the request input.
        Returns:
            tensor: Returns the tensor data of the input
        """
        return torch.as_tensor(data, device=self.device)

Does it means that data input is a python list of dict objects? Or, is it list of integers, floats, or is it list of tensor inputs? Next, does the preprocess function is called upon HTTP POST request? If, so where is it done?

To understand the problem we need to know the internals of the data flow inside torch_serve and the actual data format of the expected input.

If I look at the unit test code it defines on line 22 handle_fn which initialize BaseHandler. Then, I can see in various unit tests that the actual JSON is passed as

[{'body':{
        'instances': [[1.0, 2.0], [4.0, 3.0]]
    }}]

e.g, see line 30, 40, 54. This leads me to conclude that actual JSON structure expected by torch_serve should be presented as list data type, with a dict which contains body key which by itself has instnaces key which point to a list of data. Something like: [{"body": {"instances":[list1, list2]}}] where list1 and list2 are values. Is it correct evaluation? If so, where the input data format is defined? Please note that this unit test also define more complex cases where multiple body records can be supplied, e.g. see line 54, and where body itself can have values, see line 68. Moreover, someone can pass just instnaces record with binary encoded input, see line 78.

@lgiommi I suggest to try out examples from a unit test and adjust your input accordingly.

To all, I also suggest to properly document the input/output data formats, the expected data-types to avoid frustration from end-users to deal with internal code organization.

[msaroufim]

Thank you for your patience @lgiommi, changed the title of your issue to make this improvement