python tritonclient stream_infer should send end signal to callback

this maybe a new baby question, I think we should add self._callback(result=None, error=None) after finish iteration on responses to let callback know stream response is complete and no more response will be sended. Am I right?

python triton client code

https://github.com/triton-inference-server/client/blob/d8bc5b8d4a9297e8e6b470fe96ef82a4470234a2/src/python/library/tritonclient/grpc/__init__.py#L2083

    def _process_response(self, responses):
        """Worker thread function to iterate through the response stream and
        executes the provided callbacks.

        Parameters
        ----------
        responses : iterator
            The iterator to the response from the server for the
            requests in the stream.

        """
        try:
            for response in responses:
                if self._verbose:
                    print(response)
                result = error = None
                if response.error_message != "":
                    error = InferenceServerException(msg=response.error_message)
                else:
                    result = InferResult(response.infer_response)
                self._callback(result=result, error=error)
            ############ add here ##############
            self._callback(result=None, error=None) # new code
        except grpc.RpcError as rpc_error:
            error = get_error_grpc(rpc_error)
            self._callback(result=None, error=error)

[Tabrizian]

@Jackiexiao This sounds like a reasonable request to me. I think we should deliver the flag to the client indicating whether this is the last response or not. Right now, it looks like the client must know before hand the number of responses that the model is going to send. @tanmayv25 / @jbkyang-nvi What do you think?

[dyastremsky]

Not yet. It’s in our queue.