Provide example for InferenceGraph Ensemble voting

[asd981256]

Describe the change you'd like to see
A clear and concise description of what you want to happen.

We are deploying ensemble learning model with InferenceGraph Ensemble node, and would like to provide our python code and yaml file for document example.

This example would be helpful for understanding how to deploy another node in InferenceGraph and how ensemble node gathers parallel isvc outputs.

And, it can be adapted by anyone who has similar model output format, which is a list of probabilities of each label, to use our code directly for convenience. (v1 for now)

For example, will provide yaml and python code similar as following after we finsih our work and change our classifiers to more simple ones.

apiVersion: serving.kserve.io/v1beta1
kind: InferenceService
metadata:
  name: avg-vote
spec:
  predictor:
    containers:
      - name: avg-vote
        image: {avg-vote-image}
        args:
          - --model_name=avg-vote
---
apiVersion: serving.kserve.io/v1beta1
kind: InferenceService
metadata:
  name: dummy-classifier-1
spec:
  predictor:
    containers:
      - name: dummy-classifier-1
        image: {dummy-classifier-1-image}
        args:
          - --model_name=dummy-classifier-1
---
apiVersion: serving.kserve.io/v1beta1
kind: InferenceService
metadata:
  name: dummy-classifier-2
spec:
  predictor:
    containers:
      - name: dummy-classifier-2
        image: {dummy-classifier-2-image}
        args:
          - --model_name=dummy-classifier-2

apiVersion: "serving.kserve.io/v1alpha1"
kind: "InferenceGraph"
metadata:
  name: "ensemble-2-avg-vote"
spec:
  nodes:
    root:
      routerType: Sequence
      steps:
        - nodeName: ensemble-2
          name: ensemble-2
        - serviceName: avg-vote
          name: avg-vote
          data: $response
    ensemble-2:
      routerType: Ensemble
      steps:
        - serviceName: dummy-classifier-1
          name: dummy-classifier-1
        - serviceName: dummy-classifier-2
          name: dummy-classifier-2

import argparse
from typing import Dict, Union
import numpy as np
from kserve import (
    Model,
    ModelServer,
    model_server,
    InferRequest,
    InferOutput,
    InferResponse,
    logging,
)
from kserve.errors import InvalidInput
from kserve.model_server import app
from kserve.utils.utils import get_predict_response

class AvgVoting(Model):
    def __init__(self, name: str):
       super().__init__(name)
       self.model = None
       self.ready = False
       self.load()

    def load(self):
        self.ready = True

    def predict(self, payload: Union[Dict, InferRequest], headers: Dict[str, str] = None) -> Union[Dict, InferResponse]:
        # assume payload as ensemble node output: {"sklearn-iris":{"predictions":[1,1]},"xgboost-iris":{"predictions":[1,1]}}
        tmp = []
        for isvcName, output in payload.items():
          prediction = output['predictions']
          tmp.append(prediction)

        result = [sum(x)/len(tmp) for x in zip(*tmp)] # assume same number of label
        return get_predict_response(payload, result, self.name)

parser = argparse.ArgumentParser(parents=[model_server.parser])
args, _ = parser.parse_known_args()

if __name__ == "__main__":
    if args.configure_logging:
       logging.configure_logging(args.log_config_file)

    model = AvgVoting(args.model_name)
    app.add_middleware(
        CORSMiddleware,
        allow_origins=["*"],
        allow_credentials=True,
        allow_methods=["*"],
        allow_headers=["*"],
    )
    ModelServer().start([model])

Additional context
Add any other context or screenshots about the feature request here.
Should be related to #288
any suggestion or telling somebody already doing this that i didnt notice would be nice.