Setup Initial APIs for Model Management Page for Deployment

[sooraj1002]

• define schema with support from @GautamR-Samagra

Models

Models <> Tasks <> Dataset

1 User
Colbert <> Agri <> UP - MLInstance
Colbert <> Fishing <> Orissa

MLWorkflow --> Change WorkflowConfig

Workflow

• Generation (Params: Prompts, Examples (Optional))

• HF Interface -- User ID (HF User ID) (Params: Dataset Definition)

• ML Model Tune -- (Params: Model, Dataset)

• ML Model Deploy -- (Params: Model, Deploy Location (GHA))

Usecase:
RLHF:

Workflow(Model, Task, Dataset)

• Generation: Generation training examples(Prompts - (Model Config), Correct Examples are taken as valid)
• HF Interface

Model [Show Dataset],[Deploy]

• /ml-workflows
  • Data: Array[]
    • Model: [Name]
    • Workflow Metadata: [ID, Desc, Name, Last Tuned, Last Deployed]
    • Dataset[]: [Last Updated, Name, Desc]
• /deploy {workdlow-identifier: Names, IDs}
• /dataset {workdlow-identifier: Names, IDs}

LS
Bots <> Task <> Telemetry Event --> {LS Element: [Config]}, {Add to Dataset}

Model, Dataset === MLWorkflow === Task : Telemetry Event

[sooraj1002]

Get the config:

curl --location 'localhost:8000/v1/workflow/models' \
--header 'user-id: 2e822dd0-6920-410a-a123-d7eac029a3a0' \
--header 'role: user'

response:

[
    {
        "id": "067a985a-7213-4ce2-86ff-6a87d161a737",
        "created_at": "2024-07-01T06:53:02.460183Z",
        "updated_at": "2024-07-01T06:53:02.460205Z",
        "name": "BERT",
        "huggingface_id": "SamagraDataGov/e2e-test",
        "last_trained": null,
        "latest_commit_hash": null,
        "is_trained_at_autotune": false,
        "is_locally_cached": false,
        "label_studio_element": {
            "name": "Text Classification",
            "config": {
                "choices": [
                    "pest",
                    "agricultural_scheme",
                    "agriculture",
                    "seed",
                    "weather",
                    "price",
                    "non_agri"
                ]
            }
        },
        "telemetry_data_field": {
            "input": "query",
            "output": null
        },
        "deployed_at": null,
        "task": "text_classification",
        "trained_on": null,
        "config": "53185125-22f7-424a-af3e-25aab48a1ca8",
        "user": "2e822dd0-6920-410a-a123-d7eac029a3a0"
    },
    {
        "id": "60e578d7-c74f-45ba-897a-7573b61c6bae",
        "created_at": "2024-07-01T06:53:03.015595Z",
        "updated_at": "2024-07-01T06:53:03.015621Z",
        "name": "distilbert-finetuned",
        "huggingface_id": "",
        "last_trained": null,
        "latest_commit_hash": null,
        "is_trained_at_autotune": false,
        "is_locally_cached": false,
        "label_studio_element": {
            "name": "Named Entity Recognition",
            "config": {
                "labels": [
                    {
                        "name": "pest",
                        "value": "pest"
                    },
                    {
                        "name": "crop",
                        "value": "crop"
                    },
                    {
                        "name": "seed_type",
                        "value": "seed_type"
                    },
                    {
                        "name": "email",
                        "value": "email"
                    },
                    {
                        "name": "phone_number",
                        "value": "phone_number"
                    },
                    {
                        "name": "time",
                        "value": "time"
                    },
                    {
                        "name": "date",
                        "value": "date"
                    }
                ]
            }
        },
        "telemetry_data_field": {
            "input": "query",
            "output": "NER"
        },
        "deployed_at": null,
        "task": "NER",
        "trained_on": null,
        "config": "0b39f453-d0ba-4ea8-9270-770a7508079d",
        "user": "2e822dd0-6920-410a-a123-d7eac029a3a0"
    },
    {
        "id": "538cfaff-d18d-4c9e-a553-7e929dcca528",
        "created_at": "2024-07-01T06:53:03.647402Z",
        "updated_at": "2024-07-01T06:53:03.647419Z",
        "name": "FCoref",
        "huggingface_id": "",
        "last_trained": null,
        "latest_commit_hash": null,
        "is_trained_at_autotune": false,
        "is_locally_cached": false,
        "label_studio_element": {
            "name": "Translation",
            "config": {
                "leftHeader": "Read the previous conversation",
                "rightHeader": "Provide coreferenced text",
                "leftTextAreaName": "Previous conversation",
                "rightTextAreaName": "Coreferenced text"
            }
        },
        "telemetry_data_field": {
            "input": "query",
            "output": "coreferencedText"
        },
        "deployed_at": null,
        "task": "Neural Coreference",
        "trained_on": null,
        "config": "ea6a0932-655d-4f0d-8229-2a38e37d66ad",
        "user": "2e822dd0-6920-410a-a123-d7eac029a3a0"
    }
]

[sooraj1002]

iteration API

curl --location 'localhost:8000/model/iterate/' \
--header 'user-id: d4017ab8-4348-4cd2-94ae-5317e98dd9aa' \
--header 'role: user' \
--header 'Content-Type: application/json' \
--data '{
    "task_type":"text_classification",
    "dataset":"SamagraDataGov/akai_text_01",
    "input":"what is the weather in puri", 
    "output":"weather"
}
'

Response:

[
    {
        "label": "weather",
        "sentence": "what is the weather in bhubaneswar"
    },
    {
        "label": "weather",
        "sentence": "what is the weather in cuttack today"
    },
    {
        "label": "weather",
        "sentence": "weather forecast for balasore"
    },
    {
        "label": "weather",
        "sentence": "show me the weather in baripada"
    },
    {
        "label": "weather",
        "sentence": "tell me the weather in rourkela"
    },
    {
        "label": "weather",
        "sentence": "is it going to rain in kendrapara"
    },
    {
        "label": "weather",
        "sentence": "today's weather update for sambalpur"
    },
    {
        "label": "weather",
        "sentence": "what's the weather like in jagatsinghpur"
    },
    {
        "label": "weather",
        "sentence": "weather conditions in nuapada"
    },
    {
        "label": "weather",
        "sentence": "weather report for jharsuguda"
    }
]

[sooraj1002]

POST records - updated for multiple records

curl --location 'localhost:8000/datasets/' \
--header 'user-id: d4017ab8-4348-4cd2-94ae-5317e98dd9aa' \
--header 'role: user' \
--header 'Content-Type: application/json' \
--data '{
    "task_type":"text_classification",
    "dataset":"SamagraDataGov/akai_text_01",
    "data":[
        {
            "input":"abc",  
            "output":"hehee"
        },
        {
            "input":"sample2",
            "output":"output22"
        }
    ]
}'

response:

{
    "message": "Dataset data saved successfully.",
    "workflow_id": "5f6922e1-bac7-4f55-8a80-200602c81858",
    "dataset_id": "638b3871-3851-492f-92c7-f9087dc8d583"
}