Exclude namespace queries to fetch container metric data

[shreyabiradar07]

Description

This PR filters out

• namespace queries

• redundant maxDate query (already executed to fetch the interval start and end time before iterating through the list of metrics)

to fetch container metric data to generate recommendations for a container based experiment to run only the relevant queries and not overload the datasource while running multiple experiments.

Fixes #1308

Type of change

• Bug fix
• New feature
• Docs update
• Breaking change (What changes might users need to make in their application due to this PR?)
• Requires DB changes

How has this been tested?

Tested by running local monitoring demos on ResourceHub cluster - ./local_monitoring_demo.sh -c openshift -i quay.io/shbirada/autotune_operator:filter-namespace-for-container-reco

pod logs with queries run for tfb-server and tfb-database containers - https://privatebin.corp.redhat.com/?d50352a3d066c26b#4mGGpw7q2TXbrF1pN6C7yUykuphywPSwstMez2RkKWvB

• New Test X
• Functional testsuite

Test Configuration

• Kubernetes clusters tested on: Openshift (ResourceHub cluster)

Checklist 🎯

• Followed coding guidelines
• Comments added
• Dependent changes merged
• Documentation updated
• Tests added or updated

Additional information

docker image - quay.io/shbirada/autotune_operator:filter-namespace-queries

[msvinaykumar]

Shekar also implemented similar filtering. Can't we create a common function and reference it statically?

Is it possible to design the metric profile with subsections to handle both namespace and container queries? In the future, we might need separate queries for different workloads, so it would be better to design the metric profile object with distinct and clear separations, rather than relying on 'startswith'.

[shreyabiradar07]

Shekar also implemented similar filtering. Can't we create a common function and reference it statically?

Added a common function for filtering metrics in both the namespace and container experiment types

[msvinaykumar]

LGTM

[dinogun]

@shreyabiradar07 Any reason why we are not using kubernetes_object to check if it is container or namespace as it is meant for exactly this purpose?

[shreyabiradar07]

@dinogun updated the function to filter metrics based on experiment_type and kubernetes_object

[dinogun]

LGTM