[Extensions] Configure a ScheduledJobParser/ScheduledJobRunner to send a parse ScheduledJobParameter/run job request to AD Extension from JS

[joshpalis]

Is your feature request related to a problem?

The ScheduledJobParser is used by the JobSweeper to parse the ScheduledJobParameter (AnomalyDetectorJob) object here. The ScheduledJobParameter is then further used to validate the job prior to scheduling. In order to support this for extensions, we must create a proxy ScheduledJobParser which shall pass the ScheduledJobParameter bytesReference and jobDocVersion back to Anomaly Detector extension to parse into an AnomalyDetectorJob.

The ScheduledJobRunner is used by the JobScheduler to invoke the runJob implementation of the dependent plugin here. The ScheduledJobRunner merely passes the ScheduledJobParameter and the JobExecutionContext back to the dependent plugin, which in turn runs the job. In order to support this for extensions, we must create a proxy ScheduledJobRunner which shall pass the JobExecutionContext back to Anomaly Detector extension to configure and run its job.

Task : To ensure consistency, update indicesToListen set for every index operation

• Add JobDetailsService as an indexOperationListener onIndexModule for .opensearch-plugin-job-details
• Post Index (getJobIndex): Pull Job Details from index and put into map with documentId
• Put jobIndexName in indicesToListen set
• Post Index (getJobType) : Pull jobType from index and put into map for documentId

Task : Whenever there is an operation on opensearch-plugin-job-details, parse the index source into a JobDetails object, extract the JobIndex and JobType, create proxies for the ScheduledJobParser and ScheduledJobRunner, and consolidate all these objects into a ScheduledJobProvider to add to indexToJobProviders.

• Pull in the indexToJobProviders hashmap from JobSchedulerPlugin into JobDetailsService
• Create method updateIndexToJobProviders which will be called via postIndex of JobDetailsService, which acts as an index operation listener to the JobDetails metadata index .opensearch-job-scheduler-job-details. The JobDetails bytesReference should be extracted from the Engine.Index parameter and passed to this method in order to update the indexToJobProviders
• Create Proxy ScheduledJobParser by overriding the parse method, extract the ScheduledJobParameter bytesRef from the provided xContentParser, and invoke the provided job_parameter_action using the NodeClient. This should return the extension's ScheduledJobParameter
• Create Proxy ScheduledJobRunner by overriding the runJob method, extract the expectedExecutionTime, jobDocVersion, jobIndexName, and jobId from the JobExecutionContext and invoke the provided job_runner_action using the NodeClient. This should return an acknowledgement from the extension
• Combine the JobDetail's jobIndex, jobType with both the proxy ScheduledJobParser and ScheduledJobRunner into a ScheduledJobProvider entry to insert into the indexToJobProviders
• Modify ExtensionJobParameter to check if jitter and lock duration seconds fields are set before passing to constructor
• Modify JobRunnerRequest to return the documentId of the ScheduledJobParameter back to the extension
• Add unit tests