Skip to content

Commit

Permalink
Change benchmark ingest took metric to total time (#268)
Browse files Browse the repository at this point in the history
Measuring total took time will give a better picture of how long it
will take to make a particular set of documents searchable.

Signed-off-by: John Mazanec <jmazane@amazon.com>
  • Loading branch information
jmazanec15 authored Jan 26, 2022
1 parent 34e02cb commit 6e859f5
Show file tree
Hide file tree
Showing 2 changed files with 2 additions and 7 deletions.
2 changes: 1 addition & 1 deletion benchmarks/perf-tool/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -226,7 +226,7 @@ Ingests a dataset of vectors into the cluster.

| Metric Name | Description | Unit |
| ----------- | ----------- | ----------- |
| took | Took times returned per bulk request aggregated as total, p50, p90 and p99 (when applicable). Note - this number does not mean the time it took to made the vectors searchable. Time from the refresh step should be looked at as well to determine this.| ms |
| took | Total time to ingest the dataset into the index.| ms |

#### query

Expand Down
7 changes: 1 addition & 6 deletions benchmarks/perf-tool/okpt/test/steps/steps.py
Original file line number Diff line number Diff line change
Expand Up @@ -279,7 +279,6 @@ def __init__(self, step_config: StepConfig):
Context.INDEX)

def _action(self):
results = {}

def action(doc_id):
return {'index': {'_index': self.index_name, '_id': doc_id}}
Expand All @@ -295,13 +294,9 @@ def action(doc_id):
index_responses.append(result)
id += self.bulk_size

results['took'] = [
float(index_response['took']) for index_response in index_responses
]

self.dataset.reset()

return results
return {}

def _get_measures(self) -> List[str]:
return ['took']
Expand Down

0 comments on commit 6e859f5

Please sign in to comment.