Search analytics #6019
Search analytics #6019
Conversation
dojutsu-user
added
the
PR: work in progress
readthedocs/search/models.py
Outdated
| _('Query'), | ||
| max_length=4092, | ||
| ) | ||
| count = models.PositiveIntegerField( |
There was a problem hiding this comment.
I'm wondering if we want more data here. Should we be storing an object each time a search happens? That way we can show the frequency of a search over time. Currently, this only tells us how many times a search has happened.
I think if we plan to delete the data every 3 months, we can probably store every search query with it's own timestamp. I'm fine with shipping this initially though, before we start storing a lot more data.
There was a problem hiding this comment.
I am not sure what you meant be more data.
Storing search object everytime a search was made is a better idea. I realised that the graphs were wrong before. And going this way makes them correct and easier.
That way we can show the frequency of a search over time.
Can you exapand this more?
- Do we want this to be selected by user? Like the user can select a date and we show him the frequency of searches made vs time for that day.
- Or we just show this for today/yesterday?
There was a problem hiding this comment.
Just that we will be able to see when a specific search was done each time it was done. The current modeling only shows the number of times a search was done, but no time data about each search.
|
@ericholscher Screenshot - https://ibb.co/xzX54Ty |
dojutsu-user
removed
the
PR: work in progress
| project_slug | ||
| ) | ||
| # data for plotting the doughnut-chart | ||
| distribution_of_top_queries = SearchQuery.generate_distribution_of_top_queries( |
There was a problem hiding this comment.
I'm a little worried this will be slow in production after we have a lot of data, but we can deal with it then.
|
|
||
| response = HttpResponse(content_type='text/csv') | ||
| response['Content-Disposition'] = f'attachment; filename="{file_name}"' | ||
| template = loader.get_template('projects/search_analytics/csv_data_template.txt') |
There was a problem hiding this comment.
Why are we writing this with a template instead of a CSV library?
| verbose_name=_('Version'), | ||
| related_name='search_queries', | ||
| on_delete=models.CASCADE, | ||
| ) |
There was a problem hiding this comment.
Not sure if we really even want to cascade these deletes. Is there a reason we don't want to store Version here as a string, so we can keep them forever even if a version is deleted?
| .order_by('created_date') | ||
| .annotate(count=Count('id')) | ||
| .values_list('created_date', 'count') | ||
| ) |
There was a problem hiding this comment.
This looks really slow. We will see in prod, hopefully it won't be an issue.
|
|
||
|
|
||
| @app.task(queue='web') | ||
| def record_search_query(project_slug, version_slug, query, total_results): |
There was a problem hiding this comment.
Hrm yea, that seems less than ideal. We should probably think more about the right approach for "search as you type" -- probably adding Autocomplete vs. search as you type in some cases.
|
|
||
| project_qs = Project.objects.filter(slug=project_slug) | ||
|
|
||
| if not project_qs.exists(): |
There was a problem hiding this comment.
This should probably log a warning.
| @@ -0,0 +1,3 @@ | |||
| serial_no,date_time,query | |||
| {% for row in data %}{{ forloop.counter }},"{{ row.0|addslashes }}","{{ row.1|addslashes }}" | |||
| {% endfor %} | |||
There was a problem hiding this comment.
Definitely we should use the csv library for this.
Closes #5967
WIP