Query terms of the same category within the same query stage

[Spiess]

Cineast currently does not allow multiple different query terms with the same feature category within the same query stage:

cineast/cineast-api/src/main/java/org/vitrivr/cineast/api/util/QueryUtil.java

Line 148 in e5587fc

LOGGER.warn("Staged query contained the category \"{}\" multiple times.", category);

I agree that it makes little sense in many cases, such as allowing multiple sketches with the same features for known item search tasks, but I also think, that it arbitrarily restricts query flexibility.
I understand that it is possible to work around this limitation by specifically engineering features to allow this on a per-feature basis, but I don't think this is a good solution.

For example, if I want to find the media item most similar to n other media items according to the same feature, I have the options to:

1. Execute n individual queries and merge the results myself
2. Execute a query with n stages, forcing me to pick an order of importance for the media items
3. Individually fetching the feature vectors of the n media items, somehow computing a single vector per feature to represent all media items and executing a query with this

I am aware that the utility of allowing this heavily depends on the number of retrieved media items for each query term, however this is a separate issue, that affects the combination of any query terms within the same stage.

[lucaro]

I don't understand the problem. The way result processing is currently done, there is no way to benefit in having multiple terms for the same category in the same stage. There is currently also no direct way to specify that you would want elements that are the most similar to all query elements specifically. Adding 'redundant' query terms would result in you receiving results that are similar to any of your terms, which you can then aggregate such that the ones that are similar to more of them are ranked higher. In fact, that is what the default aggregation would do. If you want to enforce the similarity to all your terms, you can indeed achieve this with stages, and since if is a hard filter, the ordering of them would not be important. Could you maybe give a more specific example of what you would want to express that would benefit from such 'redundancy'?

[Spiess]

I'm not sure I understand what you mean.

If for example I want to find the most similar segments to the segments i_0001006_1 and i_0000808_1 in the category visualtextcoembedding I can make the following request:

curl -X POST "http://city-stories.dmi.unibas.ch:4567/api/v1/find/segments/similar" -H "accept: application/json" -H "Content-Type: application/json" -d '{"terms":[{"type":"ID","data":"i_0001006_1","categories":["visualtextcoembedding"]},{"type":"ID","data":"i_0000808_1","categories":["visualtextcoembedding"]}], "config": {"maxResults": 10}}'

But Cineast only returns one of the two results lists.
Now I could move the two terms to separate query stages, but that makes little conceptual sense to me, as stages are meant to impose an ordering of importance.

[lucaro]

But what is the expected behavior of this? If you are fine with this being treated as if the two terms would be evaluated by different features within the same category, then it should be reasonably straight forward to do. In that case, you would get results that match either query with the ones matching both having a potentially higher score. If you want to only have results that explicitly match both terms, then that would require some more elaborate processing.

[silvanheller]

I'm completely fine with adding this. The reason it's currently implemented that way is because this was minimally invasive. The main change would be changing some maps to lists in the query-processing, I don't see any other complications.

[lucaro]

The reason it was originally implemented like this is since there was the assumption that within one term, the match to two independent terms would make little semantic sense. Consider trying to match an image to two different sketches using the same notion of similarity, that would not produce reasonable results, at least not in a classical retrieval context. As long as the intended semantic of such a query is clearly defined and agreed upon, I don't see a reason why we would not want to allow for such a thing though, as there might be terms or features for which legitimate use cases might exist. This might however necessitate some changes to the score fusion mechanism, such that it correctly deals with such cases.