Question from Ignite-Best practices around RAG with embeddings

[chrisbardon]

I was talking to @moonbox3 at Ignite last week, and I was asking about MS recommendations around RAG best practices, and whether there was a better approach than the basic semantic embeddings we've all been using. The embedding model is going to calculate based on a single version of semantics, but my question was whether that was reliable in all cases. Obviously something like a hybrid search improves this with things like traditional keywords (like in Azure AI Search), but the question was really whether there was a better way to use semantics in things like a classifier taxonomy, or to use multiple embedding models to extract different representations of semantics. I did see a really good session on advanced RAG and llamaindex that gave me some ideas, but the central problem still remains that a semantic embedding really only represents one thing, so the best practice seems to be to make those things as small as possible (single topic document chunks).

I've got some ideas of things to try, but don't necessarily want to reinvent the wheel, so I thought I'd ask around to see if there were any recommendations from the team first.

[moonbox3]

Tagging @westey-m for visibility.

[westey-m]

@chrisbardon, in general I haven't come across a lot of better alternatives, but here are couple of my thoughts:

1. Experiment with different approaches, including Hybrid search as well as different chunking strategies, e.g. having different chunk sizes or having overlapping chunks. Not all datasets are the same, so yours may benefit differently from these strategies to other datasets and therefore it's good to experiment and measure the results for your dataset.
2. Check out GraphRAG. It can be very expensive but will most likely continue to improve. It does give significantly better results than traditional RAG and can answer broad questions about a corpus of text that traditional RAG cannot.