Page History
...
Description: Hybrid Retrieval combines the strengths of knowledge graphs and dense vector representations to improve information retrieval. This approach leverages the structured, relational data from knowledge graphs and the semantic similarity captured by dense vectors, resulting in enhanced retrieval capabilities. Hybrid retrieval systems can improve semantic understanding and contextual insights while addressing scalability and integration complexity challenges.
- First, the user submits a query that is analyzed to select which retrieval approach (1.*) (Arbitrator or Classification)
- The retrieval components are called either in parallel or sequentially (Hybrid Retrieval Process)
- Vector Search: Retrieves data based on vector embeddings
- Keyword Search: Retrieves data based on keyword matching
- Graph Queries: Retrieves structured data from the knowledge graph
- Then, we combine results from all retrieval methods. (Result Integration)
- Response Generation: LLM generates and delivers the response.
Considerations:
- Requires efficient result fusion techniques
- Addresses diverse data types and sources
- Increase in latency of response
REFERENCE TO BE REMOVED
- Dense and sparse vectors (https://infiniflow.org/blog/best-hybrid-search-solution, https://aclanthology.org/2023.findings-acl.679.pdf)
- Hybrid Retrieval (https://arxiv.org/html/2408.05141v1, https://haystack.deepset.ai/blog/hybrid-retrieval, https://arxiv.org/pdf/1905.07129)
- Graph Embeddings (https://www.dfki.de/~declerck/semdeep-4/papers/SemDeep-4_paper_2.pdf, https://arxiv.org/pdf/1711.11231)
- Re-ranking, scoring, and filtering by fusion (https://www.elastic.co/blog/improving-information-retrieval-elastic-stack-hybrid, https://arxiv.org/pdf/2004.12832, https://arxiv.org/pdf/2009.07258)
- Integration of KG with dense vectors (https://github.com/InternLM/HuixiangDou)
- Benefits (enhance semantic understanding, contextual and structure insights, improve retrieval accuracy)
- Challenges (scalability, integration complexity) https://ragaboutit.com/how-to-build-a-jit-hybrid-graph-rag-with-code-tutorial/
...
