Page History
...
Draft from Daniel Burkhardt
Problem Statement (one paragraph)
Short definition/description of this topic: KG-Enhanced LLM Interpretability refers to the use of knowledge graphs to improve the transparency and explainability of large LLMs. By integrating structured knowledge from KGs, LLMs can generate more interpretable outputs, providing justifications and factual accuracy checks for their responses. This integration helps in aligning LLM-generated knowledge with factual data, enhancing trust and reliability.
...
Explanation of concepts
- Analysis of https://github.com/zjukg/KG-LLM-Papers?tab=readme-ov-file#resources-and-benchmarking
- Overview of methods for LLM probing https://ar5iv.labs.arxiv.org/html/2309.01029
- KG Alignment
- KG-guided Explanation Generation
- Factuality and Verification https://arxiv.org/abs/2404.00942
Brief description of the state of the art (one paragraph)
References
Answer 1: Measuring KG Alignment in LLM Representations
...
literature: https://arxiv.org/abs/2311.06503 , https://arxiv.org/abs/2406.03746, https://arxiv.org/abs/2402.06764
Contributors:
- Daniel Burkhardt (FSTI)
- Content ...
- Content ...
- Content ...
Answer 2: KG-Guided Explanation Generation
Draft from Daniel Burkhardt:
Answer 2: KG-Guided Explanation Generation
Draft from Daniel Burkhardt:
Short definition/description of this topic: KG-Guided Explanation Generation uses knowledge graphs to provide explanations for the outputs of LLMs. By leveraging the structured data and relationships within KGs, this approach can generate detailed and contextually relevant explanations, enhancing the interpretability and transparency of LLM outputs.
...
- Daniel Burkhardt (FSTI)
- Rene Pietzsch (ECC)
- ...
- Content ...
- Content ...
- Content ...
Answer 3: KG-Based Fact-Checking and Verification
...
- Daniel Burkhardt (FSTI)
- Robert David (SWC)
- ...
Draft from Daniel Burkhardt:
...
- https://graphdb.ontotext.com/documentation/10.7/graph-path-search.html
- https://neo4j.com/docs/graph-data-science/current/algorithms/pathfinding/
- Content ...
- Content ...
- Content ...
How do I enhance LLM reasoning How do I enhance LLM reasoning through KGs? (2.3 – Answer Augmentation) – length: up to one page
...
Draft from Daniel Burkhardt:
Problem Statement (one paragraph)
Short definition/description of this topic: KG-Enhanced LLM Reasoning refers to the use of knowledge graphs to improve the reasoning capabilities of LLMs. By incorporating structured knowledge, LLMs can perform more complex reasoning tasks, such as multi-hop reasoning, where multiple pieces of information are connected to derive a conclusion.
Explanation of concepts
- Reasoning https://ieeexplore.ieee.org/abstract/document/10387715
- Domain focus https://arxiv.org/html/2404.10384v1
...
- Leo H: could also include Tool-Augmented Learnin approaches as a subtopic, e.g. "KG-enhanced generation" (querying KGs during inference for factual input or additional verification context): https://arxiv.org/abs/2302.04761, https://arxiv.org/pdf/2305.11554
- Content ...
- Content ...
- Content ...
Brief description of the state of the art (one paragraph)
References
Answer 1: KG-Guided Multi-hop Reasoning
...
- Daniel Burkhardt (FSTI)
- Daniel Baldassare (doctima)
- Please add yourself if you want to contribute ...
- ...
Draft from Daniel Burkhardt:
Short definitionShort definition/description of this topic: This involves using knowledge graphs to facilitate multi-hop reasoning, where LLMs connect multiple entities and relationships to answer complex questions. This approach enhances the reasoning depth of LLMs by providing a structured path through interconnected data points in KGs.
literature: https://neo4j.com/developer-blog/knowledge-graphs-llms-multi-hop-question-answering/, https://link.springer.com/article/10.1007/s11280-021-00911-5
...
Answer 2: KG-Based Consistency Checking in LLM Outputs
...
- Daniel Burkhardt (FSTI)
- Daniel Baldassare (doctima)
- Michael Wetzel (Coreon)...
Draft from Daniel Burkhardt:
Short definition/description of this topic: KG-Based Consistency Checking involves using knowledge graphs to ensure the consistency of LLM outputs. By comparing generated content with the structured data in KGs, this method can identify inconsistencies and improve the coherence of LLM-generated information.
- Content ...
- Content ...
- Content ...
How do I evaluate LLMs How do I evaluate LLMs through KGs? (3) – length: up to one page
Target applications that can not be evaluated with gold/reference data because the gold standard is changing over time (grounded knowledge), similarity-based methods are failing (hallucination), or to enhance test data with RAG (Bias detection).
- General Methods:
- extracting KG triples from LLM outputs and evaluate the results
- enhancing example inputs for the LLM and evaluate biased results
- When are KG needed in LLM evaluation:
- analyzing grounding capabilities of LLMs (knowledge coverage)
- analyzing hallucination of LLMs (factuality)
- analyzing inherent bias from training data
Automatic evaluation of LLMs is usually done by cleverly comparing a desired result. The desired output can be evaluated using direct matching or similarity metrics (BLEU, N-gram, ROUGE, BERTScore). However, there are various reasons why KG can be used in the evaluation to support or enhance these evaluation techniques.
Firstly, KG triplets can be extracted from the output of an LLM and then analyzed. The triplets can be compared with a KG to check factuality or knowledge coverage. Examples of this knowledge coverage would be political positions, cultural or sporting events, or current news information. Furthermore, the extracted KG triplets can be used to evaluate tasks/features where a similarity comparison of the LLM output is undesirable. This is the case for identifying and evaluating hallucinations of LLMs.
The second reason is to use KGs to enhance LLM inputs with relevant information. This method is beneficial, for example, if the goal is to use in-context learning to provide relevant information for a specific task to the LLM. In addition, planned adversarial attacks can be carried out on the LLM to uncover biases or weak points.
Both variants are explained in more detail below as examples.
Answer 1: Using KGs to Evaluate LLM Knowledge Coverage
...
- Daniel Burkhardt (FSTI)
- Daniel Baldassare (doctima)
- Fabio Barth (DFKI)
- Max Ploner (HU)
- ...
Draft from Daniel Burkhardt:
...
- Daniel Burkhardt (FSTI)
- Daniel Baldassare (doctima)
- Fabio Barth (DFKI)Max Ploner (HU)
- ...
Draft from Daniel Burkhardt:
...
