1. Background and Motivation

Research and development organisations work with distributed knowledge from requirements, test cases, defect reports, product structures, simulations, customer feedback, and technical documentation.

Large language models can improve access to this knowledge but may struggle with domain-specific accuracy, traceability, information currency, and the integration of relationships across different engineering artefacts. Traditional retrieval-augmented generation addresses some of these limitations by providing relevant documents or text passages as additional context.

Ontology- and knowledge-graph-based approaches additionally represent entities, concepts, and relationships explicitly. They may therefore provide advantages for complex domain contexts, cross-document knowledge integration, and multi-step reasoning. However, constructing and maintaining ontologies and knowledge graphs requires considerable effort. Existing research also indicates that GraphRAG does not outperform conventional RAG for every type of task.

A systematic evaluation is needed to determine in which R&D applications graph-based approaches provide measurable value and when their additional complexity is not justified.

2. Objective

The objective of this thesis is to evaluate whether ontology- and knowledge-graph-enhanced LLM applications improve quality, traceability, and reliability compared with direct LLM usage and conventional RAG.

Potential R&D use cases include:

• analysing test cases and test coverage,
• identifying inconsistent requirements,
• deriving missing tests from customer feedback,
• linking requirements, defects, and product components, and
• searching across distributed engineering documentation.

The thesis should develop and experimentally compare three prototypes:

1. direct LLM usage,
2. conventional document- or vector-based RAG, and
3. ontology- or knowledge-graph-enhanced RAG.

Suitable evaluation criteria may include:

• answer correctness,
• source attribution and traceability,
• hallucination rate,
• coverage of relevant information,
• performance on multi-hop questions,
• processing time,
• user acceptance, and
• implementation and operating costs.

The experiments should use identical tasks, source data, and evaluation procedures wherever possible. Quantitative evaluation may be supplemented by expert assessments or user studies.

A cost-benefit analysis should compare the measurable improvements with the effort required for ontology development, data integration, graph construction, and maintenance.

The expected result is an evaluation framework that identifies:

• which R&D use cases benefit substantially from knowledge graphs,
• which characteristics make a use case suitable for GraphRAG,
• whether improvements justify the additional implementation effort, and
• when direct LLM usage or conventional RAG remains the preferable solution.

3. Profile

Suitable for students in information systems, computer science, data science, artificial intelligence, industrial engineering, software engineering, or related disciplines.

Applicants should have:

• strong analytical and technical skills,
• an interest in LLMs and knowledge management,
• experience with Python and data processing,
• basic knowledge of RAG, databases, or knowledge graphs, and
• an interest in experimental design and performance evaluation.

Experience with graph databases, ontologies, semantic technologies, or LLM application development is beneficial.

4. Contact

Prof. Dr. Clemens van Dinther

3DSE Management Consultants GmbH

Email: C vanDinther ∂does-not-exist.3dse de

5. Starting Literature

• Lewis et al. (2020): “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks.”
• Pan et al. (2023): “Unifying Large Language Models and Knowledge Graphs: A Roadmap.”
• Edge et al. (2024): “From Local to Global: A Graph RAG Approach to Query-Focused Summarization.”
• Han et al. (2025): “Retrieval-Augmented Generation with Graphs (GraphRAG).”
• Han et al. (2025): “RAG vs. GraphRAG: A Systematic Evaluation and Key Insights.”
• Xiang et al. (2025/2026): “When to Use Graphs in RAG: A Comprehensive Analysis for Graph Retrieval-Augmented Generation.”