1. Background and Motivation

Many companies are investing in AI applications for product development, engineering, software development, requirements management, testing, and knowledge management. At the same time, it often remains unclear whether these investments result in measurable productivity improvements, shorter lead times, higher quality, or financial benefits.

Experimental research has demonstrated that Generative AI can improve performance for selected knowledge-work tasks. For example, measurable improvements have been found in professional writing and customer-service settings. However, these effects cannot automatically be transferred to complex development organizations. The impact of AI depends on the task, the quality of implementation, user capabilities, process integration, data availability, and complementary organizational changes.

A distinction must therefore be made between access to AI tools, actual use, integration into workflows, operational performance improvements, and realized financial value. Organizations may report time savings without converting them into additional output, lower cost, improved quality, or shorter time-to-market. Moreover, traditional return-on-investment calculations often fail to capture indirect effects such as avoided rework, faster learning, improved knowledge availability, or increased development capacity.

This thesis addresses the need for a robust and practical approach to measuring the organizational and economic value of AI in development environments.

2. Objective

The objective of this thesis is to develop and empirically test a benchmark model for measuring the business value of AI in development organizations.

The central research question is:

How can the business value of AI in development organizations be measured systematically, and what relationships exist between AI investments, adoption maturity, productivity, and realized financial value?

Supporting research questions include:

• Which input, maturity, usage, performance, and financial indicators are suitable for assessing AI value in development organizations?
• How do companies currently quantify AI investments and benefits?
• Do organizations measure AI benefits in monetary terms, and which assumptions and allocation mechanisms do they use?
• Is higher AI investment associated with greater adoption maturity, productivity, or financial value?
• Does process integration mediate the relationship between AI investment and business value?
• Which types of organizations and AI use cases exhibit similar investment and value patterns?
• Which contextual factors must be considered when comparing organizations?

The benchmark model may cover the following dimensions:

1. AI investment
   • software licenses and model usage,
   • infrastructure and integration cost,
   • internal development resources,
   • data preparation,
   • training and change-management expenditure.
2. Use-case portfolio
   • number and type of AI use cases,
   • development functions covered,
   • user groups and frequency of use,
   • degree of deployment and scalability.
3. Organizational and technological maturity
   • process integration,
   • data availability and quality,
   • tool and platform landscape,
   • governance and accountability,
   • user competencies, adoption, and acceptance.
4. Operational performance
   • engineering hours per output,
   • cycle and lead times,
   • throughput,
   • rework and defect rates,
   • quality and compliance indicators,
   • knowledge-access and decision-making time.
5. Product and business outcomes
   • time-to-market,
   • development cost,
   • capacity gains,
   • avoided cost,
   • quality-related savings,
   • additional revenue or margin,
   • return on investment and payback period.

A Mixed-Methods approach is recommended:

• First, a KPI and maturity model should be derived from the literature and expert interviews.
• Second, a quantitative survey or benchmark study should be conducted with development organizations.
• The data may be analyzed using descriptive statistics, correlation and regression analyses, cluster analysis, or maturity-based comparisons.
• Where sufficient longitudinal or control-group data are available, quasi-experimental approaches may be considered.
• The benchmark model should be validated with practitioners regarding relevance, measurability, comparability, and decision usefulness.

The analysis should explicitly distinguish correlation from causality. Higher-performing organizations may invest more in AI, while higher AI investment may also contribute to better performance. Variables such as organization size, industry, development complexity, prior digital maturity, and use-case type should therefore be considered.

The expected results are:

1. a structured AI Business Value Benchmark Model for development organizations,
2. a defined set of investment, maturity, productivity, quality, and financial KPIs,
3. a survey and benchmarking instrument,
4. an empirical analysis of relationships between investment, maturity, usage, and value,
5. a classification of organizations or use cases into relevant benchmark clusters, and
6. recommendations for establishing a reliable AI value-measurement system.

The final model should help organizations assess AI investments not only from a technical perspective but also in terms of organizational maturity and economic impact.

3. Profile

We are looking for a student in Business Administration, Information Systems, Industrial Engineering, Economics, Technology and Innovation Management, Data Science, or a comparable degree program.

The ideal candidate has:

• an interest in AI, product development, performance measurement, and business value,
• knowledge of quantitative research methods and statistical analysis,
• an interest in conducting expert interviews and organizational surveys,
• basic knowledge of productivity measurement, investment appraisal, or management accounting,
• experience with statistical tools such as R, Python, SPSS, or comparable software as an advantage,
• the ability to work with potentially incomplete and confidential organizational data,
• a structured and analytically rigorous working style, and
• an interest in developing a practical benchmarking instrument for industrial application.

4. Contact

Prof. Dr. Clemens van Dinther

3DSE Management Consultants GmbH

c.vandinther∂3dse.de

5. Starting Literature

• Brynjolfsson, E.; Li, D.; Raymond, L. (2025): Generative AI at Work. The Quarterly Journal of Economics, 140(2), 889–942.
• Noy, S.; Zhang, W. (2023): Experimental Evidence on the Productivity Effects of Generative Artificial Intelligence.Science, 381(6654), 187–192.
• Dell’Acqua, F. et al. (2026): Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of Artificial Intelligence on Knowledge Worker Productivity and Quality. Organization Science, 37(2), 403–423.
• Mikalef, P.; Gupta, M. (2021): Artificial Intelligence Capability: Conceptualization, Measurement Calibration, and Empirical Study on Its Impact on Organizational Creativity and Firm Performance. Information & Management, 58(3), 103434.
• Mooney, J. G.; Gurbaxani, V.; Kraemer, K. L. (1996): A Process-Oriented Framework for Assessing the Business Value of Information Technology. ACM SIGMIS Database, 27(2), 68–81.
• Schryen, G. (2013): Revisiting IS Business Value Research: What We Already Know, What We Still Need to Know, and How We Can Get There. European Journal of Information Systems, 22(2), 139–169.
• Jöhnk, J.; Weißert, M.; Wyrtki, K. (2021): Ready or Not, AI Comes—An Interview Study of Organizational AI Readiness Factors. Business & Information Systems Engineering, 63(1), 5–20.
• McKinsey & Company (2025): The State of AI: How Organizations Are Rewiring to Capture Value.
• Becker, J.; Knackstedt, R.; Pöppelbuß, J. (2009): Developing Maturity Models for IT Management—A Procedure Model and Its Application. Business & Information Systems Engineering, 1(3), 213–222.
• Peffers, K. et al. (2007): A Design Science Research Methodology for Information Systems Research. Journal of Management Information Systems, 24(3), 45–77.