LLM-Based AI Tools in Practice
How helpful is AI? Potential and limitations of Large Language Models
AI tools based on Large Language Models (LLMs), such as ChatGPT and Copilot, are already in use in many companies. In practice, however, they do not always meet the businesses’ expectations. AI experts Gerhard Schuster and Matthias Puhr from DCCS explain where the problems are and how companies can successfully integrate language-based or generative AI tools to create added value for their business.
Large Language Models (LLM)
Large Language Models (LLMs) play an increasingly important role in today's information technology, enabling significant advances in human-machine interaction. LLM-based tools can assist people in their daily work, freeing them from repetitive tasks, and give them time for more important activities. However, there are also fundamental challenges when using them in operational environments. These include limited adaptability to specific requirements and tasks, as well as high resource demands for hardware and energy. Alternative algorithms are often available to solve problems more efficiently. Data protection is another critical factor. The EU AI Act imposes explicit limits that must be respected when using AI tools. Although there are many use cases, not all deliver the expected business value
Language-based AI models for numerous applications
OpenAI GPT-4, an advanced language processing tool, has become the most widely used AI language model. The AI market also offers many other functional tools. Claude AI delivers empathetic interactions and contextual responses. Mistral AI is recognized for cost-effective and reliable translations, summaries, and sentiment analysis. Perplexity AI supports scientific work with high accuracy and precise sourcing. DeepL delivers accurate translations in numerous languages, including features for marketing and social media content. Furthermore, AI systems are becoming increasingly multimedia-capable. Leonardo AI, for example, can generate images and videos. Search engines such as Google Bard, Bing AI, and Perplexity AI also provide enhanced results with personalized, contextual, and often more accurate results compared to traditional search engines.Expectations vs. reality
The AI hype has raised high expectations, but in practice, there are significant gaps between anticipated outcomes and the actual capabilities of intelligent tools. One common reason why AI implementations fail in companies is the approach taken. Often, companies buy an AI tool first and only then consider how it fits into their system landscape and can be used effectively. A better approach is to start with a problem or potential analysis, identify and evaluate use cases, and only then choose the right technology. Simpler AI tools may sometimes be more appropriate, as LLMs, while powerful, can be expensive and resource-intensively. Many LLMs generate excellent texts, but their factual accuracy can be difficult for non-experts to verify. Therefore, additional checks for technical, legal, and ethical correctness are often required. Similarly, using LLMs as a search or research tool may provide only moderately accurate answers. For instance, when deploying an HR chatbot, it is crucial to ensure that the answers are legally and ethically correct. LLMs often do not have access to corporate data, which limits their use in business contexts. To generate real value, effective integration of LLM tools into the company’s system and data environment is essential. Retrieval-Augmented Generation (RAG) architectures are a proven method for connecting local data with LLMs. In addition, hosting LLMs as open-source software in an in-house data center helps to improve data security.
Various use cases
LLM tools are particularly useful in industries with high text volumes and repetitive tasks. They offer significant advantages in handling repetitive content. AI applications can serve as intelligent assistants automatically answering recurring, similar questions as well as reviewing, comparing, classifying and summarizing complex tender documents. On corporate websites, AI-powered search functions or chatbots can provide quick, application-specific responses and suggest relevant products. AI applications can also streamline procurement processes by automating the creation, review, comparison, and classification of offers. In software development, LLM technology accelerates the development process through integrated coding assistants. LLMs can explain program code or assist with software modernisation to save effort. Support departments also benefit from LLM-based applications, reducing the workload by providing immediate responses to customers. Particularly useful are chatbots that provide on-demand information from manuals, work instructions, or documentation in multiple languages. However, LLM has limitations in contexts that require factual, legal, or ethical accuracy, such as automated CV scoring in recruitment or legal queries.
Generative AI is changing the workplace
Generative AI, which creates new content and ideas, is becoming increasingly important. AI-based functions support and simplify daily work by translating, recording, drafting texts, and enabling situational learning. The development of this technology increases efficiency, speeds up results, and helps employees focus on core tasks. Certain professions, such as translators, will have to adapt significantly. The fact is that AI applications are evolving rapidly, becoming more practical and offering enormous potential. Data protection concerns and user skepticism about technological advances are slowing these developments.
Opportunities for real value
The dynamic development of LLMs and related technologies makes it clear that companies cannot ignore AI. Providing employees with access to LLMs is advisable, while considering whether to use costly licensed versions, free tools, or locally hosted open-source solutions. Professional guidance on the capabilities, opportunities and limitations of AI, as well as its integration into an organization´s systems and data landscape, is critical to success. Working with an experienced digitalisation partner with broad AI knowledge is recommended. AI technology will continue to develop rapidly and become embedded in many systems and solutions. When used correctly, these technologies offer significant opportunities for companies to achieve real business value.
About the Authors:
Gerhard Schuster is an Executive Advisor specialising in strategic initiatives, helping clients leverage AI for business value by applying his extensive experience in digital transformation projects and strategic management.
Matthias Puhr is a Machine Learning Consultant who advises clients on the implementation of AI, machine learning, and data science projects. His broad technical knowledge and long-standing experience enable him to develop optimal solutions for complex challenges.