In simple terms, a large language model (LLM) is a computer program that has been fed enough examples of data to be able to recognise and interpret human language or other types of complex data. LLMs are what lies behind famous tools like ChatGPT.
LLMs are typically trained on a large text-based data pool, typically scraped from the open internet. This would include web pages and – depending on the LLM – other sources such as scientific research, books or social media posts.
The model analyses the relationship between different words and learns the probabilities of word sequences, grammar, and context from the training data. It is then possible to give the model a ‘prompt’ (for example, by asking a question), and it will provide an answer based on the relationship of the words it has learned during training. Usually, the model data is static after it has been trained, although it can be refined by ‘fine-tuning’ (which is training on additional data) and ‘prompt augmentation’ (which is providing context information about the question).
What can LLMs do?
- Automation: generative AI can automate the creation of many tasks, such as by quickly making written documents or by data generation
- Simplifying and Translating: LLMs can help people to better or more quickly understand complex blocks of text and complex concepts or even text written in another language.
- Scalability and versatility: as the pool of data used by LLMs increases the ability of LLMs to provide better responses improves, such as by providing more accurate results.
Common problems when using LLMs
- Incorrect information: LLMs can provide false information. Because they work by using patterns and connections instead of actually understanding things like a person, they can give answers that are not correct. For anything important, such as fact-finding or research, people using LLMs should be aware that they can create false information. It is a good idea to use or compare other sources of information.
- Bias: the responses given by LLMs can often contain information that follows the political or cultural opinions of some people and not others. These biased responses can give a distorted impression of a topic or situation.
- Privacy concerns: to improve how LLMs work, data is continually added. This data could include answers to questions that have been asked in the past. If these questions had personal or sensitive information then this could be transferred and stored in the LLMs data-set. Naturally, this raises concerns about how this data might be used in the future and how secure it is from people who might use it for commercial or malicious purposes.
- High computational cost: training and deploying LLMs require a large amount of computational power and resources. Computational requirements that translate to environmental costs.
Business use of LLMs – potential vulnerabilities
Businesses that wish to consider implementing LLMs should consider the potential risks owing to the unique vulnerabilities that may arise from their use and misuse.
Prompt Injection: This manipulates a large language model (LLM) through specially crafted inputs that can lead an LLM to do unexpected things, such as by giving outputs according to the attacker’s intentions.
Sensitive Information Disclosure: LLMs may inadvertently reveal confidential data in its responses, leading to unauthorised data access, privacy violations, and security breaches. There is no guarantee that any information entered as part of a query won’t be fed back to another user at a later date.
Supply-chain vulnerabilities: LLM application lifecycle can be compromised by vulnerable components or services, leading to security attacks. Using third-party datasets, pre- trained models, and plugins can add vulnerabilities.
Overreliance: Systems or people overly depending on LLMs without oversight may face misinformation, miscommunication, legal issues, and security vulnerabilities due to incorrect or inappropriate content generated by LLMs.
Insecure Plugin Design: LLM plugins can have insecure inputs and insufficient access control. This lack of application control makes them easier to exploit and can result in consequences like remote code execution.
Insecure Output Handling: This vulnerability occurs when an LLM output is accepted without scrutiny, exposing backend systems. Misuse may lead to severe consequences like XSS, CSRF, SSRF, privilege escalation, or remote code execution.