Right now, there are several flavors of large language models on the market from Google, OpenAI, xAI, Anthropic, and so on. One such company is Meta, and it has its LlamaV2 7B language model. Well, a company called Deep Keep did a risk assessment of the LlamaV2 7B language model, and it discovered some concerning facts along with a high tendency to hallucinate.
Deep Keep is a leading provider of AI-native Trust and risk security management (TRiSM). It’s a company dedicated to assessing risks related to AI models. The company’s AI security “safeguards machine learning Pipelines in order to promote unbiased, error-free, secure, and trustworthy AI Solutions.”
The LlamaV2 7B LLM may be highly susceptible to hallucinations
When it comes to these large language models, accuracy and security are extremely important. Many businesses are relying on these large language models to be as accurate and secure as possible. This is why it’s a bit scary that Deep Keep’s assessment is showing some red flags.
The company discovered that LlamaV2 7B is highly susceptible to certain types of attacks including denial of service (DoS), direct prompt injection, indirect prompt injection, and PII (Personal Identifiable Information) leakage.
Prompt injection is when the user gets the LLM to produce unintended responses by manipulating the prompt. A DoS attack is when you get the LLM to use an excessive amount of tokens at once.
LlamaV2 7B is also very susceptible to hallucinations. This is when an LLM seems to fabricate facts that are not grounded in reality. This is a major issue with all AI tools on the market today.
The LLM is also moderately susceptible to attacks such as adversarial jailbreaking. It involves tricking the LLM into producing responses that violate Meta’s ethical guidelines. During testing, Deep Keep was able to see that, during such an attack, LlamaV2 7B is less likely to deny certain prompts.
If you’re using this model for any professional uses, you shouldn’t have to switch to something else. However, it’s important to know the risks associated with using it. Hopefully, Meta will be able to deliver improvements before any major attacks.
