Despite a meteoric rise, in the wake of DeepSeek models like R1 and V3 is a growing number of research highlighting the AI model’s safety measures, or rather, a lack thereof.
In the past week, researchers from Cisco and the University of Pennsylvania found DeepSeek R1 failed to block a single harmful prompt during safety tests, while evaluations by security platform Enkrypt AI uncovered the Chinese AI model was 11 times more likely to generate harmful output compared to OpenAI’s o1.
Meanwhile, the Australian government became the first in the world to ban all government devices from accessing the DeepSeek app over national security fears.
The myriad of tests and the ban in Australia came just days after the first rules of the EU’s AI Act, the continent’s comprehensive regulation governing the use of AI technologies and their impact on citizens’ rights, came into force on 2 February.
New research came to light this week from LatticeFlow AI, showing DeepSeek’s flagship R1 model, in its current form, likely wouldn’t be compliant under the AI Act’s rules for language models due to subpar cybersecurity, bias, and robustness.
Subscribe today for free
Using COMPL-AI, a compliance readiness tool for the AI Act developed with ETH Zurich and INSAIT, LatticeFlow identified weaknesses in distilled versions of DeepSeek R1 — models designed to be smaller yet more efficient, including DeepSeek-R1-Distill-Llama-8B and DeepSeek-R1-Distill-Qwen-14B.
The evaluation ranked DeepSeek models lowest for cybersecurity, highlighting risks such as goal hijacking, where an AI veers from its intended task to pursue an unintended objective, and prompt leakage, where unintended details in a prompt influence responses in unpredictable ways.
DeepSeek also scored below average for biased outputs, exhibiting significantly more bias than its base models, despite having gone through the distillation process.
Dr Petar Tsankov, CEO and co-founder of LatticeFlow AI, told Capacity that the models “definitely would not pass compliance tests” despite there being no formal way yet to demonstrate compliance.
Tsankov said the team behind the DeepSeek models “focused mainly on capabilities and toxicity and ignored other aspects” which would be required for compliance.
“The way [DeepSeek] trained them, they did not have these explicit goals to optimise security, because the base models were trained in a secure way but they forgot how to actually be secure by the way they fine-tuned them and adapted them further.”
Why would DeepSeek not be compliant with the EU AI Act?
The protection of citizens' rights is at the core of the EU’s sweeping AI regulations.
The legislation would force all AI systems deployed in the EU to be subject to categorisation based on their likelihood to impact an individual’s rights.
AI systems deemed to pose extreme risk would be banned outright, while high-risk AI systems would be subject to strict obligations, such as risk assessments prior to deployment, adequate levels of human oversight, and requiring developers to keep detailed logs of the AI’s development.
For foundation models like DeepSeek, which generate general-purpose outputs, the AI Act introduces additional requirements to mitigate risks related to bias, misinformation, and potential manipulation.
Developers of such models must ensure transparency, maintain high-quality training datasets that do not embed systemic biases, and provide clear documentation on model capabilities and limitations.
Further, under the Act’s prohibitions, AI systems cannot be designed in ways that manipulate users or exploit their vulnerabilities — concerns that have been raised about DeepSeek’s output.
AI models deployed in the EU are also obligated not to produce outputs that violate the bloc’s fundamental rights protections, including safeguards against discrimination and disinformation.
Given that multiple evaluations of DeepSeek, including tests by Capacity, have shown it to generate propagandistic content, it would likely fall afoul of these governance rules, making its compliance with the EU AI Act highly questionable.
Tsankov highlighted that AI Act compliance is not just about AI capabilities but also security, bias, and robustness — all critical factors for enterprise adoption.
DeepSeek, he explains, performed particularly poorly in cybersecurity assessments, with vulnerabilities that could potentially expose sensitive business information.
Given its failure to meet these key compliance dimensions, its deployment within the EU under the AI Act would be highly questionable.
AWS adds DeepSeek support, but is it safe for enterprises?
While the first stage of the EU AI Act only just came into force, the transition periods for complying with various requirements range from six to 24 months.
That might seem far off, but for businesses looking at DeepSeek for potential deployments, it’s something to consider.
While anyone can download and use R1 from platforms like GitHub and Hugging Face, AWS became the first hyperscaler to extend support for DeepSeek R1 via its SageMaker JumpStart and Bedrock platforms.
AWS CEO Matt Garman said in a post that “no single model is right for every use case” — but given its safety concerns, DeepSeek AI systems may not be safe to deploy at this time.
Users of platforms like AWS Bedrock do have the added benefit of pairing the model with external guardrails like Amazon Bedrock Guardrails, Nemo Guardrails, or Azure Content Safety, to provide an extra layer of security to their applications.
Dr Stuart Battersby, chief technology officer at responsible AI software firm Chatterbox Labs, who previously uncovered severe safety weaknesses with DeepSeek models, suggested such guardrails are a great first step to improving the safety, robustness and security of DeepSeek, but “alone they are also insufficient”.
“A full safety testing process must be carried out (specific to the AI use case at hand) so that the guardrails themselves are tested, iterated upon, and demonstrated as safe,” Dr Battersby said.
“The results of this testing are then fed into the EU AI Act's conformity assessment to demonstrate the risk assessment of the system as a whole, not just the standalone model.”
Following AWS’ lead, Microsoft added support for DeepSeek R1 in its Azure AI Foundry, while Alibaba Cloud integrated DeepSeek's AI models into its PAI Model Gallery.
Dr Seth Dobrin, founder and CEO of Qantm AI and former chief AI officer of IBM, warned that even using hosted versions of DeepSeek comes with “significant privacy risks”.
“The Ts & C’s clearly state all information will be sent to China, and they will use it for almost anything they want — as a company, this is a no-go, full stop prohibited for use in my mind, regardless of where the data is being sent.”
Dobrin noted that the development and release of DeepSeek’s AI systems were a good thing for diversity of thought, adding that injecting Chinese ways of thinking with Western ways of thinking has “already shown signs of value for coding”.
“Early users are reporting completely different function structures in object-oriented languages like Python and Java that perform better,” Dobrin said, adding that private-hosted versions with well-implemented guardrails could provide an opportunity to bring a diversity of thought into the wider AI arena.
“This being an open-source model provides another avenue for developers of small models to use it as a starting point without the restrictions imposed by Meta on both the extent of the derivatives permitted as well as the amount of revenue permissible before having to give them a cut,” Dobrin added.
Is DeepSeek making changes to improve safety?
It’s a question that needs asking in the wake of repeated research highlighting pretty damning results. There are early signs that DeepSeek is making some changes, though.
Wallarm, a San Francisco-based API security firm, uncovered a jailbreak that allowed its researchers to convince DeepSeek to share restricted data about itself, how it was trained, and even policies applied to its behaviour.
The Wallarm found that once jailbroken, DeepSeek reveals references to OpenAI models, suggesting that technology or data from Microsoft-backed startup may have been used to help shape DeepSeek’s knowledge base.
OpenAI has raised concerns about whether its data was used without authorisation by the Chinese researchers — a particularly ironic given that OpenAI is currently facing multiple lawsuits from rightsholders who claim that the company used their data to train its AI models without permission.
Wallarm told Capacity that its researchers contacted DeepSeek via email, providing the startup with details of the jailbreak and an example of it working.
DeepSeek subsequently fixed the reported jailbreak “within a couple of hours,” the firm added.
While DeepSeek fixed this jailbreak, the ever-growing studies into its current safety levels show that compliance issues run far deeper than a single vulnerability.
As Tsankov pointed out, the lack of explicit security goals in DeepSeek’s training process means that its models not only fall short on compliance but could pose significant risks for enterprise adoption.
“Right now, vendors are still figuring out how to make their models compliant, but DeepSeek has focused almost entirely on capabilities while neglecting fundamental security and governance concerns,” Tsankov said.
With AI regulation tightening globally (except in the US after President Trump ripped up Biden’s executive AI order), models that fail to meet baseline safety, security, and transparency requirements may struggle to gain long-term traction, regardless of how efficient or cost-effective they claim to be.
RELATED STORIES
DeepSeek failed all safety tests, responding to harmful prompts, Cisco data reveals
DeepSeek 'highly vulnerable' to generating harmful content, study reveals
Meta’s AI chief: DeepSeek proves AI progress isn’t about chips
Behind the DeepSeek hype: Costs, safety risks & censorship explained
