Inside Falcon: The UAE’s open source model challenging AI giants

In the wake of Stable Diffusion and ChatGPT, a myriad of AI models came and went, with many causing initial ripples of excitement, only to ultimately succumb to obscurity.

One family of AI models endured, however, with the team behind it routinely iterating, making strides to take open source developments to new heights: Falcon.

Developed by the Technology Innovation Institute (TII) in the United Arab Emirates (UAE), Falcon was first unveiled back in March 2023 as a large language model (LLM) trained on 1 trillion tokens, providing researchers and SME innovators with a powerful AI tool.

The end of 2024 saw Falcon 3 take to the skies, making the original look antiquated. It surpassed rival-sized models from the likes of Meta, Google and Alibaba on industry-standard benchmark tests used to evaluate AI model performance.

TII shows no signs of slowing, with plans to continually improve Falcon with capabilities to power next-generation use cases in robotics and smart cities.

Dr Hakim Hacid, chief researcher at TII’s AI and Digital Science Research Centre, sat down with Capacity ahead of Capacity Middle East 2025 to talk about the future of Falcon and how the research lab has positively impacted AI development in the Middle East.

The first Falcon was 40 billion parameters in size, putting it over four times smaller than OpenAI’s GPT-3 model, its last pre-ChatGPT LLM.

After creating a larger 180 billion version, TII took a smaller approach with Falcons 2 and 3, creating smaller-scale models that, despite their size, vastly outperform their predecessors.

Falcon 3 comes in four model sizes: 1 billion parameters, 3 billion parameters, 7 billion parameters, and 10 billion.

Dr Hacid explained to Capacity that with newer versions of the model, the TII team sought to get the most power from the smaller models to unlock use cases closer to end users.

The Falcon 3 models are so resource-efficient and lightweight that users can run them on some laptops offline without sacrificing performance, enabling businesses to more cost-effectively run applications at the edge and individual developers to access powerful open-source systems previously reserved for higher-end industrial computing.

“Not everyone has access to high-end GPUs to run models,” Dr Hacid says. “We paid a lot of attention to offering models that can run on consumer devices so that people can access this model to run applications on their devices instead of running them in the cloud.”

The decision to go small and focus on edge applications also sees TII looking beyond what LLMs are doing today. The next generation of AI, at least according to industry leaders like Nvidia CEO Jensen Huang, is physical AI — everything from autonomous vehicles to smart cities, and even robots.

Physical AI is an area on which TII firmly has its eyes, with Dr Hacid saying: “The robot will take actions, and you don't want the request or the observation to be sent to the cloud, computed and then brought back — as by the time you get everything done, the robot may have done something wrong.

“We want to get the maximum power in the smallest possible model so that we can unlock all these use cases that are much closer to the end user. We need to go much further than the classical use cases like chatbots.”

Despite shifting sizes for later Falcon models, the TII team significantly increased the volume of data used to train its latest models, employing 14 trillion tokens for Falcon 3 compared to 5.5 trillion for Falcon 2.

Tokens, which are numerical representations of text, serve as the foundation for training large language models. However, quantity alone doesn’t equate to improved performance. As Dr Hacid explains: “The data alone is not that important if you don’t get the right quality.

“We spent a lot of time building the right data set and making sure the data we used was the right one to increase performance.”

TII’s meticulous approach to data preparation was coupled with advancements in architecture and fine-tuning techniques. For Falcon 3, TII significantly expanded post-training and supervised the fine-tuning stages, which, according to Dr Hacid, “boosted performance” and contributed to the model’s safety improvements.

Looking ahead, TII hopes to expand Falcon 3 into multimodality — enabling AI systems to process and generate data across multiple formats, such as text, images, and video.

The move into multimodality aligns with TII’s broader ambition to power physical AI applications like robotics and smart cities — areas where integrating data from diverse sources is key to unlocking next-generation AI capabilities.

TII has been working on multimodality for some time, having published Falcon 2 11B VLM, the first multimodal version of the model, in May 2024, and teased new Falcon family members “with an emphasis on multimodal functionalities” in December.

Falcon 2 11B VLM is capable of handling images and returning text, but compared future efforts, Dr Hacid suggested there was a big gap, with improvements again down to an increased focus on data: “We have spent a lot of time working on the data, working on how to understand the data to maximum scalability,” he explained. “We learned from [Falcon 2 11B VLM] how to better structure and train the model in a much better way.”

Another area its AI researchers have been exploring is a Mixture of Experts (MoE), where AI models produce a series of responses based on the inputs of multiple smaller systems, all working in tandem to produce an answer.

It’s an emerging field of AI research, with systems like DeepSeekMoE showing that such an approach can improve a model’s accuracy and boost its decision-making abilities.

Dr Hacid said TII is “definitely exploring” MoE research, having hinted at efforts back during Falcon 2 development, as well as experimenting with pushing its model architectures and reasoning capabilities. These efforts aim to enable models to spend more time refining their responses. He added that the team expects “extremely exciting results” with the latter project to come.

TII’s work on Falcon has helped position the UAE as a regional leader in AI development, inspiring other Middle Eastern countries to invest in the technology.

“This has created a very positive impact,” Dr Hacid says. “Most of the countries in the region are now getting into AI because they believe that something can be done.”

He highlight how TII’s success has shifted perceptions, both within the Middle East and globally. “It’s changed how the outside world views the region,” he notes, adding that the UAE is now seen as a trusted player in AI and IT innovation.

This influence extends beyond technology, with Falcon’s open-source approach encouraging collaboration and knowledge-sharing. By publishing training data and methodologies, TII has fostered a sense of transparency and community, which Dr Hacid believes will help the region become a “serious hub” for AI research and development.

Moreover, Falcon’s applications align with key regional priorities, such as smart cities, energy management and logistics, showcasing how AI can drive innovation across industries.

“We started with the model, but this is growing,” Dr Hacid says. “It’s becoming more exciting for everyone involved.”

Falcon 3: TII's new AI model takes flight to challenge Meta's Llama 3

Women leading the charge in open source AI

Show your training data: New AI definition challenges what it means to be open source