Industry News

Microsoft recently released a lightweight AI model called Phi-3 Mini, which is designed to run on smartphones and other local devices. The 3.8 billion parameter model is the first of three Phi-3 language models to be released by Microsoft. The goal is to provide a more affordable alternative to cloud-driven large language models, making AI technology accessible to smaller organizations.

Microsoft says that the Phi-3 Mini easily outperforms previous Phi-2 mini-models and is on par with larger models like the Llama 2. Surprisingly, the responsiveness of this new model is close to that of those with 10 times the size of the parameters.

In their paper, the research team notes that the key to this innovation lies in the training dataset they use. The dataset is based on the Phi-2 model, but incorporates "tightly filtered network data and synthetic data." This unique way of constructing datasets is inspired by children's books that use simpler language to illustrate complex topics, effectively enabling smaller language models to be more effective.

While Phi-3 Mini may not be able to compete with cloud-powered large language models in some ways, it has demonstrated strength over Phi-2 and other small language models (e.g., Mistral, Gemma, Llama-3-In) in tasks ranging from math to programming to academic testing. What's more, this model can run on a device as simple as a smartphone, and it doesn't require an internet connection.

However, the main limitation of the Phi-3 Mini is its breadth of "factual knowledge", which is mainly due to its relatively small training dataset. As a result, it did not perform very well in the "TriviaQA" test. Still, it's a big step forward for models that only need to work with smaller internal datasets. Microsoft hopes that by rolling out such a model, AI technology will be accessible to companies that can't afford to connect to a large language model in the cloud.

Currently, Phi-3 Mini is already available on platforms such as Azure, Hugging Face, and Ollama. Microsoft also plans to release more powerful Phi-3 Small and Phi-3 Medium models, with 7 billion and 14 billion parameters, respectively.