Microsoft has announced a smaller AI model called Phi-3 to lower the cost for users, who could not afford its Large Language Models (LLM).
Announcing the new model via a statement on Tuesday, Microsoft said the Phi-3-mini can outperform models twice its size across a variety of benchmarks that evaluate language, coding, and math capabilities.
According to the company, smaller AI models are designed to perform simpler tasks, making it easier to use by companies with limited resources. A business could use Phi-3 to summarize the main points of a long document, and extract relevant insights and industry trends from market research reports, Microsoft added.
How Phi-3 was trained
Commenting on the model, the Corporate Vice President of Microsoft Azure AI Platform, Eric Boyd, said Phi-3 Mini is as capable as LLMs like GPT-3.5 “just in a smaller form factor.” Boyd added that developers trained Phi-3 with a “curriculum.”
- “They were inspired by how children learned from bedtime stories, books with simpler words, and sentence structures that talk about larger topics.
- “There aren’t enough children’s books out there, so we took a list of more than 3,000 words and asked an LLM to make ‘children’s books’ to teach Phi,” he said.
He added that Phi-3 simply built on what previous iterations learned. Microsoft released Phi-2 in December, which performed just as well as bigger models like Llama 2. However, the company said the Phi-3 performs better than the previous version and can provide responses close to how a model 10 times bigger than it can.
What you should know
Because the smaller models require less processing, big tech providers can charge customers less to use them. They hope that means more customers can apply A.I. in places where the bigger, more advanced models have been too expensive to use. Though Microsoft said using the new models would be “substantially cheaper” than using larger models like GPT-4, it did not offer specifics.
Microsoft’s competitors have their own small AI models as well, most of which target simpler tasks like document summarization or coding assistance.
Google’s Gemma 2B and 7B are good for simple chatbots and language-related work. Anthropic’s Claude 3 Haiku can read dense research papers with graphs and summarize them quickly, while the recently released Llama 3 8B from Meta may be used for some chatbots and coding assistance.
