Snowflake revealed its new state-of-the-art, enterprise-grade open source LLM (large language model) Snowflake Arctic on April 24, 2024. Arctic stands out from the rest due to its unique and innovative Dense Mixture of Experts hybrid transformer architecture, which delivers top-tier intelligence with exceptional efficiency at large scale. It has an astonishing 480 billion parameters spread across 128 fine-grained experts and uses a top-2 gating technique to choose 17 Billion active parameters. A standout feature of Snowflake Arctic is its open nature—Snowflake has released its weights under an Apache 2.0 license—setting a new standard for openness in enterprise AI technology. Arctic’s outstanding performance surpasses both open source and closed source models, such as DBRX, Llama, Mixtral, Grok and others, effectively cementing its position as the best LLM for enterprise applications.
In this article, we cover everything you need to know about Snowflake Arctic: its architecture, training process, model variants, benchmark performance against DBRX, Llama, Mixtral and more, and where you can try it right now.
What is a LLM, and why does it matter?
Large language models (LLM for short) are advanced machine learning models that are trained using a massive corpus of text data, allowing them to generate and analyze language just like we do. They can tackle a variety of tasks, such as:
- Creating meaningful text based on the given prompts
- Translating between languages
- Providing context-aware responses to queries
- Identifying sentiment or emotions in text
The secret sauce behind LLMs is a mix of deep learning techniques and powerful transformer-based architectures. These systems help analyze and understand language more effectively by focusing on the most significant parts of the input.
LLMs have numerous real-world use cases, such as:
- Assisting in writing articles, stories and even poetry
- Powering conversational agents and chatbots
- Providing seamless language translation services
… and so much more!
So, the next time you engage with an AI that feels almost human, remember that a Large Language Model is likely the intelligence behind it!
Several open LLMs are now gaining popularity and adoption. Let’s look at some prominent names:
- Grok-1 was developed by xAI and uses a sparse state-of-the-art Mixture of Experts (MoE) architecture with 314 billion parameters. It uses top-2 gating (8 experts total, 2 active per token), meaning it’s efficient at inference despite its large parameter count.
- Llama (Large Language Model Meta AI) is a family of autoregressive dense models from Meta AI. The original lineup ranged from 7 billion to 65 billion parameters. Later releases like Llama 2 pushed that to 70 billion, and Llama 3 brought 8 billion and 70 billion variants trained on up to 15 trillion tokens, with a 405 billion parameter version in the mix too.
- DBRX comes from Databricks. It has 132 billion total parameters and uses a fine-grained MoE architecture with 16 experts per layer, activating 4 per token for 36 billion active parameters at any time. It was pre-trained on 12 trillion tokens.
- Snowflake Arctic sits in a different category architecturally. We’ll get into the details below.
What is Snowflake Arctic?
Snowflake Arctic, a state-of-the-art large language model developed by Snowflake’s AI research team, was announced on April 24, 2024. Arctic is a groundbreaking achievement in enterprise AI, combining top-tier intelligence with unprecedented efficiency and a solid dedication to openness.
Snowflake Arctic utilizes a unique Dense Mixture of Experts (MoE) Hybrid transformer architecture, with 480B parameters distributed over 128 fine-grained experts and uses top-2 gating to choose 17B active parameters.
Snowflake Arctic has several key features, such as:
1) Efficiently intelligent
Snowflake Arctic excels at complex enterprise tasks, such as SQL generation, coding and instruction following. It frequently surpasses other models in industry benchmark tests, proving its ability to deal with complicated, real-world situations.
2) Low training cost
Snowflake Arctic is powered by a unique Dense-MoE Hybrid transformer architecture, Arctic delivers top-tier performance at a fraction of the development cost compared to similar models.
3) Truly Open Source
Snowflake has released Snowflake Arctic under an Apache 2.0 license, providing ungated access to its weights and code.
4) Enterprise focus
Snowflake Arctic model is specifically tailored for enterprise AI needs, focusing on high-quality tasks for enterprise. It’s great for core applications like data analysis and automation and shines in outperforming larger models without needing lots of computing power.
Snowflake Arctic model variants
To meet various enterprise AI needs, Snowflake has introduced two major models within the Snowflake Arctic family:
1) Snowflake Arctic Instruct is a fine-tuned version optimized for following natural language instructions. It’s the variant we compare in the benchmarks throughout this article. Think of it as the production-ready version for most enterprise query and assistant tasks.
2) Snowflake Arctic Base is the raw pre-trained checkpoint. It’s a solid foundation for further fine-tuning or use cases like retrieval-augmented generation (RAG) where you want to layer task-specific behavior on top of a strong base.
Alongside these, Snowflake also just launched the Snowflake Arctic family of text embedding models, which includes five text embedding models. These embedded text models, which were optimized for retrieval tasks, were released to the open source community in mid-April under an Apache 2.0 license. The models are:
- Snowflake-arctic-embed-m
- Snowflake-arctic-embed-l
- Snowflake-arctic-embed-m-long
- Snowflake-arctic-embed-xs
- Snowflake-arctic-embed-s
Snowflake Arctic’s training process and architecture: what makes it work?
What actually distinguishes Snowflake Arctic from other LLMs is its unique training approach and architecture, which were carefully crafted by the Snowflake AI research team. Their goal was to pioneer new ground in cost-effective training and transparency. Snowflake Arctic provides a unique solution for enterprises that prioritizes performance and accessibility.
Dense + Mixture of Experts (MoE) hybrid architecture
Snowflake Arctic combines a 10 billion dense transformer model with a massive residual 128 experts * 3.66 Billion parameter Mixture of Experts (MoE) Multilayer perceptron (MLP) component, resulting in a total of 480 Billion parameters and 17 Billion active parameters chosen using top-2 gating technique.
Many-but-condensed experts
One significant innovation is Snowflake Arctic’s “many-but-condensed” expert approach. It uses 128 fine-grained experts, far more than typical Mixture of Experts (MoE) models, which often use only ~8-16 experts. Having a large number of experts increases routing flexibility and combinatorial modeling power. However, Snowflake Arctic mitigates potential overhead by making these experts “condensed”—each expert is 3.66 billion parameters rather than the hundreds of billions used in other models. This condensed size allows for efficient expert activation during inference.
Architecture
Training vanilla Mixture of Experts (MoE) architectures with many experts is computationally inefficient due to the massive communication overhead between experts. Arctic’s architecture cleverly mitigates this through system-model co-design.
Snowflake Arctic’s architecture allows for overlapping communication and computation by combining the dense transformer with the residual Mixture of Experts (MoE) component. This combination between model and systems allows hiding a significant portion of the communication overhead, allowing for exceptionally efficient Mixture of Experts (MoE) training.
How was Snowflake Arctic created?
Snowflake Arctic was created through extensive training effort, using cutting-edge infrastructure and techniques.
1) Hardware infrastructure
Snowflake Arctic was created through a massive training effort leveraging cutting-edge infrastructure. The model was trained using a specialized cluster of over 1,000 GPUs. This immense computational power required around $2 million in computational resources, which is remarkably cost-effective compared to other models.
2) Training process
Most open source LLMs follow a generic pre-training approach. In contrast, Snowflake Arctic was trained in a three-stage curriculum, with each stage focusing on different data compositions to optimize the model’s performance on enterprise-focused tasks. The training process for Arctic was split into three distinct stages totaling around 3.5 Trillion tokens:
- Phase 1: 1 Trillion tokens
- Phase 2: 1.5 Trillion tokens
- Phase 3: 1 Trillion tokens
This multi-stage approach allowed different competencies to be wired logically, analogous to how humans acquire knowledge incrementally.
3) Development process and timeline
Snowflake states that the AI research team relied on publicly available data throughout the procedure. Some of the key public datasets used are:
The entire process of dataset collection, model architecture design, multi-phase training and iterative refinement spanned approximately 3 months from start to the release of Snowflake Arctic.
Snowflake AI research team was able to improve Arctic’s performance on complex enterprise tasks like SQL creation, coding and instruction following by merging these disparate data sources and using a precisely crafted three-stage curriculum.
How does Snowflake Arctic perform against other open models?
Snowflake has introduced two models—Snowflake Arctic Base and Snowflake Arctic Instruct. Snowflake Arctic Instruct is a general-purpose model that excels in various benchmarks. Let’s delve into the technical details of how Snowflake Arctic Instruct compares against popular open LLM models. Here’s a breakdown of how Arctic stacks up against models like DBRX, Llama (3 8B, 2 70B, 3 70B) and Mixtral (8x7B, 8x22B):
What is the difference between DBRX and Arctic?
Let’s dive into the detailed comparison between Snowflake Arctic and Databricks DBRX across several key benchmarks:
| Benchmark | Arctic Instruct | DBRX Instruct |
| MMLU (general knowledge) | 67.3% | 73.7% |
| Commonsense reasoning | 73.1% | 74.8% |
| SQL generation (Spider) | 79.0% | 76.3% |
| HumanEval (coding) | 64.3% | 61.0% |
| GSM8k (math reasoning) | 74.2% | 73.5% |
| IFEval (instruction following) | 52.4% | 27.6% |
1) Architecture difference
DBRX uses a fine-grained Mixture of Experts (MoE) architecture, whereas Arctic uses a distinctive Dense + Mixture of Experts (MoE) Hybrid transformer architecture, combining a dense model with a residual MoE component for efficient training and inference.
2) Active parameters
In terms of active parameters, DBRX has a significant advantage over Snowflake Arctic, with a remarkable 36 Billion active parameters to Snowflake Arctic’s 17 Billion.
3) Active experts
Snowflake Arctic can activate up to 128 experts at the same time. This demonstrates its potential to leverage a larger number of experts than DBRX, which can activate 16 experts.
4) Training tokens
Snowflake Arctic’s training involved 3.5 Trillion tokens, while DBRX was trained on a substantially larger dataset of 12 Trillion tokens.
5) General knowledge
MMLU (Multiple-choice Model-Linguistic Understanding) benchmark shows that DBRX outperforms Snowflake Arctic.
- MMLU score: 67.3% (Snowflake Arctic Instruct) vs. 73.3% (DBRX Instruct)
6) Commonsense reasoning
Both Snowflake Arctic and DBRX perform similarly on commonsense reasoning benchmarks.
- Score: 73.1% (Snowflake Arctic Instruct) vs. 74.8% (DBRX Instruct)
7) SQL generation
Snowflake Arctic beats DBRX in SQL generation tasks, as shown by the Spider benchmark test.
- SQL Generation (Spider) score: 79.0% (Snowflake Arctic Instruct) vs. 76.3% (DBRX Instruct)
8) Programming and mathematical reasoning
Snowflake Arctic excels at programming and mathematical reasoning, scoring higher on the HumanEval and GSM8k benchmarks.
- HumanEval score: 64.3% (Snowflake Arctic Instruct) vs. 61.0% (DBRX Instruct)
- GSM8k score: 74.2% (Snowflake Arctic Instruct) vs. 73.5% (DBRX Instruct)
9) Instruction Following (IFEval)
This benchmark assesses a model’s ability to follow textual instructions in a variety of tasks, including reasoning, planning and problem-solving. Snowflake Arctic outperformed DBRX in this benchmark. Here is the score:
- IFEval Score: 52.4% (Snowflake Arctic Instruct) vs. 27.6% (DBRX Instruct)
Snowflake Arctic vs Llama 3 8B
Let’s take a closer look at Snowflake Arctic vs Llama 3 8B:
| Benchmark | Arctic Instruct | Llama 3 8B |
| MMLU | 67.3% | 65.7% |
| Commonsense reasoning | 73.1% | 68.5% |
| SQL generation (Spider) | 79.0% | 69.9% |
| HumanEval | 64.3% | 59.2% |
| GSM8k | 74.2% | 75.4% |
| IFEval | 52.4% | 42.7% |
1) Architecture difference
Llama 3 8B uses a traditional Dense architecture, while Snowflake Arctic utilizes a unique Dense + Mixture of Experts (MoE) Hybrid transformer architecture.
2) Active parameters
Snowflake Arctic has a significant advantage over Llama 3 8B, with 17 Billion active parameters compared to Llama 3 8B’s 8 Billion.
3) Training tokens
Snowflake Arctic trained using 3.5 Trillion tokens, but Llama 3 8B trained with a much bigger dataset of 15 Trillion tokens.
4) General knowledge
MMLU (Multiple-choice Model-Linguistic Understanding) (or General Knowledge) benchmark shows that Snowflake Arctic outperforms Llama 3 8B.
- MMLU Score: 67.3% (Snowflake Arctic Instruct) vs. 65.7% (Llama 3 8B)
5) Commonsense reasoning
In terms of common sense reasoning, Snowflake Arctic outperforms Llama 3 8B.
- Score: 73.1% (Snowflake Arctic Instruct) vs. 68.5% (Llama 3 8B)
6) SQL generation
Snowflake Arctic completely outperforms Llama 3 8B in SQL generation tasks, as shown by the Spider benchmark test.
- SQL Generation (Spider) Score: 79.0% (Snowflake Arctic Instruct) vs. 69.9% (Llama 3 8B)
8) Programming and mathematical reasoning
Snowflake Arctic excels at programming but falls slightly behind Llama 3 8B in mathematical reasoning. Here’s the HumanEval and Math(GSM8k) benchmark scores.
- HumanEval score: 64.3% (Snowflake Arctic Instruct) vs. 59.2% (Llama 3 8B)
- GSM8k Score: 74.2% (Snowflake Arctic Instruct) vs. 75.4% (Llama 3 8B)
7) Instruction Following (IFEval)
Snowflake Arctic outperforms Llama 3 8B in instruction following benchmark. Here is the score:
- IFEval Score: 52.4% (Snowflake Arctic Instruct) vs. 42.7% (Llama 3 8B)
Snowflake Arctic vs Llama 2 70B
Let’s take a closer look at Snowflake Arctic vs Llama 2 70B:
| Benchmark | Arctic Instruct | Llama 2 70B |
| MMLU | 67.3% | 68.6% |
| Commonsense reasoning | 73.1% | 72.1% |
| SQL generation (Spider) | 79.0% | 62.8% |
| HumanEval | 64.3% | 33.7% |
| GSM8k | 74.2% | 52.6% |
| IFEval | 52.4% | — |
1) Architecture difference
Llama 2 70B utilizes a conventional Dense architecture, whereas Snowflake Arctic uses a distinctive Dense + Mixture of Experts (MoE) Hybrid transformer architecture.
2) Active parameters
Llama 2 70B has a huge advantage with 70 Billion active parameters, significantly more than Snowflake Arctic’s 17 Billion.
3) Training tokens
Snowflake Arctic’s training involved 3.5 Trillion tokens, while Llama 2 70B was trained on 2 Trillion tokens.
4) General knowledge
MMLU (Multiple-choice Model-Linguistic Understanding) (or General Knowledge) benchmark shows that Llama 2 70B outperforms Snowflake Arctic.
- MMLU Score: 67.3% (Snowflake Arctic Instruct) vs. 68.6% (Llama 2 70B)
5) Commonsense reasoning
Snowflake Arctic outperforms Llama 2 70B in terms of common sense reasoning.
- Score: 73.1% (Snowflake Arctic Instruct) vs. 72.1% (Llama 2 70B)
6) SQL generation
Snowflake Arctic outperforms Llama 2 70B in SQL generation tasks, as demonstrated by the Spider benchmark test.
- SQL Generation (Spider) Score: 79.0% (Snowflake Arctic Instruct) vs. 62.8% (Llama 2 70B)
8) Programming and mathematical reasoning
Snowflake Arctic excels at programming and mathematical reasoning, scoring higher on the HumanEval and GSM8k benchmarks than Llama 2 70B.
- HumanEval score: 64.3% (Snowflake Arctic Instruct) vs. 33.7% (Llama 2 70B)
- GSM8k Score: 74.2% (Snowflake Arctic Instruct) vs. 52.6% (Llama 2 70B)
Snowflake Arctic vs Llama 3 70B
Let’s take a closer look at Snowflake Arctic vs Llama 3 70B:
| Benchmark | Arctic Instruct | Llama 3 70B |
| MMLU | 67.3% | 79.8% |
| Commonsense reasoning | 73.1% | 72.6% |
| SQL generation (Spider) | 79.0% | 80.2% |
| HumanEval | 64.3% | 71.9% |
| GSM8k | 74.2% | 91.4% |
| IFEval | 52.4% | 43.6% |
1) Architecture difference
Llama 3 70B utilizes a conventional Dense architecture, whereas Snowflake Arctic uses a distinctive Dense + Mixture of Experts (MoE) Hybrid transformer architecture.
2) Active parameters
In terms of active parameters, Llama 3 70B has a huge advantage over Snowflake Arctic, with an amazing 70 Billion active parameters over Snowflake Arctic’s 17 Billion active parameters.
3) Training tokens
Snowflake Arctic was trained using 3.5 Trillion tokens, whereas Llama 3 70B was trained on a massively larger dataset of 15 Trillion tokens.
4) General knowledge
MMLU (Multiple-choice Model-Linguistic Understanding) (or General knowledge) benchmark indicates better performance for Llama 3 70B over Snowflake Arctic.
- MMLU Score: 67.3% (Snowflake Arctic Instruct) vs. 79.8% (Llama 3 70B)
5) Commonsense reasoning
Snowflake Arctic performs really well on common sense reasoning compared to Llama 3 70B.
- Score: 73.1% (Snowflake Arctic Instruct) vs. 72.6% (Llama 3 70B)
6) SQL generation
Snowflake Arctic is head-to-head with Llama 3 70B in SQL generation tasks, as demonstrated by the Spider benchmark test.
- SQL Generation (Spider) Score: 79.0% (Snowflake Arctic Instruct) vs. 80.2% (Llama 3 70B)
8) Programming and mathematical reasoning
Snowflake Arctic excels at programming but falls slightly behind Llama 3 70B in mathematical reasoning. Here’s the HumanEval and Math(GSM8k) benchmark scores.
- HumanEval score: 64.3% (Snowflake Arctic Instruct) vs. 71.9% (Llama 3 70B)
- GSM8k Score: 74.2% (Snowflake Arctic Instruct) vs. 91.4% (Llama 3 70B)
7) Instruction Following (IFEval)
Snowflake Arctic performs slightly better than Llama 3 70B in instruction following benchmark. Here is the score:
- IFEval Score: 52.4% (Snowflake Arctic Instruct) vs. 43.6% (Llama 3 70B)
Snowflake Arctic vs Mixtral 8x7B
Let’s take a closer look at Snowflake Arctic vs Mixtral 8x7B:
| Benchmark | Arctic Instruct | Mixtral 8x7B |
| MMLU | 67.3% | 70.4% |
| Commonsense reasoning | 73.1% | 74.1% |
| SQL generation (Spider) | 79.0% | 71.3% |
| HumanEval | 64.3% | 48.1% |
| GSM8k | 74.2% | 63.2% |
| IFEval | 52.4% | 52.2% |
1) Architecture difference
Mixtral 8x7B utilizes a conventional Mixture of Experts (MoE) architecture, whereas Snowflake Arctic employs a distinctive Dense + Mixture of Experts (MoE) Hybrid transformer architecture.
2) Active parameters
Snowflake Arctic has 17 Billion active parameters, while Mixtral 8x7B contains 13 Billion active parameters.
3) Active experts
Snowflake Arctic can activate up to 128 experts at the same time. This demonstrates its ability to use a larger number of experts than Mixtral 8x7B, which can activate 8 experts at once.
4) Training tokens
Snowflake Arctic’s training required 3.5 Trillion tokens, while Mixtral 8x7B is not disclosed.
5) General knowledge
MMLU (Multiple-choice Model-Linguistic Understanding)(or General knowledge) benchmark indicates better performance for Mixtral 8x7B over Snowflake Arctic.
- MMLU score: 67.3% (Snowflake Arctic Instruct) vs. 70.4% (Mixtral 8x7B)
6) Commonsense reasoning
Both Snowflake Arctic and Mixtral 8x7B perform similarly on commonsense reasoning benchmarks.
- Score: 73.1% (Snowflake Arctic Instruct) vs. 74.1% (Mixtral 8x7B)
7) SQL generation
Snowflake Arctic outperforms Mixtral 8x7B in SQL generation tasks, as demonstrated by the Spider benchmark test.
- SQL Generation (Spider) Score: 79.0% (Snowflake Arctic Instruct) vs. 71.3% (Mixtral 8x7B)
8) Programming and mathematical reasoning
Snowflake Arctic excels at programming and mathematical reasoning, scoring higher on the HumanEval and GSM8k benchmarks.
- HumanEval score: 64.3% (Snowflake Arctic Instruct) vs. 48.1% (Mixtral 8x7B)
- GSM8k score: 74.2% (Snowflake Arctic Instruct) vs. 63.2% (Mixtral 8x7B)
9) Instruction Following (IFEval)
Snowflake Arctic performs slightly better than Llama 3 8B in the instruction following benchmark. Here is the score:
- IFEval Score: 52.4% (Snowflake Arctic Instruct) vs. 52.2% (Mixtral 8x7B)
Snowflake Arctic vs Mixtral 8x22B
Let’s take a closer look at Snowflake Arctic vs Mixtral 8x22B:
| Benchmark | Arctic Instruct | Mixtral 8x22B |
| MMLU | 67.3% | 77.5% |
| Commonsense reasoning | 73.1% | 75.6% |
| SQL generation (Spider) | 79.0% | 79.2% |
| HumanEval | 64.3% | 69.9% |
| GSM8k | 74.2% | 84.2% |
| IFEval | 52.4% | 61.5% |
1) Architecture difference
Mixtral 8x22B utilizes a conventional Mixture of Experts (MoE) architecture, whereas Snowflake Arctic utilizes a distinctive Dense + MoE Hybrid transformer architecture.
2) Active parameters
Mixtral 8x22B has a significant advantage over Snowflake Arctic in terms of active parameters, with 44 Billion vs 17 Billion.
3) Active experts
Snowflake Arctic can activate up to 128 experts at the same time. This demonstrates its ability to use a larger number of experts than Mixtral 8x22B, which can activate only 8 experts at once.
4) Training tokens
Snowflake Arctic’s training involved 3.5 Trillion tokens, while Mixtral 8x22B is not disclosed.
5) General knowledge
MMLU (Multiple-choice Model-Linguistic Understanding) benchmark shows that Mixtral 8x22B outperforms Snowflake Arctic.
- MMLU score: 67.3% (Snowflake Arctic Instruct) vs. 77.5% (Mixtral 8x22B)
6) Commonsense reasoning
Mixtral 8x22B performs relatively better performance levels on commonsense reasoning benchmarks.
- Score: 67.3% (Snowflake Arctic Instruct) vs. 75.6% (Mixtral 8x22B)
7) SQL generation
Snowflake Arctic and Mixtral 8x22B perform relatively similarly in SQL generation tasks, as demonstrated by the Spider benchmark test.
- SQL Generation (Spider) score: 79.0% (Snowflake Arctic Instruct) vs. 79.2% (Mixtral 8x22B)
8) Programming and mathematical reasoning
Mixtral 8x22B excels at programming and mathematical reasoning, scoring higher on the HumanEval and GSM8k benchmarks.
- HumanEval score: 64.3% (Snowflake Arctic Instruct) vs. 69.9% (Mixtral 8x22B)
- GSM8k score: 74.2% (Snowflake Arctic Instruct) vs. 84.2% (Mixtral 8x22B)
9) Instruction Following (IFEval)
Mixtral 8x22B performs slightly better than Snowflake Arctic in instruction following benchmark. Here is the score:
- IFEval Score: 52.4% (Snowflake Arctic Instruct) vs. 61.5% (Mixtral 8x22B)
Check out the following table to get more in-depth insights into the benchmark performance differences between Snowflake Arctic and other open source LLM models.
Snowflake Arctic’s context window and long-context handling
Snowflake Arctic excels in long-context tasks with its impressive 4096 token attention context window. This capability allows Snowflake Arctic to process and analyze up to 4096 tokens simultaneously, significantly boosting its ability to handle complex and lengthy inputs. The result is improved context awareness and more accurate, logical outputs. Although its context window is smaller than that of some larger LLMs, such as Gpt 4 Turbo(32k tokens), Claude 2(100k tokens), Claude 3 Opus(200K tokens) and Gemini 1.5 Pro(1 Million tokens), it is still competent at performing the majority of long-context tasks found in enterprise environments.
What makes this even more impressive is that the Snowflake team is actively working on extending this capability to support unlimited sequence generation through a sliding window approach and special attention “sinks”, which will further enhance Snowflake Arctic’s performance on long-context tasks and solidify its position as a top contender in the world of enterprise LLM.
Where to try Snowflake Arctic?
To experience the power of Snowflake Arctic firsthand, you can explore various live demo platforms where the model is showcased:
1) Snowflake Cortex
You can use the Snowflake Arctic model in Snowflake Cortex’s COMPLETE LLM function. Check out this step-by-step guide for setting up and using the Snowflake Arctic model with Snowflake Cortex functions. All you need to do is specify ‘snowflake-arctic‘ as the model argument when calling the COMPLETE function.
For example:
SELECT SNOWFLAKE.CORTEX.COMPLETE('snowflake-arctic', 'Write a brief introduction about Snowflake Arctic');2) Streamlit Community Cloud
Snowflake has partnered with Streamlit to provide a live demo of Arctic on the Streamlit Community Cloud platform.
3) Hugging Face
You can find Snowflake Arctic demos and interactive playground environments on the Hugging Face platform.
4) NVIDIA API Catalog
NVIDIA offers Arctic demos and integration samples through their API catalog.
5) Replicate
Replicate hosts live demos and API integrations for Snowflake Arctic, allowing you to experience the model’s capabilities firsthand.
Coming Soon:
Snowflake has announced that Snowflake Arctic will be available for live demos and integrations across multiple platforms:
- AWS (Now available)
- Microsoft Azure (Now available)
- Lamini (Coming soon)
- Perplexity (Coming soon)
- Together.ai (Now available)
…and more coming soon!
Save up to 30% on your Snowflake spend in a few minutes!
Conclusion
And that’s a wrap! Snowflake just took its first big step towards building an enterprise-grade truly open source LLM called Snowflake Arctic. Arctic’s pioneering Dense-Mixture of Experts (MoE) Hybrid architecture establishes a new standard for cost-effective training and inference of large language models, making it accessible to enterprises of any size. Snowflake Arctic is designed to enable enterprises to fully utilize AI by excelling at complicated tasks such as SQL creation, coding and instruction following. Its higher performance, combined with its genuinely open nature and the availability of open research results, promotes collaboration and innovation in the AI community.
In this article, we have covered:
- What is a Large Language Model?
- What is Snowflake Arctic?
- List of available Snowflake Arctic models
- Snowflake Arctic Training process and architecture
- How was Snowflake Arctic created?
- How does Snowflake Arctic performance compare against other models?
- How does Snowflake Arctic perform on Long-Context tasks ?
- Step-by-step guide to setting up Snowflake Arctic
- Where totry out Snowflake Arctic?
… and so much more!
FAQs
What is Snowflake Arctic?
Snowflake Arctic is an open source LLM developed by the Snowflake AI Research Team, released under an Apache 2.0 license. It uses a Dense-MoE Hybrid transformer architecture with 480 billion total parameters and 17 billion active parameters.
What does the Dense-MoE Hybrid architecture actually mean?
Every token passes through a 10 billion parameter dense transformer, which handles general contextual processing. Then a residual MoE component routes each token to 2 of 128 experts (each 3.66 billion parameters) for specialized processing. The two components run together, with the MoE output added residually on top of the dense transformer output.
How many parameters does Snowflake Arctic have?
480 billion total. 17 billion are active at any given time, selected via top-2 gating across 128 experts.
What is top-2 gating?
The gating mechanism scores all 128 experts for each input token and routes that token to the 2 highest-scoring experts. Only those 2 experts process the token; the other 126 are not activated for that token.
How is Arctic different from DBRX architecturally?
DBRX uses a pure MoE architecture with 16 experts per layer, activating 4 per token. Arctic uses a hybrid where a dense transformer handles the base computation and the MoE component adds a residual layer with 128 experts, 2 activated per token. Arctic also has more total parameters (480B vs. 132B) but fewer active parameters than DBRX (17B vs. 36B).
How many tokens was Arctic trained on?
3.5 trillion tokens across three phases: 1 trillion, 1.5 trillion and 1 trillion.
What datasets were used to train Arctic?
The main publicly available datasets include RefinedWeb, C4, RedPajama and StarCoder. Snowflake supplemented these with proprietary data compositions designed for enterprise tasks.
How much did it cost to train Arctic?
Under $2 million in compute costs. Snowflake achieved this through architectural efficiency: the Dense-MoE Hybrid design reduces communication overhead during training, and the three-stage curriculum targets compute toward the tasks that matter most.
What is Arctic’s context window?
4,096 tokens (4K). Snowflake was working on extending this via sliding window attention and attention sinks at the time of release.
What’s the difference between Arctic Instruct and Arctic Base?
Arctic Instruct is fine-tuned for following natural language instructions and is the production-ready model for most enterprise use cases. Arctic Base is the raw pre-trained checkpoint, better suited as a starting point for custom fine-tuning.
What are the Arctic embedding models?
A separate family of five smaller models (xs, s, m, m-long, l) optimized for retrieval tasks like semantic search and RAG. They’re not the same as the 480B LLM and were released at roughly the same time under the same Apache 2.0 license.
Where can I run Arctic?
Via Snowflake Cortex (SQL interface), Hugging Face, Streamlit Community Cloud, NVIDIA API Catalog, Replicate, AWS and Microsoft Azure.
Is Arctic suitable for math-heavy tasks?
Not at the level of Llama 3 70B or Mixtral 8x22B. Arctic scores 74.2% on GSM8k vs. 91.4% for Llama 3 70B. It’s not Arctic’s design target, and the benchmark reflects that. If mathematical reasoning is a core requirement, a different model is a better fit.
How long did Arctic take to build?
Roughly three months from dataset collection through model architecture design, multi-phase training and final release.
