Meta releases Llama 4 Scout and Maverick, its first mixture-of-experts models
Llama 4 Scout and Maverick are multimodal, open-weight mixture-of-experts models with 17 billion active parameters and large context windows.
Meta has released Llama 4 Scout and Llama 4 Maverick, described by the company as its first mixture-of-experts models, released in open-weight form at the same time.
Mixture-of-experts is an architecture in which the model consists of many specialised branches and activates only part of them for each pass, so compute cost is much lower than running a full neural network of the same total size. According to the announcement, Scout has 109 billion total parameters and a 10 million token context window, Maverick has 400 billion total parameters; both are natively multimodal and use 17 billion active parameters.
For enterprises, the architecture targets cost: a model that is large in total parameters but uses only a fraction during inference is cheaper in compute than a dense model of the same scale. The large context window suits long document collections, while image reading opens applications with invoices, forms and photos of documents.
One point to note is that the models still require considerable compute infrastructure when self-operated, even though active parameters are lower than the total. Vietnamese-language quality and understanding of specific business domains need separate testing and should not be inferred from English benchmark scores.
Source: ai.meta.com
