Aya
Aya is an open-science initiative from Cohere Labs that produces a family of open-weights multilingual models spanning text-only and multimodal tasks.
Publisher review
Aya is an open-science initiative from Cohere Labs that produces a family of open-weights multilingual models spanning text-only and multimodal tasks. It targets researchers and developers who need strong language coverage across 101 languages without relying on proprietary APIs. The project includes three distinct research models: Aya Vision (8B and 32B parameter multimodal models), Aya Expanse (a text-only model mastering 101 languages through instruction tuning), and Aya 101 (an earlier instruction-tuned model covering 101 languages). These models are designed for translation, cross-lingual transfer, and vision-language tasks, with particular emphasis on languages underrepresented in existing LLM training data.
Aya Vision achieves leading performance across both multimodal and text-only tasks, outperforming significantly larger models on benchmarks spanning 23 languages. It uses a synthetic annotation framework to curate high-quality multilingual multimodal instruction data, and a cross-modal model merging technique that mitigates catastrophic forgetting — preserving text-only capabilities while adding vision. Aya Expanse applies innovative instruction tuning and cross-lingual transfer techniques to handle 101 languages, reducing computational overhead by up to 40% via optimized training. The models are released under open weights, allowing fine-tuning and deployment on local infrastructure.
In the open-weight multilingual space, Aya competes with Mistral and Llama 3, both of which offer strong English-centric performance but narrower language coverage. Mistral provides 7B and mixture-of-experts models (8x7B Mixtral, 8x22B Mixtral) under Apache 2.0, while Llama 3 offers 8B and 70B parameter models under a custom commercial license. Aya differentiates by explicitly targeting 101 languages and providing multimodal capabilities in the Vision variant, though it lacks the extensive ecosystem and community tooling that both Mistral and Llama 3 have accumulated.
The honest trade-off: Aya's broad language coverage comes at the cost of English-centric benchmark performance — it is not designed for coding or general-purpose tasks. The models are research-focused, meaning they may require more manual tuning for production use. The open-weights release enables customization but also means users must manage their own infrastructure. Additionally, the project's documentation and community support are less mature than those of Mistral or Llama 3, which have larger user bases and more third-party integrations.
How it works
-
Multilingual coverage
Supports 101 languages, including many underrepresented ones, through instruction tuning and cross-lingual transfer techniques.
-
Multimodal capability
Aya Vision handles both images and text, achieving state-of-the-art performance across 23 languages on multimodal benchmarks.
-
Open-weights release
Models are available under open weights, enabling local deployment, fine-tuning, and research without vendor lock-in.
-
Synthetic data generation
Uses a synthetic annotation framework to create high-quality multilingual multimodal instruction data, addressing data scarcity.
-
Cross-modal model merging
Merges vision and text models to mitigate catastrophic forgetting, preserving text-only performance while adding vision capabilities.
-
Optimized training efficiency
Reduces computational overhead by up to 40% through optimized training techniques compared to baseline multilingual models.
-
Research-first design
Part of Cohere's open-science initiative, with detailed technical reports and reusable components for the research community.
Strengths and trade-offs
Strengths
- Achieves leading performance across both multimodal and text-only tasks, outperforming significantly larger models on 23-language benchmarks.
- Covers 101 languages through instruction tuning, far exceeding the typical 10-20 language range of most open-weight models.
- Reduces computational overhead by up to 40% via optimized training techniques, making it more accessible for resource-constrained teams.
- Uses synthetic data generation and cross-modal merging to address data scarcity and catastrophic forgetting in multilingual settings.
Trade-offs
- Not designed for coding or general-purpose tasks; primary focus is translation and cross-lingual transfer.
- Research-focused models may require additional tuning and infrastructure management for production deployment.
- Community support and documentation are less mature compared to Mistral or Llama 3, which have larger user ecosystems.
- English-centric benchmark performance lags behind specialized English-only models of similar parameter counts.
Pricing context
No pricing information available; models are open-weights and free to use under the provided license.
Getting started with Aya
-
Download model weights
Visit the official Aya repository on Hugging Face or GitHub. Select the desired model variant (Aya Vision, Aya Expanse, or Aya 101) and download the open-weights files to your local machine or server.
-
Set up Python environment
Create a Python virtual environment with Python 3.10 or later. Install the required dependencies including PyTorch, Transformers, and Accelerate libraries using pip. Ensure your environment has sufficient GPU memory for the model size.
-
Load the model
Use the Transformers library to load the downloaded model and tokenizer. For example, call `AutoModelForCausalLM.from_pretrained()` with the local path to the model directory. Set the device map to 'auto' for multi-GPU support.
-
Run a translation task
Prepare a text input in one of the 101 supported languages. Tokenize the input using the loaded tokenizer, generate output with `model.generate()`, and decode the response. Verify the translation quality for your target language pair.
-
Fine-tune for production
Collect a domain-specific multilingual dataset and format it for instruction tuning. Use the Hugging Face Trainer API or a custom training loop to fine-tune the model on your data. Save the fine-tuned weights and deploy using a serving framework like vLLM.
Frequently Asked Questions
What is Aya and who created it?
Aya is an open-science initiative from Cohere Labs that produces a family of open-weights multilingual models. It includes text-only and multimodal variants covering 101 languages, targeting researchers and developers who need strong language coverage without relying on proprietary APIs.
How many languages does Aya support?
Aya supports 101 languages through instruction tuning and cross-lingual transfer techniques. This includes many underrepresented languages that are often missing from other large language models, making it a strong choice for multilingual tasks like translation and cross-lingual transfer.
What are the different Aya models available?
Aya includes three research models: Aya Vision (8B and 32B parameter multimodal models), Aya Expanse (a text-only model mastering 101 languages), and Aya 101 (an earlier instruction-tuned model). Each serves different needs, from vision-language tasks to pure text translation.
How does Aya compare to Mistral and Llama 3?
Aya offers broader language coverage with 101 languages, while Mistral and Llama 3 focus on English-centric performance with narrower language support. However, Aya lags in English benchmark performance and has less mature community support and documentation compared to its competitors.
Can I use Aya for free and deploy it locally?
Yes, Aya models are released under open weights, allowing free use, fine-tuning, and local deployment without vendor lock-in. There is no pricing information available, and users must manage their own infrastructure for production use.
What are the main weaknesses of Aya models?
Aya is not designed for coding or general-purpose tasks, and its English-centric benchmark performance lags behind specialized English-only models. The research-focused models may require additional tuning for production, and community support is less mature than Mistral or Llama 3.
Alternatives
How Aya compares
Direct head-to-head against 2 competitors. Picked by 7wData.
Aya
- Pricing
- No pricing information available; models are open-weights and free to use under the provided license.
- Target
- Aya is an open-science initiative from Cohere Labs that produces a family of open-weights multilingual models spanning text-only and multimodal tasks.
- Strength
- Achieves leading performance across both multimodal and text-only tasks, outperforming significantly larger models on 23-language benchmarks.
- Watch for
- Not designed for coding or general-purpose tasks; primary focus is translation and cross-lingual transfer.
Command R
- Pricing
- Custom/Contact sales
- Target
- Enterprises needing accurate, citation-backed text generation
- Deployment
- Cloud API
- Strength
- Clear citations to reduce hallucinations
- Watch for
- Pricing not publicly disclosed; may escalate with usage
Falcon 2
- Pricing
- Free (open-source)
- Target
- Developers needing open-source multilingual multimodal models
- Deployment
- Self-hosted
- Strength
- Vision-to-language capabilities in an 11B model
- Watch for
- Smaller community and fewer integrations than Llama 3
User reviews
No user reviews yet. Be the first to write one.
Sources
Reporting on this tool draws on these publicly available sources.