Oct 2nd, 2023

We are beyond excited to unveil not just one, but two groundbreaking advancements in open-source multilingual language models: Vulture-180B and Vulture-40B. Engineered on top of the formidable Falcon-180B and Falcon-40B by TII, these models cater to a diverse range of computational needs while providing state-of-the-art multilingual capabilities in 12 different languages.

A Spectrum of Choices: 180B and 40B

The introduction of Vulture-40B alongside Vulture-180B offers a more versatile range of options for developers and researchers. While Vulture-180B is designed for heavy computational tasks, offering the highest level of performance, Vulture-40B provides a more accessible entry point with reduced hardware requirements. Both models are fine-tuned with our curated 80GB multilingual dataset, ensuring top-tier natural language understanding and generation across multiple languages. Following that, we performed instructional fine-tuning under Alpaca's technique.

Technical Details and Accessibility

Both models come freely available under the APACHE-2.0 license, ensuring that they can be integrated into a variety of applications with ease. However, it's important to note that Falcon-180B, the foundational model, operates under its specific license and acceptable use policy.

For those concerned about hardware specifications, Vulture-180B requires approximately 8xA100 80GB or equivalent for full bfloat16 precision. On the other hand, Vulture-40B is designed to run efficiently on less resource-intensive setups, making it suitable for smaller-scale projects.

Partnerships and Collaborations

To ensure we are on the right track, we are open to partnerships with academic institutions, non-profit organizations, and community initiatives that share our vision. Through collaborative efforts, we aim to accelerate the pace of research and development in multilingual NLP.

If you are interested in collaboration with us, please e-mail: quan@vilm.org

Acknowledgments and Next Steps

We extend our heartfelt thanks to TII for the Falcon series and Google for their Cloud credits. With the dual launch of Vulture-180B and Vulture-40B, we're taking a giant leap towards making advanced multilingual NLP accessible to a broader community.

We will continue to update the Vulture series with more language being supported as one of our goal in democratizing AI for the community is to make sure that no language is left behind.

Visit our website, Virtual Interactive, for upcoming technical reports, how-to guides, and more.

Whether you're a startup looking to integrate NLP solutions or a research institution pushing the boundaries of what's possible, our Vulture series offers a scalable path to meet your needs.

Embark on your multilingual NLP journey with Vulture-180B and Vulture-40B today!