Stability AI Releases StableLM-Alpha, Two Open Source Language Models

ChatGPT has been garnering all the mainstream attention as far as LLMs are concerned, but the company behind the most popular image generation AI has come up with its own open-source competitor.

Stability AI, the company behind image generation model Stable Diffusion, has released two open-source Language Models. The two StableLM-alpha models have 3B and 7B parameters respectively. Stability AI says that two more models, with 15B and 30B models respectively, are on the way. The models models have been released under CC BY-SA-4.0 license.

“StableLM-Alpha models are trained on the new dataset that build on The Pile, which contains 1.5 trillion tokens, roughly 3x the size of The Pile. These models will be trained on up to 1.5 trillion tokens. The context length for these models is 4096 tokens,” the release says.

StableLM-Alpha can do tasks such as chit-chatting with users, formal writing and creative writing. It can even write code, and Stability AI has demonstrated examples of each kind of use with its release.

The reactions to the new models have been largely positive. For starters, the models are open source, so unlike ChatGPT, they can be used for free. Also, their code and weights are publicly available, so users can edit the code as they please. However, there was some consternation online over Stability Ai releasing the models under a CC-4.0 license, which requires users to credit Stability AI when using the model.

Stablity AI CEO Emad Mostaque has held an AMA on Twitter after the model’s release. He said that they expected to release many more models in the coming months. “Our aim is benchmark models & open datasets for every modality, sector, nation & culture,” he said. He further said that the research-only license for the instruction-tuned variants was because of Alpaca, and they were working on CC licensed variants. Mostaque also said that the 4096 token limit will be increased in future versions. “Yes, it will increase with flash attention and some other upcoming things,” he told a Twitter user.

Stability AI has long maintained that it wants to keep developments in AI open source, as to not let any one organization or country get a say in what an AI-driven future could look like. It’s being going head to head against OpenAI in this regard, which, far from open-sourcing models, has stopped giving even the minutest details about its new releases. Stability AI, though, is helping the open source community hold its own against OpenAI’s advances — while OpenAI has DALL-E, Stability AI has the very competent Stable Diffusion, and now while OpenAI has taken the world by storm with ChatGPT, Stability AI has come up with its own Language Model in StableLM-Alpha. It remains to be seen how StableLM competes against GPT-4 and the litany of other open-source LLMs out there, but there’s certainly a thriving open source community that’s looking to match the most advanced releases by their closed-source counterparts.