Qwen (also called Tongyi Qianwen) is a family of large language models developed by Alibaba. In July of 2024 it ranked as the top Chinese language model in some benchmarks and third globally behind the top models of Anthropic and OpenAI.[1]
Models
editAlibaba first launched a beta of Qwen in April 2023 under the name Tongyi Qianwen.[2] The model was based on the llama LLM developed by Meta AI with various modifications.[3] It was publicly released in September 2023 after receiving approval from the Chinese government.[4] In December 2023 it released its 72B and 1.8B models as open source, while Qwen 7B was open sourced in August.[5][6]
In June 2024 Alibaba launched Qwen 2 and in September it released some of its models as open source, while keeping its most advanced models proprietary.[7][8] Qwen 2 employs a Mixture of experts.[9]
In November 2024, QwQ-32B-Preview, a model focusing on reasoning similar to OpenAI's o1 was released as open source under the Apache 2.0 License, although only the weights were released, not the dataset or training method.[10][11] QwQ has a 32,000 token context length and performs better than o1 on some benchmarks.[12]
The Qwen-Vl series is a line of Visual Language Models that consists of a Vision transformer and LLM.[3][13] Alibaba released Qwen-VL2 in 7 and 2 billion parameters.[14][15] Qwen-vl-max is Alibaba's flagship vision model as of 2024 and is sold by Alibaba Cloud at a cost of US$0.00041 per thousand input tokens.[16]
Alibaba has released several other model types such as Qwen-Audio and Qwen2-Math.[17] In total, it has released more than 100 models as open source, with its models having been downloaded more than 40 million times.[8][18] Finetuned versions of Qwen have been developed by enthusiasts, such as “Liberated Qwen,” developed by San Francisco-based Abacus AI, which attempts to remove all safety guardrails from the model.[19]
References
edit- ^ Jiang, Ben (11 July 2024). "Alibaba's open-source AI model tops Chinese rivals, ranks 3rd globally". South China Morning Post.
- ^ Chiang, Sheila (11 April 2023). "Alibaba to roll out its rival to ChatGPT across all its products". CNBC.
- ^ a b Bai, Jinze; et al. (28 Sep 2023). "Qwen Technical Report". arXiv:2309.16609 [cs.CL].
- ^ Jiang, Ben (13 September 2023). "Alibaba opens Tongyi Qianwen model to public as new CEO embraces AI". South China Morning Post.
- ^ Fan, Feifei (2023-12-01). "Alibaba unveils new Tongyi Qianwen AI language model". global.chinadaily.com.cn.
- ^ Ye, Josh (August 3, 2023). "Alibaba rolls out open-sourced AI model to take on Meta's Llama 2". reuters.
- ^ Jiang, Ben (7 June 2024). "Alibaba says new AI model Qwen2 bests Meta's Llama 3 in tasks like maths and coding". South China Morning Post.
- ^ a b Kharpal, Arjun (19 September 2024). "China's Alibaba launches over 100 new open-source AI models, releases text-to-video generation tool". CNBC.
- ^ Yang, An; et al. (10 Sep 2024). "Qwen2 Technical Report". arXiv:2407.10671 [cs.CL].
- ^ Dickson, Ben (29 November 2024). "Alibaba releases Qwen with Questions, an open reasoning model that beats o1-preview". VentureBeat.
- ^ 故渊 (2024-11-28). "阿里通义千问 QwQ 登场：开源 AI 推理新王，MATH 测试超 OpenAI o1 模型 - IT之家". www.ithome.com.
- ^ Wiggers, Kyle (27 November 2024). "Alibaba releases an 'open' challenger to OpenAI's o1 reasoning model". TechCrunch.
- ^ Browne, Ryan (31 December 2024). "Alibaba slashes prices on large language models by up to 85% as China AI rivalry heats up". CNBC.
- ^ 沛霖 (2024-08-30). "阿里通义千问推出 Qwen2-VL：开源 2B / 7B 参数 AI 大模型，处理任意分辨率图像无需分割成块". ithome.com.
- ^ Wang, Peng; Bai, Shuai; Tan, Sinan; Wang, Shijie; Fan, Zhihao; Bai, Jinze; Chen, Keqin; Liu, Xuejing; Wang, Jialin; Ge, Wenbin; Fan, Yang; Dang, Kai; Du, Mengfei; Ren, Xuancheng; Men, Rui; Liu, Dayiheng; Zhou, Chang; Zhou, Jingren; Lin, Junyang (September 18, 2024). "Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution". Cs.CV. arXiv:2409.12191.
- ^ Jiang, Ben (31 December 2024). "Alibaba Cloud cuts AI visual model price by 85% on last day of the year". South China Morning Post.
- ^ Franzen, Carl (8 August 2024). "Alibaba claims no. 1 spot in AI math models with Qwen2-Math". VentureBeat.
- ^ "Alibaba accelerates AI push by releasing new open-source models, text-to-video". Reuters. September 19, 2024.
- ^ Mims, Christopher (April 19, 2024). "Here Come the Anti-Woke AIs". WSJ.