Last Thursday, Meta CEO Mark Zuckerberg announced on Threads that he will focus on building comprehensive universal artificial intelligence and then release it as open source software to everyone. For this, Meta will prepare to gather the most powerful AI computing power in the industry. Zuckerberg stated that the company will purchase over 350000 Nvidia H100 GPUs, which are currently the most powerful chips in the industry for building generative AI performance.
01 | Meta is conducting a large entrepreneurial venture capital investment. Research from third-party investment institutions has estimated that Nvidia's H100 shipment to Meta can reach 150000 yuan in 2023, which is on par with its shipment to Microsoft and at least three times that of other companies. Zuckerberg stated that if Nvidia A100 and other artificial intelligence chips are included, Meta's GPU computing power will reach an equivalent of nearly 600000 H100 by the end of 2024. Although H100 has powerful performance, its price is also extremely high. The cost of each GPU is approximately $25000 (according to Nvidia's early product roster, the system consisting of 16 H100 GPUs costs approximately $400000) to $30000. If calculated based on this number, Meta's pursuit of universal artificial intelligence on GPUs could cost between $8.75 billion and $10.5 billion.
02 | Meta is training Llama
3Meta's new and broader focus on AGI has been influenced by its own Llama 2 model. Meta believes that the ability of Llama 2 to generate code is meaningless for people to use large models in Meta applications, but it remains one of the important skills for building smarter AI. However, Llama
The coding ability of 2 is relatively poor. Zuckerberg made a hypothesis: for large models, code capability doesn't seem to be that important because there won't be many people asking coding questions on WhatsApp. However, it has been proven that coding is very important as it allows large models to understand the rigor and hierarchy of knowledge, and often has more intuitive logic. Therefore, Zuckerberg revealed that Meta is training Llama
3 will have stronger code generation capabilities. And like Google's Gemini model, Llama
3 will also have advanced reasoning and planning abilities. From Llama
2 Jump to Llama
3 may not only be a simple extension, but may also be better than from Llama
Jump to Llama
2 requires a longer time. Llama2 has reached the GPT-3.5 level in some applications and has been optimized by the open source community through fine-tuning and additional features. For example, the recently released CodeLlama based on Llama2 was fine tuned in Human
The Eval coding benchmark achieved GPT-3.5 and GPT-4 levels (depending on the measurement type).
03 | The important issues of open source and closed source are different from OpenAI and Google
DeepMind, Aerospace, Cohere, and other companies choose to close source and make their most advanced models proprietary. Meta is one of the few companies that supports open source and chooses to publicly disclose its most advanced models. But it is worth noting that Llama
2 is not completely free. According to Meta's authorization terms, if in LlamaOn the date of release of version 2, if the monthly active users of products or services provided by the licensee or its affiliated companies exceed 700 million, the user or company must apply for a license from Meta, and Meta will strictly limit such authorization. 04 Jingtai Viewpoint | Meta joins the giants in intelligent combat. OpenAI and Google have both clearly committed to the development of General Artificial Intelligence (AGI), and now Meta has also joined the battle. Zuckerberg's goal is to promote AI more directly to his billions of users, which can be seen as product driven technology research and development. At present, the competition among major American giants around generative AI is becoming increasingly fierce, and they are engaging in various battles such as talent, computing power, products, ecology, and users. Each company has different focuses, but they will not allow themselves to have obvious shortcomings. So, is Meta unable to continue in the metaverse and transitioning to generative AI? This issue may not be the most important. The most important thing is whether Meta can build its own moat based on the social attributes and user traffic pool of the metaverse.
