Industry news--Karmin Electronic (HK) Limited

Amazon Cloud Technology launches a new generation of self-developed chips
2023-12-04

• Amazon Graviton4 is currently the most powerful and energy-efficient self-developed chip in Amazon Cloud Technology, supporting a wide range of cloud workloads

• Amazon Traineum2 provides the highest computational performance on Amazon Cloud Technology for model training, improving training speed, reducing costs, and energy consumption

Amazon Cloud Technology announced at the 2023 re: Invent Global Conference that two series of its self-developed chip family have launched new generations, including Amazon Graviton4 and Amazon Traineum2, providing higher cost-effectiveness and energy efficiency for a wide range of workloads such as machine learning (ML) training and generative artificial intelligence (AI) applications. Graviton4 and Traineum2 are the latest innovations in Amazon Cloud Technology's self-developed chips. Amazon Cloud Technology continuously improves cost-effectiveness and energy efficiency with each generation of self-developed chips, providing customers with more choices beyond the latest chip and instance combinations based on AMD, Intel, Nvidia, and others, allowing Amazon Elastic Compute Cloud (Amazon EC2) to virtually run almost all applications and workloads for customers.

Compared to the current generation of Graviton3 processors, Graviton4 has a performance improvement of up to 30%, an increase of over 50% in independent cores, and a memory bandwidth improvement of over 75%, providing the best performance and energy efficiency for workloads running on Amazon EC2.

Compared to the first generation Training chip, Traineum2 has a training speed improvement of up to 4 times and can deploy up to 100000 chips in EC2 UltraClusters. It can train basic models (FMs) and large language models (LLMs) in extremely short time, while improving energy efficiency by up to 2 times.

David Brown, Vice President of Computing and Networking at Amazon Cloud Technology, stated: "Chips are the foundation of all user workloads, which is why Amazon Cloud Technology has always regarded this field as a crucial innovation area. By focusing chip design on the actual workloads that customers truly care about, we can provide customers with the most advanced cloud infrastructure. Graviton4 is the fourth generation of our series launched in just five years, and it is our most powerful and energy-efficient chip to date.", Provide support for a wide range of workloads for customers. With the widespread attention paid to generative AI, Tranium2 can help customers train machine learning models faster at lower costs and better energy efficiency

Graviton4 provides better cost-effectiveness and higher energy efficiency for customers across a wide range of workloads

Today, Amazon Cloud Technology offers over 150 types of Graviton based Amazon EC2 instances on a global scale, with over 2 million Graviton processors built and over 50000 customers. These customers cover the top 100 customers of EC2, who use Graviton based instances to provide the best value for money for their applications. Customers such as Datadog, DirecTV, Discovery, Formula 1 (F1), NextRoll, Nielsen, Pinterest, SAP, Snowflake, Sprinklr, Stripe, and Zendesk are using Graviton based instances to run a wide range of workloads, including databases, data analytics, network servers, batch processing, advertising services, application servers, and microservices. As customers migrate larger in memory databases and analysis workloads to the cloud, their requirements for computing, memory, storage, and networking also increase. For this, they need higher performance and larger instances to run these demanding workloads, while also optimizing costs. For these workloads, customers also hope to use more energy-efficient computing resources to reduce the impact on the environment. At present, many Amazon Cloud Technology hosting services support the use of Graviton, including Amazon Aurora, Amazon ElastiCache, Amazon Elastic MapReduce (Amazon EMR), Amazon MemoryDB, Amazon OpenSearch, Amazon Relational Database Service (Amazon RDS), Amazon Fargate, and Amazon Lambda, bringing Graviton's cost-effectiveness advantages to users who use these services.

Compared to the Graviton3 processor, the Graviton4 processor has a 30% improvement in performance, an increase of over 50% in independent cores, and an increase of over 75% in memory bandwidth. Graviton4 further enhances security through full encryption of high-speed physical hardware interfaces. The Amazon EC2 R8g memory optimization instance will adopt the latest Graviton4 to improve the efficiency of customers running high-performance databases, memory caching, big data analysis and other workloads. The R8g instance provides a larger instance size compared to the current generation R7g instance, with a 3x increase in virtual processor (vCPU) and memory. This allows users to handle larger amounts of data, larger workloads, obtain running results faster, and reduce total cost of ownership.The R8g instance based on Graviton4 is now available in preview and will be officially available in the coming months. To learn more about R8g instances based on Graviton4, please visit aws.amazon.com/ec2/instance types/r8g.

Traineum2's EC2 UltraClusters are committed to providing customers with the highest performance and most energy-efficient AI model training infrastructure in the cloud

The basic models and big language models behind the increasing number of generative AI applications nowadays require training with massive datasets. These models help customers reconstruct their user experience by creating a large amount of new content such as text, audio, images, videos, and even software code. The most advanced basic models and large language models today typically contain billions or even trillions of parameters or variables, requiring reliable high-performance computing power that can support tens of thousands of machine learning chips for expansion. Amazon Cloud Technology now offers a wide and in-depth range of Amazon EC2 instance options supported by machine learning chips, including the latest Nvidia GPU, Training, and Inferentia2. Numerous clients such as Databricks, Helixon, Money Forward, and the Amazon Search team are using Training to train large-scale deep learning models, benefiting from its many advantages such as high performance, scalability, reliability, and low cost. But even though they are already using the fastest acceleration instances today, customers still hope to achieve stronger performance and scale to train these increasingly complex models, thereby improving training speed, reducing costs, and reducing energy consumption.

The Traineum2 chip is designed for high-performance training of basic models and large language models with trillions of parameters or variables. Compared to the first generation Training chip, the Traineum2 has a performance improvement of up to 4 times, memory improvement of up to 3 times, and energy efficiency (performance per watt) improvement of up to 2 times. The Amazon EC2 Trn2 instance uses the latest Traineum2, with a single instance containing 16 Traineum acceleration chips. The Traineum2 instance is committed to expanding up to 100000 Traineum2 acceleration chips for customers in the new generation EC2 UltraClusters, and is connected to the Amazon Elastic Fabric Adapter (EFA) PB level network, providing up to 65 exaflops of computing power, allowing customers to obtain supercomputing level performance on demand. With this level of scale, customers can train a large language model with 300 billion parameters in weeks rather than months. By providing the highest horizontal scalability model training at significantly reduced costs, Trainum2 instances can help customers unlock and accelerate a new round of innovation in generative AI. To learn more about Trainum2, please visit aws. amazon. com/machine learning/training/.

Anthropic is an artificial intelligence security and research company, a leading advocate for responsible deployment of generative artificial intelligence, committed to creating reliable, interpretable, and controllable artificial intelligence systems. Anthropic has been using Amazon Cloud Technology since 2021. Recently, Anthropic has launched Claude - an AI assistant that focuses on providing assistance, harmlessness, and honesty. Tom Brown, co-founder of Anthropic, stated: Since supporting Amazon Bedlock, Claude has been widely adopted by Amazon Cloud Technology customers. Traineum2 will help us build and train models on a large scale, and for some workloads, Traineum2 has increased the speed of the first generation Traineum chip by at least four times. Our collaboration with Amazon Cloud Technology will help organizations of all sizes have the opportunity to simultaneously benefit from Anthropic's secure and advanced artificial intelligence systems, as well as Amazon "Cloud technology is a reliable cloud technology that unleashes new possibilities."

Databricks helps over 10000 organizations worldwide, including Comcast, Cond é Nast, and over 50% of Fortune 500 companies, unify their data, analysis, and AI. Naveen Rao, Vice President of Generative AI at Databricks, stated: Thousands of customers are running Databricks on Amazon Cloud Technology, using MosaicML to pre train, fine tune, and perform other operations on the basic models of various use cases. Amazon Training provides us with the scale, high performance, and low cost required to train Mosaic MPT models. Traineum2 makes it possible to build next-generation Mosaic MPT models faster, giving us the opportunity to provide customers with unprecedented scale and performance, helping them outperform others Previously, they launched their own generative AI applications faster

Prev：Pickering can switch up to 1kV for a new type of high-voltage SMD reed relay Next：Photovoltaic reshuffle! 70% of photovoltaic enterprises may be eliminated