Breakthrough AI Chips, GPT-4 Voice Delay, Claude 3.5 Dominance, and AI-Powered Games

The first story today is about a new AI chip company called Etched, which claims to be able to generate over 500,000 tokens per second running LLaMA 70B. This chip, called Sohu, is the first specialized ASIC (Application-Specific Integrated Circuit) for Transformer models.

Sohu is said to be more than 10 times faster and cheaper than even Nvidia's next-generation Blackwell GPUs. One Sohu server can run over 500,000 LLaMA 70B tokens per second, which is 20 times more than an Nvidia H100 and 10 times more than a Blaize B2.

The key advantage of Sohu is its specialization for Transformer models, which are the foundation of most major AI products today. The company claims that as Transformer models become more dominant, custom chips like Sohu will be essential, as GPUs are not improving in compute density fast enough to keep up.

Etched believes that the architecture that runs Transformers the fastest and cheapest on hardware will win the "hardware lottery." They argue that AI labs are spending hundreds of millions optimizing kernels for Transformers, and startups are using specialized Transformer software libraries, making it difficult for "Transformer Killers" to catch up on GPUs.

Overall, the Sohu chip from Etched represents a significant advancement in AI hardware, potentially disrupting the dominance of GPUs in this space.

OpenAI has announced a delay in the rollout of the advanced voice mode for ChatGPT, which was previously demonstrated during the spring update. The company cites the need to improve the model's ability to detect and refuse certain content, as well as to enhance the user experience and prepare the infrastructure to scale to millions of users while maintaining real-time responses.

The company plans to start rolling out the alpha version of the advanced voice mode to a small group of ChatGPT Plus users in late June, with the goal of making it available to all Plus users in the fall. However, the exact timeline will depend on meeting their high safety and reliability standards.

OpenAI is also working on rolling out new video and screen-sharing capabilities, which will be announced separately. The advanced voice mode is expected to bring ChatGPT closer to real-time, natural conversations with AI, with the ability to understand and respond with emotions and non-verbal cues.

The delay in the launch of these features is disappointing, as it echoes the prolonged wait for the release of Whisper, OpenAI's voice model. However, the company emphasizes the importance of ensuring a high level of safety and reliability before making these capabilities widely available.

Anthropic's Claude 3.5 model has made a significant leap, securing the number one spot in the coding Arita hard prompts Arena and the number two spot in the overall leaderboard. The new Sonet model has surpassed Opus at a fraction of the cost, making it competitive with Frontier models like GPT-4 and Gemini 1.5 Pro across the board.

In a previous video, the creator tested the Claude 3.5 Sonet model and found it to be the best model they had ever tested. The model completely demolished the creator's tests, prompting a request for new, more challenging tests.

Comparisons between Claude 3.5 and GPT-4 on coding tasks show that Claude 3.5 outperforms GPT-4 in task success and full project success, although GPT-4 has a slightly higher build success rate. Additionally, Claude 3.5 is more verbose, which can be beneficial for longer pieces of code, but may slow down generation. The model also tends to be less reliable in following instructions in large prompts compared to GPT-4.

Overall, the dominance of Anthropic's Claude 3.5 model in the coding and hard prompts arenas is a significant achievement, showcasing the continued advancements in large language models and their applications.

The video showcased by Twitter user chubby is truly remarkable, as it demonstrates the incredible capabilities of AI in generating realistic video game footage. The visuals, which appear to be inspired by a Call of Duty-style game, are stunningly realistic, with the AI-generated sound and visuals blending seamlessly to create an immersive experience.

While the quality of the footage is impressive, the amount of compute power required to achieve this level of realism in real-time is still a significant challenge. As mentioned by Jensen, the CEO of Nvidia, this is the future of video games, and it will take the industry to new heights.

The video highlights the rapid advancements in AI-powered content generation, where the line between reality and artificial creation is becoming increasingly blurred. As the technology continues to evolve, we can expect to see even more impressive and realistic AI-generated video game experiences in the near future.

Apple has decided not to integrate Meta's AI models, such as the Llama 3 model, into Siri due to privacy concerns. This decision comes just days after reports surfaced that Apple was in talks with Meta to integrate the latter's AI models.

The primary reason for Apple's decision is the company's concerns over Meta's privacy practices. Apple has been vocal in its criticism of Meta's approach to user privacy, and the tech giant likely does not want to be associated with a company that has faced numerous privacy-related scandals.

Even though Apple could have hosted the AI models themselves and maintained control over the privacy aspects, the company still opted against the integration. This suggests that Apple's concerns go beyond just the technical implementation and extend to the broader reputational risks of partnering with Meta.

The decision highlights Apple's commitment to user privacy and its willingness to forgo potential AI capabilities if they come at the expense of its core values. It also underscores the growing importance of privacy in the tech industry, as companies navigate the balance between innovation and protecting user data.