Good morning! First, we’re discussing the latest upgrades to Google Cloud's AI tools, including improvements to the Gemini models for code generation and inference. Also we’re explaining Netflix's development of a predictive CPU isolation system called titus-isolate to optimize container placement and improve performance. Finally, Intel's unveiling of its powerful new Gaudi 3 AI accelerator and its strategy to create an open platform for enterprise AI.

— Forrest Knight & Meghanadh Vasireddy

Gemini Upgrades Improve Google Cloud's AI Tools

Google Cloud Next was this week, and the company unveiled a lot of innovations related to AI, such as two new Gemma models for code generation and inference.

Google announced that Gemini 1.5 Pro will be entering public preview for Google Cloud customers, and it's available through Vertex AI. This version of the model was benefited by a breakthrough in long context understanding that allows it to run 1 million tokens of information consistently. This opens up use cases, such as enabling a gaming company to provide gamers with a video analysis of their performance and tips to improve.

Gemini Code Assist has also been upgraded with Gemini 1.5 Pro and the larger context window improves its ability to provide code suggestions and enables deeper insights. Google's Threat Intelligence is also bolstered by the improvement and can now analyze larger samples of malicious code.

Additionally, Gemini for Google Workspace is getting a new feature called Google Vids that allows users to create videos for work. It also added Gemini improvements across Gmail, Meet, and Chat.

As Sundar Pichai, CEO of Google, wrote in a blog post: "Just as cloud computing changed how businesses worked a decade ago, AI is going to drive incredible opportunity and progress all over again. Google Cloud is how we'll continue to help organizations everywhere do transformational things, and we can't wait to see what's next."

Read More Here

Predictive CPU Isolation at Netflix

Netflix runs millions of containers every month on their Titus platform, supporting a wide range of critical services and batch workloads. Maintaining performance isolation between these different applications is crucial for providing a good experience for both internal and external customers. Traditionally, the Linux kernel's Completely Fair Scheduler (CFS) has been responsible for managing CPU resources and mitigating performance interference between co-located containers. However, Netflix found that CFS's heuristics were often suboptimal for their diverse workloads.

To address this, Netflix developed a system called titus-isolate that takes a more data-driven approach to container placement. The key ideas are:

Predictive Modeling: Titus-isolate uses a gradient boosted regression model to predict the future CPU usage of each container, based on historical telemetry and container metadata.

Combinatorial Optimization: With the predicted CPU needs, titus-isolate formulates the container placement problem as a mixed-integer program and solves it to find an optimal allocation that minimizes cache contention and resource fragmentation.

The impact has been substantial:

• Netflix saw average runtime improvements of multiple percent for batch jobs, while dramatically reducing the long tail of outliers.

• For latency-sensitive services, they were able to reduce capacity requirements by over 13% while maintaining the same service level agreements.

Read More Here

Intel Unveils Powerful New AI Chip and Open Platform Strategy

At its Intel Vision 2024 event, the company made a series of major announcements that demonstrate its commitment to making AI accessible, open, and scalable for enterprises across all workloads. The centerpiece of Intel's AI strategy is the unveiling of the Intel Gaudi 3 AI accelerator. Compared to Nvidia's top-of-the-line H100 GPU, Gaudi 3 promises some impressive performance gains:

• 50% faster training time on large language models like GPT-3 and Llama2

• 50% higher inference throughput

• 40% better power efficiency

But the capabilities of the Gaudi 3 hardware are just one part of Intel's comprehensive AI strategy. The company also announced plans to create an open platform for enterprise AI, in collaboration with a wide ecosystem of partners including SAP, Red Hat, VMware, and many others. This open platform aims to simplify the deployment of secure, high-performance generative AI systems that can leverage enterprises' existing data and infrastructure. As part of this effort, Intel will release reference implementations, publish technical frameworks, and expand its Tiber Developer Cloud.

Intel is further strengthening its AI offerings with the new Intel Xeon 6 processor line. The Xeon 6 will come in two variants - efficient E-cores for power-sensitive workloads, and high-performance P-cores optimized for AI. Combining these new Xeon processors with Intel's client roadmap, including the upcoming Core Ultra chips with over 100 TOPS of AI performance, Intel is delivering a unified AI platform from the edge to the data center.

Read More Here

🔥 More Notes

• Postman Acquires Orbit to Boost API Collaboration Among Developer Communities on the Postman API Network

• Our Response to Hashicorp's Cease and Desist Letter

• Army announces new policy to drive adoption of agile software development practices

Youtube Spotlight

A Complete Overview of DevOps

Was this forwarded to you? Sign Up Here