Good morning! Google has donated $1 million to help integrate the Rust programming language with existing C++ codebases, aiming to make Rust a “first class” language like C++ for engineering use. To manage billions of real-time emojis sent during the 2019 Cricket World Cup event, Disney+ Hotstar engineered a custom publish-subscribe architecture focused on throughput and concurrency. An NPM package called “everything” that depends on over 3,000 other packages has caused issues, tightening package coupling in undesirable ways and preventing easy fixes.

— Forrest Knight & Meghanadh Vasireddy

Google Gives $1M to Make Rust and C++ Work Better Together

Google has donated $1 million to the Rust Foundation to help integrate the Rust programming language with existing C++ code. Rust prevents common vulnerabilities but interfacing with legacy C++ systems poses adoption challenges.

What the Money Will Fund

The grant will support developing tools and libraries that:

• Make it easier to intermix Rust code with current C++ codebases

• Allow incremental adoption of Rust without high-risk rewrites

• Aim to elevate Rust to a “first-class” status like C++ for engineering use

As shared by Android head Lars Bergstrom, integrating Rust into C++ projects represents the biggest hurdle to adoption. While C APIs provide basic interface capabilities, direct C++ interoperation enables optimal performance. The funding mirrors a similar $1 million Microsoft donation with aligned goals.

Perspective from the Rust Foundation: Rust Foundation CEO Rebecca Rumbul acknowledges expecting full C++ rewrites to Rust is impractical. This initiative instead focuses on simplifying integrating Rust into continuing C++ workflows.

For organizations invested in legacy C++, the funding could speed up incremental, low-risk Rust adoption while still benefiting from its security advantages. It aims to tackle a primary barrier in leveraging Rust’s protections.

Read More Here

Disney+ Hotstar Hits Billions of Real-Time Emojis with Custom PubSub

During the 2019 Cricket World Cup, Disney+ Hotstar faced a major technical challenge - over 55 million excited fans sending over 5 billion emojis in real-time. To create an engaging, interactive viewing experience, Hotstar wanted users to express reactions via emojis while streaming matches.

Managing emojis at this massive scale required a custom publish-subscribe (pub-sub) architecture engineered specifically for throughput and concurrency.

The Front-End: Hotstar used HTTP APIs to receive emojis, buffering them locally before writing asynchronously to Kafka queues. This prevented blocking user connections. Spark Streaming aggregated the real-time emoji stream in small batches for processing. Checkpointing enabled fault tolerance. The normalized emoji data was published to another Kafka topic for delivery.

The Delivery System

• Multi-cluster MQTT setup with EMQX message broker

• Reverse bridge to connect clusters

• Autoscaling support

Read More Here

The "Everything" NPM Package Messes Things Up

A user named PatrickJS recently published a joke NPM package called "everything" that depends on every public NPM package available, totaling over 3,000 indirect dependencies. This was likely meant to be a prank, but it ended up seriously disrupting the NPM community.

What Went Wrong

By depending on so many packages, the "everything" package makes computers struggle to install it properly. Specifically:

• It can use up all of a computer's storage space and crash it when someone attempts to install the package

• NPM has rules that don't allow unpublished packages that are depended on by other packages. So PatrickJS can't simply remove his problematic "everything" package now even though he wants to.

Additionally, this incident showed flaws in how connected NPM packages are to each other. When one package depends on thousands of others like this, it:

• Removes control from authors of those dependent packages - they can't unpublish their own creations anymore

• Tightly couples unrelated packages, threatening overall security and stability

Read More Here

🔥 More Notes

1. Announcing Rust 1.76.0

2. StackOverflow Questions Down 66% in 2023 Compared to 2020

3. How Apple built iCloud to store billions of databases

4. Why Bloat Is Still Software’s Biggest Vulnerability

5. Anti-Bot Fingerprinting: How They Know You're Human, And How To Fool Them

🎥 Youtube Spotlight

Building a Gamified Productivity App with Godot 4

Ben discusses the creation of a gamified productivity app called “Range” using Godot 4. The app uses the Pomodoro Technique to structure working sessions and breaks, with a theme of embarking on a road trip during working sessions and relaxing at a campsite during breaks.

Was this forwarded to you? Sign Up Here