Ethernet is DEAD?? Mac Studio is 100x FASTER!!

About this video

The creator revisits Mac Studio clustering after an earlier five-machine cluster ran, by his account, 91% slower. Apple loaned him four fully specced Mac Studios — the creator says each has 512 GB of unified memory, 8 TB of storage and 80 GPU cores, for 2 TB of unified memory, 32 TB of storage and 320 GPU cores in total, a cluster he prices at $50,000 and compares to needing 26 Nvidia H100s costing over $780,000. The machines are meshed over Thunderbolt 5 plus Ethernet for downloads, but the real change is software: in macOS Tahoe 26.2 Apple enabled RDMA over Thunderbolt, enabled via recovery mode, which bypasses the TCP/IP stack and CPU for direct GPU-memory-to-GPU-memory access. He says this cuts latency from about 300 microseconds to about 3 microseconds. That matters because pipeline parallelism splits model layers sequentially between machines and wastes time waiting, while tensor parallelism splits the math for each layer across all machines — far more chatty, requiring about 160 communications per token — and was previously slower. With Exo Labs software and tensor parallelism plus RDMA, he reports Llama 3.3 70B rising from about 5 tokens per second (pipeline) and 3 tokens per second (tensor without RDMA) to about 16 tokens per second. A small Llama 3.2 3B model went from 147 tokens per second on one Mac to 240 clustered, and Qwen3 Coder 480B from 27 to 40. He also runs the trillion-parameter Kimi K2 Thinking model at roughly 28-30 tokens per second and loads multiple large models simultaneously, then drives them from Open WebUI, Xcode and Open Code via Exo’s API, while noting beta-software failures and lock-ups.

Key takeaways

• The creator claims RDMA over Thunderbolt in Tahoe 26.2 cuts inter-Mac latency from about 300 microseconds to about 3 microseconds by bypassing TCP/IP and the CPU
• Pipeline parallelism splits layers sequentially and creates waiting; tensor parallelism splits each layer’s math across all machines but needs about 160 communications per token
• With tensor parallelism plus RDMA, Llama 3.3 70B reportedly improved from about 5 tokens per second to about 16 tokens per second
• Four 512 GB Mac Studios give a claimed 2 TB of unified memory, enough to run trillion-parameter and 671B models locally and several models at once
• Clustering reportedly now speeds up even models that fit on one machine, but the creator stresses the software is beta and the $50,000 cluster was loaned by Apple as a proof of concept

Episode details: published December 20, 2025 · 33:14 long. Summary written by Self Growth Videos from the episode itself. Business results and any income figures mentioned are NetworkChuck’s own claims, shared for education — they are not typical or guaranteed.

← Previous: stop trusting cloud cameras!! (here’s what I use instead) · Next: FREE Phone Calls with Claude Code →

All NetworkChuck videos · NetworkChuck profile · NetworkChuck on YouTube