- Apple has deployed an internal system called ‘Minions’ to leverage idle Mac GPUs for AI inference workloads.
- The system dramatically reduces GPU queue times for Apple teams, from over ten hours to mere minutes.
- Minions creates a dynamic, distributed virtual GPU cloud by aggregating Apple Silicon resources for large-scale machine learning inference.
Apple Unleashes ‘Minions’ to Create Virtual GPU Cloud from Idle Macs
Apple has introduced an internal system named Minions, designed to harness the power of idle Graphics Processing Units (GPUs) across its Mac devices, transforming them into a distributed virtual GPU cloud for artificial intelligence (AI) inference tasks. This initiative aims to provide Apple’s employees with more readily available GPU resources.
Apple researchers describe Minions as “a dynamic and distributed virtual GPU cloud that aggregates idle Apple Silicon GPUs across macOS devices to execute large-scale machine learning inference workloads.” Prior to its adoption, some inference jobs faced queue times exceeding ten hours due to limited centralized GPU availability. With Minions, tasks can often begin execution within minutes by leveraging idle macOS devices.
Minions Architecture and Operation
The Minions system is engineered to create a fault-tolerant pool of inference workers within a shared distributed inference platform. It comprises four primary components:
- Minion Server: Provides the coordination layer, receiving user requests and breaking them down into subtasks.
- PostgreSQL Metadata Store: Stores task metadata for the Minion Server.
- Minion Executor: Runs inference tasks on a Mac’s GPU, uploads results to S3, and reports execution status via the Minion Server.
- Minion Watchdog: Monitors the process for failures, such as a Mac becoming unresponsive, and re-assigns tasks as needed.
Minions operates on a pull-based execution model, where each Minion Executor on a Mac pulls the next subtask via REST APIs. Inputs are read from Amazon S3, models are fetched if required, and results are returned to S3. The system supports two types of jobs: embeddings using open-source Hugging Face transformer models, and text generation from large language models via Ollama, which is installed on each participating Mac.
Impact and Benefits for Internal Workflows
The implementation of Minions has yielded significant benefits:
- Reduced Wait Times: GPU queues that once stretched over 10 hours are now reduced to minutes.
- Resource Optimization: Idle GPU resources are put to work, maximizing the utilization of existing hardware.
- Cost Efficiency: The system eliminates the need for new GPU requests and zeroes infrastructure costs, shifting scaling from manual to organic.
While Minions is primarily intended to redistribute idle resources rather than run models too large for a single machine, newer Mac mini and Mac Studio models support clustering via RDMA over Thunderbolt 5, which could potentially combine compute and memory for single AI workloads. However, the current Minions implementation runs an entire model on a single Mac’s hardware.
Confidentiality concerns are addressed through team- and organization-scoped deployments with separate databases and Mac pools. Future plans for Minions include extending it to distributed model training and potentially forming a shared GPU pool from securely federated idle Macs across organizations or even globally. This development comes as compute resources increasingly become a bottleneck for AI workloads, a concern Apple acknowledged in its July quarterly filing, citing “constrained supply, extended lead times, and increasing costs” for computing resources. For more on Apple’s broader AI strategy, see our report on Apple’s recent smart home lineup announcements, which also touch upon its ecosystem developments.
For Apple, Minions represents a strategic move to efficiently manage the growing demand for AI compute resources, especially amid supply constraints and rising costs of specialized hardware. The system optimizes internal processes, significantly accelerating AI model development and testing. For developers and the industry, it showcases the potential of distributed computing leveraging existing devices, although Minions is currently an internal Apple initiative. The approach emphasizes hardware availability and utilization, distinct from Nvidia’s focus on throughput for large models, highlighting a different strategy in the evolving AI landscape.
The Minions system represents a highly strategic internal innovation for Apple, effectively democratizing access to GPU compute for AI inference across their vast internal ecosystem of Apple Silicon Macs. The reduction in queue times from ten hours to minutes is a critical accelerator for ML development cycles. This distributed approach, leveraging existing idle hardware, not only optimizes resource utilization but also significantly mitigates the escalating CapEx associated with dedicated GPU clusters, particularly when considering the high cost of enterprise-grade GPUs. The integration with Hugging Face and Ollama further underscores its practical utility for contemporary LLM workloads.