Introduction

The MPMC (Multi-Producer, Multi-Consumer) queue is a crucial data structure in concurrent programming, enabling multiple producers and consumers to share data efficiently. A recent design [1] has introduced a wait-free MPMC queue with bounded waiting times, ensuring that no producer or consumer will experience starvation. In this article, we will delve into the design and its advantages, as well as the benchmarked performance results.

Design Overview

The queue works based on a ticket lock wait system, where producers and consumers take tickets from respective dispensers and wait for their turn to access the shared buffer [2]. The design uses two AtomicUsize counters, one for producers and one for consumers, along with two ring buffers: a data buffer for storing items and a state buffer for tracking ownership of each entry [3].

Advantages

The design offers several advantages, including:

  • No CAS Loops, Minimal Cache Contention: The queue minimizes cache contention by allowing producers and consumers to race ahead of each other, reducing the need for CAS loops [4].
  • Bounded Waiting: The queue ensures bounded waiting times, preventing starvation and ensuring that all enqueue and dequeue operations will finish eventually [5].
  • Minimize Head of Line Blocking: The design minimizes head of line blocking by allowing producers and consumers to proceed independently, reducing the impact of slow or hung operations [6].

Benchmark Results

The queue's performance has been benchmarked using the criterion crate, measuring the time it takes to enqueue and dequeue $2^{24}$ items [7]. The results show promising performance, with the queue scaling well as the number of producers and consumers increases.

Conclusion

The MPMC queue design with bounded waiting offers a significant improvement in concurrent programming, ensuring efficient and starvation-free data transfer between producers and consumers. The benchmarked performance results demonstrate the queue's scalability and efficiency, making it a valuable addition to the field of concurrent programming.

Sources

  1. https://nahla.dev/blog/waitfree_queue/
  2. https://www.reddit.com/r/rust/comments/1up0uhg/girls_just_wanna_have_fast_waitfree_mpmc_queues/
  3. https://github.com/max0x7ba/atomic_queue
  4. https://news.ycombinator.com/item?id=48809574
  5. https://nahla.dev/blog/waitfree_queue/
  6. https://www.reddit.com/r/rust/comments/1up0uhg/girls_just_wanna_have_fast_waitfree_mpmc_queues/
  7. https://github.com/max0x7ba/atomic_queue