Seastar's shared pointer

The programming and execution model of Seastar can really reduce a lot of the complexity normally encountered when building multi-threaded servers. For instance, it is common to see Seastar code written as if it is single threaded (and indeed it often is). This alone is a major boon for developers. It allows for faster development, and results in code being produced that is easier to inspect and understand. But Seastar doesn’t free developers from inherent complexity, it merely simplifies many common cases. Developers must still reason about and solve hard problems like managing concurrency, and controlling the lifetime of data.

Take memory management as an example. The introduction of smart pointers into the standard library (e.g. std::shared_ptr<T>) was a big deal. Smart pointers gave developers a way to avoid reproducing complex and error-prone code for managing the lifetime of memory resources. It’s common now to see very large code bases without a single instance of new or delete, and that’s generally a good thing.

But the generality of smart pointer implementations in the standard library poses challenges for Seastar, which are primarily performance related. The problem with std::shared_ptr<T> and its performance impact on Seastar applications is that a std::shared_ptr<T> assumes it will be used in a multi-threaded environment, and contains an atomic reference counter. This counter is a fairly expensive little gadget that wrecks all sorts of havoc on cache lines and multi-core scalability in the name of thread safety.

Luckily for Seastar applications, thread safety is often a non-issue. When state and execution are both pinned to a single core, Seastar ensures that only a single thread will ever access any given piece of state. Thus, a completely viable alternative implementation of std::shared_ptr<T> can eschew all the overhead of managing concurrency with an atomic counter by using a normal integer type to track the pointer’s reference count.

It’s hard to imagine building a reference counted smart pointer that is more efficient than incrementing and decrementing a primitive integer. But this is how the seastar::lw_shared_ptr<T> smart pointer is implemented, and it’s what you should generally reach for when needing to share ownership of some memory in a Seastar application.

Here’s a little example. The find_handler method below will return a shared pointer to a widget, but the remove_handler may be invoked asynchronously and remove the same widget. It would be a problem if find_handler returned a pointer to memory that had been freed. Other than using a Seastar-specific type, this example is likely to look familiar to anyone that’s used std::shared_ptr<T> to deal with a similar issue. As an aside, what looks like a race condition accessing widgets_ is actually safe in Seastar, too. But we’ll cover synchronization in a later post.

namespace ss = seastar;

class server {
 public:
  ss::future<ss::lw_shared_ptr<widget>> find_handler(int id) {
    return do_something().then([id] {
      auto it = widgets_.find(id);
      if (it != widgets_.end()) {
        return it->second;
      }
      return ss::lw_shared_ptr<widget>(nullptr);
    });
  }

  ss::future<> remove_handler(int id) {
    return do_something().then([id] {
      widgets_.erase(id);
      return do_something_else();
    });
  }

 private:
  std::unordered_map<int, lw_shared_ptr<widget>> widgets_;
};

The lw_ in ss::lw_shared_ptr<T> stands for lightweight and is usually sufficient for most use cases. However, if you need functionality like the ability to manage polymorphic types, reach for the ss::shared_ptr<T>. It still eliminates the expensive atomic reference counter, but carries around just a bit more state.

The major limitation of Seastar shared pointers are that they must only ever be accessed from a single core. It is a bug to send a copy of a shared pointer to another core as this would then result in multiple threads attempting to interact with the internal non-atomic reference counter. That would result in all sorts of bad things like memory leaks and accessing freed memory. Concretely, the following is a bug because the reference count may be modified on all cores simultaneously:

ss::lw_shared_ptr<widget> w;

ss::smp::invoke_on_all([w] {
  // do something with w
});

An exceedingly nice feature of the shared pointer implementation in Seastar is that when compiled with SEASTAR_DEBUG_SHARED_PTR accesses to the underlying reference counter are checked for correct core affinity. That is, a check is made that the reference counter is only read and modified on the core where the pointer was created. It’s not a guarantee that misuses will be found–an execution that violates the constraints must actually run–but it’s a powerful tool nonetheless that should be used.