Dynamic Buffers

This section introduces dynamic buffers—growable storage that adapts to data flow between producers and consumers.

Prerequisites

The Producer/Consumer Model

Dynamic buffers serve as intermediate storage between a producer (typically network I/O) and a consumer (your application code).

The flow:

  1. Producer writes data into the buffer

  2. Buffer grows as needed to accommodate data

  3. Consumer reads and processes data

  4. Buffer releases consumed data

This model decouples production rate from consumption rate—the buffer absorbs variations.

The DynamicBuffer Concept

template<typename T>
concept DynamicBuffer = requires(T& t, std::size_t n) {
    typename T::const_buffers_type;
    typename T::mutable_buffers_type;

    // Producer side
    { t.prepare(n) } -> std::same_as<typename T::mutable_buffers_type>;
    { t.commit(n) };

    // Consumer side
    { t.data() } -> std::same_as<typename T::const_buffers_type>;
    { t.consume(n) };

    // Capacity
    { t.size() } -> std::same_as<std::size_t>;
    { t.max_size() } -> std::same_as<std::size_t>;
    { t.capacity() } -> std::same_as<std::size_t>;
};

A dynamic buffer, within its capacity provides a potentially empty readable region and a potentially empty writeable region. Sizes of these regions change as operations are invoked on the dynamic buffer.

Producer Interface

prepare(n)

Returns a mutable buffer sequence mb for writing up to n bytes:

capy::MutableBufferSequence auto buffers = dynamic_buf.prepare(1024);  // Space for up to 1024 bytes

Effects: If n > max_size() - size() throws std::length_error(). Otherwise, ensures that a wirteable region of size at leats n exists and returns an object of type mutable_buffers_type representing this region.

Throws: std::bad_alloc in case an allocation is performed and fails.

Postcondition: buffer_size(buffers) >= n.

commit(n)

Transfers n bytes of from the beginning of the writeable storage to the end of the readable storage:

// After writing data:
dynamic_buf.commit(bytes_written);
// Data is now visible via data()

Let n1 be the smaller number of n and the size of the writeable region.

Effects: Removes n1 bytes from the front of the writeable region and adds them at the back of the readable region.

Notes: n can be smaller than the writeable region size.

Typical Producer Pattern

task<> read_into_buffer(Stream& stream, DynamicBuffer auto& buffer)
{
    // Prepare space
    auto space = buffer.prepare(1024);

    // Read into prepared space
    auto [ec, n] = co_await stream.read_some(space);

    buffer.commit(n);  // Make data readable
}

Consumer Interface

data()

Returns a const buffer sequence representing the readable region.

capy::ReadableBufferSequence auto readable = dynamic_buf.data();
// Process readable bytes

Postcondition: capy::buffer_size(readable) == dynamic_buf.size().

consume(n)

Removes n bytes from the front of readable data:

dynamic_buf.consume(processed_bytes);
// Those bytes are no longer in data()

Let n1 be the smaller of n and size().

Effects: Removes n1 bytes from the front of the readable region. Sets the size of the writeable rgion to zero. Invalidates all buffer sequences previously obtained via calls to data() or `prepare().

Typical Consumer Pattern

void process_buffer(DynamicBuffer auto& buffer)
{
    auto data = buffer.data();

    while (buffer_size(data) >= message_header_size)
    {
        auto msg_size = parse_header(data);
        if (buffer_size(data) < msg_size)
            break;  // Need more data

        process_message(data, msg_size);
        buffer.consume(msg_size);
        data = buffer.data();  // Refresh after consume
    }
}

Capacity Management

size()

Returns: The size of the readable region.

max_size()

Returns: The maximum possible sum o sizes of the readable region and the writeable region.

capacity()

Returns: Current allocated capacity.

Class invariant

size() <= capacity().

capacity() <= max_size().

DynamicBufferParam

When passing dynamic buffers to coroutines, use DynamicBufferParam for safe parameter handling:

template<typename DB>
concept DynamicBufferParam = DynamicBuffer<std::remove_reference_t<DB>>;

template<DynamicBufferParam Buf>
task<std::size_t> read_until(Stream& stream, Buf&& buffer, char delimiter);

This concept ensures proper handling of lvalues and rvalues, preventing dangling references across suspension points.

Provided Implementations

flat_dynamic_buffer

Linear storage with single-buffer sequences:

#include <boost/capy/buffers/flat_dynamic_buffer.hpp>

flat_dynamic_buffer buffer;
buffer.prepare(1024);
// ... write data ...
buffer.commit(n);

// data() returns a single const_buffer

Advantages:

  • Contiguous memory—good for parsing that needs contiguous data

  • Cache-friendly

Disadvantages:

  • May require copying when buffer wraps or grows

circular_dynamic_buffer

Ring buffer implementation:

#include <boost/capy/buffers/circular_dynamic_buffer.hpp>

circular_dynamic_buffer<1024> buffer;  // Fixed capacity

Advantages:

  • No copying on wrap—head/tail pointers move

  • Fixed memory footprint

Disadvantages:

  • data() may return two buffers (wrapped around end)

  • Fixed capacity

vector_dynamic_buffer

Backed by std::vector<char>:

#include <boost/capy/buffers/vector_dynamic_buffer.hpp>

std::vector<char> storage;
vector_dynamic_buffer buffer(storage);

Adapts an existing vector for use as a dynamic buffer.

string_dynamic_buffer

Backed by std::string:

#include <boost/capy/buffers/string_dynamic_buffer.hpp>

std::string storage;
string_dynamic_buffer buffer(storage);

Useful when you want the final data as a string.

Example: Line-Based Protocol

task<std::string> read_line(Stream& stream)
{
    flat_dynamic_buffer buffer;

    while (true)
    {
        // Prepare space and read
        auto space = buffer.prepare(256);
        auto [ec, n] = co_await stream.read_some(space);
        buffer.commit(n);
        if (ec)
            throw std::system_error(ec);

        // Search for newline in readable data
        auto data = buffer.data();
        std::string_view sv(
            static_cast<char const*>(data.data()), data.size());

        auto pos = sv.find('\n');
        if (pos != std::string_view::npos)
        {
            std::string line(sv.substr(0, pos));
            buffer.consume(pos + 1);  // Include newline
            co_return line;
        }
    }
}

Reference

Header Description

<boost/capy/concept/dynamic_buffer.hpp>

DynamicBuffer concept definition

<boost/capy/buffers/flat_dynamic_buffer.hpp>

Linear dynamic buffer

<boost/capy/buffers/circular_dynamic_buffer.hpp>

Ring buffer implementation

<boost/capy/buffers/vector_dynamic_buffer.hpp>

Vector-backed adapter

<boost/capy/buffers/string_dynamic_buffer.hpp>

String-backed adapter

You have now learned about dynamic buffers for producer/consumer patterns. This completes the Buffer Sequences section. Continue to Stream Concepts to learn about Capy’s stream abstractions.

Edit this Page