Crate rusteron_archive

rusteron-archive

rusteron-archive is a module within the rusteron project that provides functionalities for interacting with Aeron’s archive capabilities in a Rust environment. This module extends rusteron-client by offering features for recording streams, managing archives, and handling replay capabilities.

Overview

The rusteron-archive module is intended to help Rust developers leverage Aeron’s archive functionalities, including the recording and replaying of messages.

For quick start use static lib with precompiled C libraries if your running on a mac. This will not require cmake or java to be installed on your system.

rusteron-archive = { version = "0.1", features= ["static", "precompile"] }

Installation

Add rusteron-archive to your Cargo.toml:

dynamic lib:

[dependencies]
rusteron-archive = "0.1"

static lib

[dependencies]
rusteron-archive = { version = "0.1", features= ["static"] }

static lib using precompile code (no cmake or java required)

[dependencies]
rusteron-archive = { version = "0.1", features= ["static", "precompile"] }

You must also ensure that you include Aeron C libraries required by rusteron-archive when using default feature. (Using static will automatically include these dependancies in binary).

Features

• Stream Recording: Enables recording of Aeron streams.
• Replay Handling: Provides capabilities for replaying recorded messages.
• Publication: Send messages to various Aeron channels.
• Subscription: Receive messages from Aeron channels.
• Callbacks: Handle events such as new publications, new subscriptions, and errors.
• Automatic Resource Management (new method only): The wrappers attempt to automatically manage resources, specifically when using the new method. This includes calling the appropriate xxx_init method during initialisation and automatically invoking xxx_close or xxx_destroy methods (if one exists) during cleanup. However, this management is partial. For other methods, such as AeronArchive::set_aeron, it is the developer’s responsibility to ensure that the arguments remain valid and alive during their use. Proper resource management beyond initialisation requires manual handling by the user to avoid undefined behaviour or resource leaks.

General Patterns

Much like rusteron-client, the rusteron-archive module follows several general patterns to simplify usage of Aeron functionalities in Rust:

• Cloneable Wrappers: All Rust wrappers in rusteron-archive can be cloned, and they will refer to the same underlying Aeron C instance/resource. This allows safe use of multiple references to the same object. If you need a shallow copy, use clone_struct(), which copies only the underlying C struct.

• Mutable and Immutable Operations: Modifications can be performed directly with &self, allowing flexibility without needing additional ownership complexities.

• Automatic Resource Management (new method only): The wrappers attempt to automatically manage resources, clearing objects and calling the appropriate close, destroy, or remove methods when needed.

• Manual Handler Management: Callbacks and handlers require manual management. Handlers are passed into the C bindings using Handlers::leak(xxx), and need to be explicitly released by calling release(). This manual process is required due to the complexity of determining when these handlers should be cleaned up once handed off to C. For methods where the callback is not stored and only used there and then e.g. poll, you can pass in a closure directory e.g.

  subscription.poll_once(|msg, header| { println!("msg={:?}, header={:?}", msg, header) })

Handlers and Callbacks

Handlers within rusteron-archive work just like those in rusteron-client. You can attach and manage them using two main approaches:

Defining a trait for your handler and implementing it within your own struct is the recommended, most performant approach. For instance:

The recommended approach is to define a trait for your handler and implement it within your own struct. This pattern is performant and safe as it does not require additional allocations. For instance:

use rusteron_archive::*;

pub trait AeronErrorHandlerCallback {
    fn handle_aeron_error_handler(&mut self, errcode: ::std::os::raw::c_int, message: &str) -> ();
}

pub struct AeronErrorHandlerLogger;

impl AeronErrorHandlerCallback for AeronErrorHandlerLogger {
    fn handle_aeron_error_handler(&mut self, errcode: ::std::os::raw::c_int, message: &str) -> () {
        eprintln!("Error {}: {}", errcode, message);
    }
}

By passing instances of this trait to the archive context, you gain a reusable and safe way to respond to errors or other events without incurring unnecessary runtime overhead.

Wrapping Callbacks with Handler

Callbacks must be wrapped in a Handler. This ensures proper integration with the Aeron C API. Use Handlers::leak(xxx) to pass a handler into the C bindings. When your handler is no longer needed, call release() to free resources and avoid memory leaks.

Wrapping Callbacks with Handler

Regardless of the approach, callbacks must be wrapped in a Handler. This ensures proper integration with the Aeron C API. Use Handlers::leak(xxx) to pass a handler into C bindings. When your handler is no longer needed, call release() to free up resources and avoid memory leaks.

Handler Convenience Methods

If you do not need to set a particular handler, you can pass None. However, doing so manually can be awkward due to static type requirements. To simplify this, rusteron-archive (like rusteron-client) provides convenience methods prefixed with Handlers::no_..., returning None with the correct type signature. For example:

use rusteron_archive::*;
impl Handlers {
    #[doc = r" No handler is set i.e. None with correct type"]
    pub fn no_error_handler_handler() -> Option<& 'static Handler<AeronErrorHandlerLogger>> {
        None::<&Handler<AeronErrorHandlerLogger>>
    }
}

These methods make it easy to specify that no handler is required, keeping your code concise.

Error Handling with Aeron C Bindings

rusteron-archive relies on the same Aeron C bindings as rusteron-client, using i32 error codes to indicate the outcome of operations. In Rust, these are wrapped within a Result<i32, AeronCError> to provide clearer, more idiomatic error handling.

Error Type Enum

The AeronErrorType enum defines various error types that may occur:

Error Type	Description
NullOrNotConnected	Null value or not connected
ClientErrorDriverTimeout	Driver timeout error
ClientErrorClientTimeout	Client timeout error
ClientErrorConductorServiceTimeout	Conductor service timeout error
ClientErrorBufferFull	Buffer is full
PublicationBackPressured	Back pressure on publication
PublicationAdminAction	Admin action during publication
PublicationClosed	Publication has been closed
PublicationMaxPositionExceeded	Maximum position exceeded for publication
PublicationError	General publication error
TimedOut	Operation timed out
Unknown(i32)	Unknown error code

These error types help provide more context on the underlying issues when working with Aeron. For example, if a publication is closed or back-pressured, these specific errors can be captured and managed accordingly.

The AeronCError struct encapsulates the error code and provides methods to retrieve the corresponding error type and a human-readable description. Error handling in rusteron-client is designed to make working with Aeron C bindings more ergonomic by providing clear error types and descriptions for easier debugging.

Safety Considerations

Resource Management:

1. Lifetime of Aeron: The AeronArchive does not take full ownership or manage the lifetime of the Aeron instance. Instead, it calls AeronArchive::set_aeron, meaning you must ensure the Aeron object remains valid throughout the archive’s usage. Dropping or losing reference to the Aeron too soon can lead to segmentation faults or undefined behaviour.

2. Unsafe Bindings: Since rusteron-archive relies on Aeron C bindings, you must carefully manage resources (publishers, subscriptions, handlers, etc.) to avoid crashes or undefined behaviour. This includes ensuring you do not publish messages after closing the Aeron client or the associated archive context.

3. Partial Automatic Resource Management: While constructors aim to manage resources automatically, many aspects of resource lifecycles remain manual. For instance, handlers require a call to release() to clean up memory. Be especially cautious in multithreaded environments, ensuring synchronisation is properly handled.

Failure to follow these guidelines can lead to unstable or unpredictable results.

Workflow Overview

1. Initialise Contexts: Set up archive and client contexts.
2. Start Recording: Begin recording a specified channel and stream.
3. Publish Messages: Send messages to be captured by the archive.
4. Stop Recording: Conclude the recording session.
5. Locate Recording: Identify and retrieve details about the recorded stream.
6. Replay Setup: Configure replay parameters and replay the recorded messages on a new stream.
7. Subscribe and Receive: Subscribe to the replay stream, receiving the replayed messages as they appear.

Building This Project

For full details on building the rusteron project, please refer to the HOW_TO_BUILD.md file.

Benchmarks

You can view the benchmarks for this project by visiting BENCHMARKS.md.

Contributing

Contributions are welcome! Please see our contributing guidelines for more information on how to get involved.

License

This project is dual-licensed under either the MIT License or Apache License 2.0. You may choose which one to use.

Links

• Documentation on docs.rs
• API Reference on github
• GitHub Repository

Feel free to reach out with any questions or suggestions via GitHub Issues!

Features

• static: When enabled, this feature statically links the Aeron C code. By default, the library uses dynamic linking to the Aeron C libraries.
• backtrace - When enabled will log a backtrace for each AeronCError
• extra-logging - When enabled will log when resource is created and destroyed. useful if your seeing a segfault due to a resource being closed
• precompile - When enabled will use precompiled c code instead of requiring cmake and java to me installed

Modules

bindings

testing

Structs

Aeron

AeronAgentStartFuncLogger

AeronArchive

AeronArchiveAsyncConnect

AeronArchiveContext

AeronArchiveControlResponsePoller

AeronArchiveCredentialsChallengeSupplierFuncLogger

AeronArchiveCredentialsEncodedCredentialsSupplierFuncLogger

AeronArchiveCredentialsFreeFuncLogger

AeronArchiveDelegatingInvokerFuncLogger

AeronArchiveEncodedCredentials

AeronArchiveProxy

AeronArchiveRecordingDescriptor

Struct containing the details of a recording

AeronArchiveRecordingDescriptorConsumerFuncLogger

AeronArchiveRecordingDescriptorPoller

AeronArchiveRecordingSignal

Struct containing the details of a recording signal.

AeronArchiveRecordingSignalConsumerFuncLogger

AeronArchiveRecordingSubscriptionDescriptor

Struct containing the details of a recording subscription

AeronArchiveRecordingSubscriptionDescriptorConsumerFuncLogger

AeronArchiveRecordingSubscriptionDescriptorPoller

AeronArchiveReplayMerge

AeronArchiveReplayParams

Struct containing the available replay parameters.

AeronArchiveReplicationParams

Struct containing the available replication parameters.

AeronAsyncAddCounter

AeronAsyncAddExclusivePublication

AeronAsyncAddPublication

AeronAsyncAddSubscription

AeronAsyncDestination

AeronAsyncDestinationById

AeronAvailableCounterLogger

AeronAvailableCounterPair

AeronAvailableImageLogger

AeronBlockHandlerLogger

AeronBufferClaim

Structure used to hold information for a try_claim function call.

AeronCError

Represents an Aeron-specific error with a code and an optional message.

AeronClientRegisteringResource

AeronCloseClientLogger

AeronCloseClientPair

AeronCnc

AeronCncConstants

AeronCncMetadata

AeronContext

AeronControlledFragmentAssembler

AeronControlledFragmentHandlerLogger

AeronCounter

AeronCounterConstants

Configuration for a counter that does not change during it’s lifetime.

AeronCounterMetadataDescriptor

AeronCounterValueDescriptor

AeronCountersReader

AeronCountersReaderBuffers

AeronCountersReaderForeachCounterFuncLogger

AeronDataHeader

AeronError

AeronErrorHandlerLogger

AeronErrorLogReaderFuncLogger

AeronErrorLogger

AeronExclusivePublication

AeronFragmentAssembler

AeronFragmentHandlerLogger

AeronFrameHeader

AeronHeader

AeronHeaderValues

AeronHeaderValuesFrame

AeronIdleStrategyFuncLogger

AeronImage

AeronImageConstants

Configuration for an image that does not change during it’s lifetime.

AeronImageControlledFragmentAssembler

AeronImageFragmentAssembler

AeronIovec

AeronIpcChannelParams

AeronLogBuffer

AeronLogbufferMetadata

AeronLossReporter

AeronLossReporterEntry

AeronLossReporterReadEntryFuncLogger

AeronMappedBuffer

AeronMappedFile

AeronMappedRawLog

AeronNakHeader

AeronNewPublicationLogger

AeronNewSubscriptionLogger

AeronNotificationLogger

AeronOptionHeader

AeronPerThreadError

AeronPublication

AeronPublicationConstants

Configuration for a publication that does not change during it’s lifetime.

AeronPublicationErrorFrameHandlerLogger

AeronPublicationErrorValues

AeronReservedValueSupplierLogger

AeronResolutionHeader

AeronResolutionHeaderIpv4

AeronResolutionHeaderIpv6

AeronResponseSetupHeader

AeronRttmHeader

AeronSetupHeader

AeronStatusMessageHeader

AeronStatusMessageOptionalHeader

AeronStrToPtrHashMap

AeronStrToPtrHashMapKey

AeronSubscription

AeronSubscriptionConstants

AeronUdpChannelParams

AeronUnavailableCounterLogger

AeronUnavailableCounterPair

AeronUnavailableImageLogger

AeronUri

AeronUriParam

AeronUriParams

AeronUriParseCallbackLogger

AeronUriStringBuilder

AtomicWideCounterBindgenTy1

ChannelUri

Represents the Aeron URI parser and handler.

Handler

Handler

Handlers

Utility method for setting empty handlers

ManagedCResource

A custom struct for managing C resources with automatic cleanup.

NoOpAeronIdleStrategyFunc

PthreadCondS

PthreadInternalList

PthreadMutexS

RecordingPos

Enums

AeronErrorType

AeronSystemCounterType

ControlMode

Enum for control modes.

Media

Enum for media types.

Constants

AERON_DIR_PROP_NAME

AERON_IPC_MEDIA

AERON_IPC_STREAM

AERON_UDP_MEDIA

DRIVER_TIMEOUT_MS_DEFAULT

SOURCE_LOCATION_LOCAL

SOURCE_LOCATION_REMOTE

SPY_PREFIX

TAG_PREFIX

Traits

AeronAgentStartFuncCallback

(note you must copy any arguments that you use afterwards even those with static lifetimes)

AeronArchiveCredentialsChallengeSupplierFuncCallback

Callback to return encoded credentials given a specific encoded challenge.

AeronArchiveCredentialsEncodedCredentialsSupplierFuncCallback

Callback to return encoded credentials.

AeronArchiveCredentialsFreeFuncCallback

Callback to return encoded credentials so they may be reused or freed.

AeronArchiveDelegatingInvokerFuncCallback

Callback to allow execution of a delegating invoker to be run.

AeronArchiveRecordingDescriptorConsumerFuncCallback

Callback to return recording descriptors.

AeronArchiveRecordingSignalConsumerFuncCallback

Callback to return recording signals.

AeronArchiveRecordingSubscriptionDescriptorConsumerFuncCallback

Callback to return recording subscription descriptors.

AeronAvailableCounterCallback

Function called by aeron_client_t to deliver notifications that a counter has been added to the driver.

AeronAvailableImageCallback

Function called by aeron_client_t to deliver notifications that an aeron_image_t was added.

AeronBlockHandlerCallback

Callback for handling a block of messages being read from a log.

AeronCloseClientCallback

Function called by aeron_client_t to deliver notifications that the client is closing.

AeronControlledFragmentHandlerCallback

Callback for handling fragments of data being read from a log.

AeronCountersReaderForeachCounterFuncCallback

Function called by aeron_counters_reader_foreach_counter for each counter in the aeron_counters_reader_t.

AeronErrorHandlerCallback

The error handler to be called when an error occurs.

AeronErrorLogReaderFuncCallback

(note you must copy any arguments that you use afterwards even those with static lifetimes)

AeronFragmentHandlerCallback

Callback for handling fragments of data being read from a log.

AeronIdleStrategyFuncCallback

(note you must copy any arguments that you use afterwards even those with static lifetimes)

AeronLossReporterReadEntryFuncCallback

(note you must copy any arguments that you use afterwards even those with static lifetimes)

AeronNewPublicationCallback

Function called by aeron_client_t to deliver notification that the media driver has added an aeron_publication_t or aeron_exclusive_publication_t successfully.

AeronNewSubscriptionCallback

Function called by aeron_client_t to deliver notification that the media driver has added an aeron_subscription_t successfully.

AeronNotificationCallback

Generalised notification callback.

AeronPublicationErrorFrameHandlerCallback

The error frame handler to be called when the driver notifies the client about an error frame being received. The data passed to this callback will only be valid for the lifetime of the callback. The user should use aeron_publication_error_values_copy if they require the data to live longer than that.

AeronReservedValueSupplierCallback

Function called when filling in the reserved value field of a message.

AeronUnavailableCounterCallback

Function called by aeron_client_t to deliver notifications that a counter has been removed from the driver.

AeronUnavailableImageCallback

Function called by aeron_client_t to deliver notifications that an aeron_image_t has been removed from use and should not be used any longer.

AeronUriParseCallbackCallback

(note you must copy any arguments that you use afterwards even those with static lifetimes)

Functions

find_unused_udp_port

is_udp_port_available

Type Aliases

SourceLocation