Decoder Lanes

A decoder with groups or var-data has two jobs: read some fields in any order, or walk the whole message once in schema order. A group decoder from a random-access getter implements Iterator for partial traversal; it is not a third message-decoding lane.

Encoding is one state machine: each stage is the cursor, so the next legal field is the only method on that type. See Encode and Decode and Method Chaining. Decoding is the choice, because a decoder can be asked to read in an order the encoder never had.

Fixed fields stay random-access in all of them. Ordered also offers a fixed callback (fixed-fields-only view, not the full decoder). A message with no groups and no var-data has one lane: the base decoder. memoized() and ordered() are not generated; AnyMessage offers only into_<name>().

Worked nested example: L3 order book. AnyMessage can land in a lane directly (into_car(), into_car_memoized()).

sbe-tool has one &mut flyweight with a limit cursor. Wrong order reads the next group’s bytes as a length prefix; re-read means re-wrap; no useful &-sharing. Ergon splits those jobs. Full mapping: Coming from sbe-tool.

In every ergon lane, try_* construction makes a short buffer a DecodeError (not a panic), and a group’s numInGroup is checked against the bytes that are actually there before an entry reaches you.

Random access

Any-order getters. The decoder holds the buffer, offset, and acting header values — nothing else. Each dynamic-tail getter re-walks from the fixed block.

#![allow(unused)]
fn main() {
pub fn demo_car_random_access(wire: &[u8]) -> Result<(), Box<dyn std::error::Error>> {
    let car = CarDecoder::try_decode(wire, 0)?;
    // Manufacturer is the first var-data field, after both groups — still legal
    // here because random access rescan preceding tails.
    assert_eq!(car.manufacturer_as_str()?, "Honda");
    assert_eq!(car.serial_number(), 1234);
    let mut speeds = Vec::new();
    for entry in car.fuel_figures()? {
        speeds.push(entry?.speed());
    }
    assert_eq!(speeds, vec![30, 60]);
    assert_eq!(car.model_as_str()?, "Civic VTi");
    Ok(())
}
}

Memoized (decoder.memoized())

Same getter names. First access to a tail walks forward from the cache frontier; later accesses reuse it. Construction is O(1) and allocates nothing. Decoded values match the base lane, including schema max-length rejection: warming the cache with encoded_length() does not accept a payload the base getter would refuse.

let decoder = CarDecoder::try_from(bytes)?;   // small, Sync, recalculates tails
let decoder = decoder.memoized();             // lazy cache, no allocation

read_header(&decoder)?;
read_groups(&decoder)?;
read_final_tail(&decoder)?;                   // reuses discovered boundaries

Build it once and pass &CarMemoizedDecoder. Calling .memoized() inside every function creates a separate empty cache. Cell interior mutability: Send but not Sync. into_inner() returns the base decoder.

The base lane’s tail_offset_k re-walks every preceding tail on every getter — quadratic even in schema order. Memoized makes that sweep linear. Staged already carries the cursor, so it needs no cache.

Use it when several helpers read the same decoder, or when you jump to a late tail and come back. Skip it for one tail and stop, a full wire-order walk (use staged), or if you need Sync.

The cache covers message-level tails only. Nested groups inside an entry use an ordinary entry decoder. just bench-diagnostics → versioned_l3_bench.

Staged (into_* / skip_*)

Sequential decode, the encoder dual. Each into_* consumes the stage. Wrong order is a missing method.

Iterator is implemented for &mut Iter only — for entry in iter does not compile, so a loop cannot drop the rest of the message. Also ExactSizeIterator and FusedIterator.

#![allow(unused)]
fn main() {
pub fn demo_car_decode_stages(wire: &[u8]) -> Result<(), Box<dyn std::error::Error>> {
    let car = CarDecoder::try_decode(wire, 0)?;
    assert_eq!(car.serial_number(), 1234);
    assert_eq!(car.model_year(), 2013);
    // Domain conversion: BooleanType → bool when configured.
    let available: bool = car.available()?;
    assert!(available);
    assert_eq!(car.code(), Model::A);
    assert_eq!(car.discounted_model(), Model::C); // constant field
    assert_eq!(car.engine().capacity(), 2000);

    // Schema order: groups then var-data, one chain. Both of Car's groups have
    // entries that carry their own tails, so both take a visit closure: the
    // completion the closure returns is where the next entry starts, so the
    // group is walked exactly once.
    let mut speeds = Vec::new();
    let mut octanes = Vec::new();
    let (mfr, decoder) = car
        .into_fuel_figures(|entry| -> Result<_, sbe_rt::DecodeError> {
            speeds.push(entry.speed());
            // usageDescription is this entry's only tail, so reading it
            // completes the entry.
            entry.into_usage_description().map(|(_usage, done)| done)
        })?
        .into_performance_figures(|entry| -> Result<_, sbe_rt::DecodeError> {
            octanes.push(entry.octane_rating());
            // acceleration has fixed-stride entries, so it is a real iterator.
            let mut accel = entry.into_acceleration()?;
            assert_eq!((&mut accel).len(), 2);
            accel.finish()
        })?
        .into_manufacturer_as_str()?;
    assert_eq!(speeds, vec![30, 60]);
    assert_eq!(octanes, vec![95]);
    let (model, decoder) = decoder.into_model_as_str()?;
    let (code, _decoder) = decoder.into_activation_code_as_str()?;
    // All three &str coexist — each borrows 'a from the original wire buffer.
    assert_eq!((mfr, model, code), ("Honda", "Civic VTi", "abcdef"));
    Ok(())
}
}

Each into_*_as_str() returns (&'a str, NextStage<'a>) — the &str borrows the original buffer. See Method chaining.

Start with try_decode / try_from. Do not verify first then walk — that is two structural passes. Inside a dynamic-entry callback, consume nested tails with into_* / skip_* / ordered() and return that completion. Random-access getters or a root manufacturer_len() before visiting preceding groups rescan.

Consuming stages and group iterators are #[must_use]. Dropping a stage without into_* / skip_* / finish() abandons remaining tails.

A fixed-stride iterator need not be drained: finish(), or the next tail’s into_* / skip_* on the iterator, skips unread entries as count × block_length. Dynamic-stride skip_* still walks each unread entry.

Asking before walking

Absent versioned tails are 0. If a primary getter already uses that name, the convenience accessor is omitted on that owner. Non-consuming is not constant-time on random-access owners — lookahead may scan preceding tails. Group byte length is not available from the count when entries have tails.

#![allow(unused)]
fn main() {
pub fn demo_car_visit_entries(wire: &[u8]) -> Result<(), Box<dyn std::error::Error>> {
    let car = CarDecoder::try_decode(wire, 0)?;

    // Non-advancing: `car` is untouched and still owns every tail.
    assert_eq!(car.fuel_figures_count()?, 2);
    assert_eq!(car.manufacturer_len()?, 5);

    // A fixed-stride group is a real iterator, and it does not have to be
    // drained. Read one acceleration entry, then let the walk move on — the
    // unread entries are skipped by the next step, not lost.
    let (mfr, _rest) = car
        .skip_fuel_figures()?
        .into_performance_figures(|entry| -> Result<_, sbe_rt::DecodeError> {
            let mut accel = entry.into_acceleration()?;
            assert_eq!((&mut accel).len(), 2);
            let first = (&mut accel).next().expect("acceleration is not empty");
            assert_eq!(first.mph(), 30);
            assert_eq!(accel.remaining_entries(), 1);
            // `finish()` skips the entry we did not read and completes.
            accel.finish()
        })?
        .into_manufacturer_as_str()?;
    assert_eq!(mfr, "Honda");
    Ok(())
}
}

Prefer staged when you skip tails, hold stages across functions, or want for entry in &mut iter. A closure error consumes the stage; there is no continuation to retry.

Ordered (decoder.ordered())

One callback per tail, same spelling at message and nested-entry level, plus EntryInfo per entry. Generated on message decoders with tails and on group entries that carry tails. An entry whose existing accessor is named ordered keeps that accessor and does not get this lane.

ordered() is sbe_rt::Ordered<S> over the staged stage — no second cursor. A nested ordered() walk is the completion the parent callback owes. .done() only unwraps the staged complete. The message chain ends on encoded_length_with_header() / as_bytes_with_header(), like encode.

#![allow(unused)]
fn main() {
pub fn demo_car_ordered_lane(wire: &[u8]) -> Result<(), Box<dyn std::error::Error>> {
    let mut speeds = Vec::new();
    let mut octanes = Vec::new();
    let mut text = Vec::new();

    let len = CarDecoder::try_decode(wire, 0)?
        .ordered()
        .fixed(|car| {
            assert_eq!(car.serial_number(), 1234);
            Ok(())
        })?
        .fuel_figures(|entry, info| {
            // `info` carries the position the wire declares.
            assert_eq!(info.count, 2);
            assert_eq!(info.is_last(), info.index == 1);
            speeds.push(entry.speed());
            Ok(entry.ordered().usage_description(|_| Ok(()))?)
        })?
        .performance_figures(|entry, _info| {
            octanes.push(entry.octane_rating());
            Ok(entry.ordered().acceleration(|a, _| {
                let _ = (a.mph(), a.seconds());
                Ok(())
            })?)
        })?
        .manufacturer_as_str(|s| {
            text.push(s.to_owned());
            Ok(())
        })?
        .model_as_str(|s| {
            text.push(s.to_owned());
            Ok(())
        })?
        .activation_code_as_str(|s| {
            text.push(s.to_owned());
            Ok(())
        })?
        .encoded_length_with_header();

    assert_eq!(speeds, vec![30, 60]);
    assert_eq!(octanes, vec![95]);
    assert_eq!(text, vec!["Honda", "Civic VTi", "abcdef"]);
    assert_eq!(len, wire.len());
    Ok(())
}
}

EntryInfo is index, wire-declared count, acting block_length, plus is_first() / is_last() / remaining(). No entry scan. Group byte length is absent: entries with tails only settle it by traversing.

Message-level fixed is optional. Skip it, or call it and lose the view. The view has no group/var-data accessors. A first tail named fixed owns the fixed() name, so the fixed-block callback yields: read fixed fields before ordered(). Every callback returns Result<_, DecodeError>; a caller error type converts after the walk. Entry-level wrappers have no fixed: the parent already holds the entry.

Empty and version-absent groups invoke the callback zero times. <group>_count() on the ordered stage reports the wire-declared count for an empty group and 0 for a group the wire version predates, which has no count.

Bytes and text callbacks borrow 'a from the original buffer. A callback error consumes the stage. Prefer ordered when you always decode whole messages and want one shape. Prefer staged to skip, hold stages, or iterate.

Full-frame bytes mid-walk

See Generated code.