Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Coming from sbe-tool

Side-by-side mapping for teams migrating from the official Simple Binary Encoding Rust generator (sbe-tool). Within the [published SBE profile] (../design-notes/feature-matrix.md) and docs/SBE_COMPATIBILITY.md, ergo-sbe aims for official-SBE wire fidelity with sbe-tool; the API shape is intentionally different. Do not read this as unqualified “binary compatible with every SBE feature.”

The #1 trap: wrap offset

sbe-tool Rustergo-sbe
wrap argumentBody offset (often 8 for a frame at 0)Message start (often 0)
Where fields livebody_offset + field_offsetmessage_offset + HEADER_LENGTH + field_offset
// Frame at buf[0..]:
// sbe-tool:  enc.wrap(buf, 8)              // body starts at 8
// ergo-sbe:  Enc::wrap(buf, 0)             // message starts at 0
//            Enc::wrap_and_apply_header(buf, 0)

Passing sbe-tool’s 8 into ergo-sbe for a frame at zero mis-aligns every field. Generated rustdoc on wrap / wrap_and_apply_header / decode repeats this callout.

Header modes (fair comparison table)

Use the same logical work on both sides when comparing or porting:

Modeergo-sbesbe-tool
Body onlywrap(buf, 0) + setters — no apply-headerwrap(buf, 8) + setters — no .header(0)
Header + bodywrap_and_apply_header(buf, 0) + setterswrap(buf, 8) then header(0).parent() then setters
Header onlywrap_and_apply_header alonewrap(buf, 8).header(0) alone
  • ergon wrap = message start; sbe-tool wrap = body offset.
  • sbe-tool encoded_length() is body only. ergon encoded_length_with_header() includes the header. Never invent 8 + encoded_length() to “prove” a header was written if .header(0) was not called on the sbe-tool arm.

Full fairness rules for benchmarks: Benchmarks methodology.

Groups: .parent() hopscotch vs closures

sbe-toolergo-sbe
Open group flyweight, fill entries, .parent() backenc.bids(n, |bids| { bids.add(|e| { … })?; Ok(()) })?
Nested groups fight the borrow checkerNested closures end; chain continues in wire order
#![allow(unused)]
fn main() {
pub fn encode_sample_car(buf: &mut [u8]) -> Result<usize, sbe_rt::EncodeError> {
    let mut extras = OptionalExtras::default();
    extras.cruise_control(true).sports_pack(true);

    // Buffer pre-sized from EncodedLength; try_* still validates extent.
    let len = CarEncoder::try_wrap_and_apply_header(buf, 0)
        .unwrap()
        .fixed(&CarFixedFields {
            serial_number: 1234,
            model_year: 2013,
            available: true.into(),
            code: Model::A,
            some_numbers: [10, 20, 30, 40],
            vehicle_code: [b'A', b'B', b'C', b'D', b'E', b'F'],
            extras,
            engine: Engine::new(
                2000,
                4,
                [b'1', b'2', b'3'],
                0i8,
                false.into(),
                Booster::new(BoostType::TURBO, 210),
            ),
        })
        .fuel_figures(2, |g| {
            g.add(|mut e| {
                e.speed(30).mpg(35.9);
                e.usage_description_as_str("Urban")
            })?;
            g.add(|mut e| {
                e.speed(60).mpg(25.0);
                e.usage_description_as_str("Highway")
            })?;
            Ok(())
        })?
        .performance_figures(1, |g| {
            g.add(|mut e| {
                e.octane_rating(95);
                e.acceleration(2, |a| {
                    a.add(|x| {
                        x.mph(30).seconds(4.0);
                        Ok(())
                    })?;
                    a.add(|x| {
                        x.mph(60).seconds(7.5);
                        Ok(())
                    })
                })
            })?;
            Ok(())
        })?
        .manufacturer_as_str("Honda")?
        .model_as_str("Civic VTi")?
        .activation_code_as_str("abcdef")?
        .encoded_length_with_header();

    Ok(len)
}
}

Compile-time order: you cannot call asks before bids — the stage type has no asks method. See Wire order via named stages.

Length APIs

Conceptsbe-toolergo-sbe
Body length after encodeencoded_length()body region only via stage encoded_length where exposed
Header + bodycompute yourself (8 + …)encoded_length_with_header() / as_bytes_with_header()
Pre-size bufferoften oversize scratchExact: const Encoder::compute_length_with_header() (stack [0u8; N]), or Encoder::compute_length() / staged *EncodedLength when the length depends on groups or var-data

Do not default to vec![0u8; 4096] or Vec::with_capacity(MAX) then truncate. See Buffer sizing and Exact sizing.

Optional fields: nullify_optional_fields() → apply_nulls()

Neither generator nullifies on wrap, so in sbe-tool an optional field you never set keeps whatever was already in the buffer unless you remember to call nullify_optional_fields(). The ergon equivalent is apply_nulls(), but the port is usually to delete the call rather than rename it:

sbe-toolergo-sbe
enc.nullify_optional_fields() on the message, then set fields.fixed(&MsgFixedFields { price: None, … }) — None writes the schema null image
nullify_optional_fields() on a group entry encodernothing to call — every add() nulls the entry’s optional scalars
Forgetting the call leaks stale bytesfixed() is the only route to tails and to complete byte views, so the ordinary path cannot leak

Optional fields are Option<T> in the generated *FixedFields struct, and fixed() writes the schema null wire image for every None — including fixed arrays and nested optional composite members.

apply_nulls() is still generated (message encoders with at least one optional field, on the unfixed stage after wrap*) for the case where you write individual optionals through raw_fixed() and have no FixedFields value describing which ones are unset.

One asymmetry to carry across: group-entry auto-nulling covers optional fields that have a declared nullValue and a width of 1-8 bytes. An optional array or composite inside a group entry is not nulled, where the message-level path does handle those shapes. If your schema has one, set it explicitly.

See Optional fields and apply_nulls and Why NullVal instead of Option.

Decode entry

ergo-sbe decoder wrap and try_wrap take the message start, including the header, while sbe-tool’s wrap takes the body offset. Use try_decode when the header metadata is not already available:

Needergo-sbe
Untrusted / networktry_decode / try_wrap / try_from → Result (all failures)
Known-good bufferbare wrap → panic if short; bare decode → hybrid (panic if short, Err on wrong template/schema)
Proven-tight hot pathunsafe wrap_unchecked; decode_unchecked = unchecked extent + checked identity

See Trust Boundary.

Decoding: lanes, not one cursor

sbe-tool gives you a single &mut decoder carrying a limit cursor. Every group and var-data accessor reads at limit and advances it, so the order you call methods in is the wire walk. ergon splits that:

  • Sequential: into_* / skip_* (or decoder.ordered() for one callback per tail)
  • Random-access: try_decode — any order, Sync

A group whose entries carry their own tails takes a visit closure; a fixed-stride group hands back an iterator.

Start here when porting:

Your sbe-tool codePort to
Straight-line walk: group, group, var-data, donestaged into_* / skip_* — one traversal, compile-time tail order
Re-wrapping the message a second time to read a field you passedthe base decoder, or decoder.memoized() if you re-read deep tails
Reading two fixed fields and dropping the restthe base decoder from try_decode — no cursor, Sync, nothing to consume
// sbe-tool: one cursor, order is a convention
let car = CarDecoder::default()
    .wrap(ReadBuf::new(buf), 8, block_len, version);   // body offset, not 0
let mut ff = car.fuel_figures_decoder();               // consumes `car`
while ff.advance()?.is_some() { /* … */ }
let mut car = ff.parent()?;                            // hand it back
let coords = car.manufacturer_decoder();               // (offset, len)
let manufacturer = car.manufacturer_slice(coords);

// ergo-sbe: consuming stages carry the next tail offset.
// fuelFigures entries carry var-data, so the step is a visit closure and the
// completion it returns is where the next entry starts — one pass, no rescan.
let (manufacturer, car) = CarDecoder::try_decode(buf, 0)?
    .into_fuel_figures(|e| e.into_usage_description().map(|(_usage, done)| done))?
    .skip_performance_figures()?
    .into_manufacturer()?;

Note what disappears in the port: no .parent() hop, no (offset, length) coordinate pair threaded to a second _slice() call, and no &mut on a decoder you only wanted to read.

Fixed-block messages — no groups, no var-data — get no memoized() or ordered(), and AnyMessage offers only into_<name>() for them. Every field is already random-access off the block.

Full per-lane pros, cons, and cost table: Decoder lanes.

Version handling

Decoders are version-aware: tail offsets use the wire acting block length, not only the compiled block length. Optional / sinceVersion fields follow schema presence rules. Prefer explicit try_* entry points when reading mixed-version streams.

What has no direct ergon equivalent (and why)

sbe-tool habitergo-sbe
.parent() ownership hopEncode: nested closures. Decode: consuming stages — a visit closure for dynamic groups, an iterator for fixed-stride ones
One &mut decoder with a limit cursorLanes: random-access &Decoder, staged into_*, optional ordered() / .memoized() (decoder lanes)
_decoder() returning (offset, len) for a second _slice() callVar-data accessors return &'a [u8] / &'a str directly
Generic Encoder<State> spellingNamed stage structs + H: HeaderState only for header mode (type-state note)
encoded_length() as full-frame sizeUse *_with_header when you need the frame
nullify_optional_fields() on group and composite encodersGroup entries null their optional scalars on add(); only the message encoder gets apply_nulls() (above)
Always-on meta / Display noiseOpt-out size knobs: with_display_debug(false), with_meta_attributes(false), with_dispatch(false)

Trust boundary

try_* returns Result on short buffers (and identity mismatches). Bare wrap panics if short. Bare decode is a hybrid: panics if short, but still returns Err on wrong template/schema. decode_unchecked is unchecked extent + checked identity. After a safe constructor succeeds, fixed-field accessors are branch-free. Full detail: Trust Boundary.

Further reading