Coming from sbe-tool
Side-by-side mapping for teams migrating from the official Simple Binary
Encoding Rust generator (sbe-tool). Within the [published SBE profile]
(../design-notes/feature-matrix.md) and
docs/SBE_COMPATIBILITY.md,
ergo-sbe aims for official-SBE wire fidelity with sbe-tool; the API shape
is intentionally different. Do not read this as unqualified “binary
compatible with every SBE feature.”
The #1 trap: wrap offset
| sbe-tool Rust | ergo-sbe | |
|---|---|---|
wrap argument | Body offset (often 8 for a frame at 0) | Message start (often 0) |
| Where fields live | body_offset + field_offset | message_offset + HEADER_LENGTH + field_offset |
// Frame at buf[0..]:
// sbe-tool: enc.wrap(buf, 8) // body starts at 8
// ergo-sbe: Enc::wrap(buf, 0) // message starts at 0
// Enc::wrap_and_apply_header(buf, 0)
Passing sbe-tool’s 8 into ergo-sbe for a frame at zero mis-aligns every
field. Generated rustdoc on wrap / wrap_and_apply_header / decode
repeats this callout.
Header modes (fair comparison table)
Use the same logical work on both sides when comparing or porting:
| Mode | ergo-sbe | sbe-tool |
|---|---|---|
| Body only | wrap(buf, 0) + setters — no apply-header | wrap(buf, 8) + setters — no .header(0) |
| Header + body | wrap_and_apply_header(buf, 0) + setters | wrap(buf, 8) then header(0).parent() then setters |
| Header only | wrap_and_apply_header alone | wrap(buf, 8).header(0) alone |
- ergon
wrap= message start; sbe-toolwrap= body offset. - sbe-tool
encoded_length()is body only. ergonencoded_length_with_header()includes the header. Never invent8 + encoded_length()to “prove” a header was written if.header(0)was not called on the sbe-tool arm.
Full fairness rules for benchmarks: Benchmarks methodology.
Groups: .parent() hopscotch vs closures
| sbe-tool | ergo-sbe |
|---|---|
Open group flyweight, fill entries, .parent() back | enc.bids(n, |bids| { bids.add(|e| { … })?; Ok(()) })? |
| Nested groups fight the borrow checker | Nested closures end; chain continues in wire order |
#![allow(unused)]
fn main() {
pub fn encode_sample_car(buf: &mut [u8]) -> Result<usize, sbe_rt::EncodeError> {
let mut extras = OptionalExtras::default();
extras.cruise_control(true).sports_pack(true);
// Buffer pre-sized from EncodedLength; try_* still validates extent.
let len = CarEncoder::try_wrap_and_apply_header(buf, 0)
.unwrap()
.fixed(&CarFixedFields {
serial_number: 1234,
model_year: 2013,
available: true.into(),
code: Model::A,
some_numbers: [10, 20, 30, 40],
vehicle_code: [b'A', b'B', b'C', b'D', b'E', b'F'],
extras,
engine: Engine::new(
2000,
4,
[b'1', b'2', b'3'],
0i8,
false.into(),
Booster::new(BoostType::TURBO, 210),
),
})
.fuel_figures(2, |g| {
g.add(|mut e| {
e.speed(30).mpg(35.9);
e.usage_description_as_str("Urban")
})?;
g.add(|mut e| {
e.speed(60).mpg(25.0);
e.usage_description_as_str("Highway")
})?;
Ok(())
})?
.performance_figures(1, |g| {
g.add(|mut e| {
e.octane_rating(95);
e.acceleration(2, |a| {
a.add(|x| {
x.mph(30).seconds(4.0);
Ok(())
})?;
a.add(|x| {
x.mph(60).seconds(7.5);
Ok(())
})
})
})?;
Ok(())
})?
.manufacturer_as_str("Honda")?
.model_as_str("Civic VTi")?
.activation_code_as_str("abcdef")?
.encoded_length_with_header();
Ok(len)
}
}
Compile-time order: you cannot call asks before bids — the stage type
has no asks method. See
Wire order via named stages.
Length APIs
| Concept | sbe-tool | ergo-sbe |
|---|---|---|
| Body length after encode | encoded_length() | body region only via stage encoded_length where exposed |
| Header + body | compute yourself (8 + …) | encoded_length_with_header() / as_bytes_with_header() |
| Pre-size buffer | often oversize scratch | Exact: const Encoder::compute_length_with_header() (stack [0u8; N]), or Encoder::compute_length() / staged *EncodedLength when the length depends on groups or var-data |
Do not default to vec![0u8; 4096] or Vec::with_capacity(MAX) then
truncate. See Buffer sizing and
Exact sizing.
Optional fields: nullify_optional_fields() → apply_nulls()
Neither generator nullifies on wrap, so in sbe-tool an optional field you
never set keeps whatever was already in the buffer unless you remember to
call nullify_optional_fields(). The ergon equivalent is apply_nulls(),
but the port is usually to delete the call rather than rename it:
| sbe-tool | ergo-sbe |
|---|---|
enc.nullify_optional_fields() on the message, then set fields | .fixed(&MsgFixedFields { price: None, … }) — None writes the schema null image |
nullify_optional_fields() on a group entry encoder | nothing to call — every add() nulls the entry’s optional scalars |
| Forgetting the call leaks stale bytes | fixed() is the only route to tails and to complete byte views, so the ordinary path cannot leak |
Optional fields are Option<T> in the generated *FixedFields struct, and
fixed() writes the schema null wire image for every None — including
fixed arrays and nested optional composite members.
apply_nulls() is still generated (message encoders with at least one
optional field, on the unfixed stage after wrap*) for the case where you
write individual optionals through raw_fixed() and have no FixedFields
value describing which ones are unset.
One asymmetry to carry across: group-entry auto-nulling covers optional
fields that have a declared nullValue and a width of 1-8 bytes. An optional
array or composite inside a group entry is not nulled, where the
message-level path does handle those shapes. If your schema has one, set it
explicitly.
See Optional fields and apply_nulls
and Why NullVal instead of Option.
Decode entry
ergo-sbe decoder wrap and try_wrap take the message start, including
the header, while sbe-tool’s wrap takes the body offset. Use try_decode
when the header metadata is not already available:
| Need | ergo-sbe |
|---|---|
| Untrusted / network | try_decode / try_wrap / try_from → Result (all failures) |
| Known-good buffer | bare wrap → panic if short; bare decode → hybrid (panic if short, Err on wrong template/schema) |
| Proven-tight hot path | unsafe wrap_unchecked; decode_unchecked = unchecked extent + checked identity |
See Trust Boundary.
Decoding: lanes, not one cursor
sbe-tool gives you a single &mut decoder carrying a limit cursor. Every
group and var-data accessor reads at limit and advances it, so the order you
call methods in is the wire walk. ergon splits that:
- Sequential:
into_*/skip_*(ordecoder.ordered()for one callback per tail) - Random-access:
try_decode— any order,Sync
A group whose entries carry their own tails takes a visit closure; a fixed-stride group hands back an iterator.
Start here when porting:
| Your sbe-tool code | Port to |
|---|---|
| Straight-line walk: group, group, var-data, done | staged into_* / skip_* — one traversal, compile-time tail order |
| Re-wrapping the message a second time to read a field you passed | the base decoder, or decoder.memoized() if you re-read deep tails |
| Reading two fixed fields and dropping the rest | the base decoder from try_decode — no cursor, Sync, nothing to consume |
// sbe-tool: one cursor, order is a convention
let car = CarDecoder::default()
.wrap(ReadBuf::new(buf), 8, block_len, version); // body offset, not 0
let mut ff = car.fuel_figures_decoder(); // consumes `car`
while ff.advance()?.is_some() { /* … */ }
let mut car = ff.parent()?; // hand it back
let coords = car.manufacturer_decoder(); // (offset, len)
let manufacturer = car.manufacturer_slice(coords);
// ergo-sbe: consuming stages carry the next tail offset.
// fuelFigures entries carry var-data, so the step is a visit closure and the
// completion it returns is where the next entry starts — one pass, no rescan.
let (manufacturer, car) = CarDecoder::try_decode(buf, 0)?
.into_fuel_figures(|e| e.into_usage_description().map(|(_usage, done)| done))?
.skip_performance_figures()?
.into_manufacturer()?;
Note what disappears in the port: no .parent() hop, no (offset, length)
coordinate pair threaded to a second _slice() call, and no &mut on a
decoder you only wanted to read.
Fixed-block messages — no groups, no var-data — get no memoized()
or ordered(), and AnyMessage offers only into_<name>() for them. Every
field is already random-access off the block.
Full per-lane pros, cons, and cost table: Decoder lanes.
Version handling
Decoders are version-aware: tail offsets use the wire acting block
length, not only the compiled block length. Optional / sinceVersion fields
follow schema presence rules. Prefer explicit try_* entry points when
reading mixed-version streams.
What has no direct ergon equivalent (and why)
| sbe-tool habit | ergo-sbe |
|---|---|
.parent() ownership hop | Encode: nested closures. Decode: consuming stages — a visit closure for dynamic groups, an iterator for fixed-stride ones |
One &mut decoder with a limit cursor | Lanes: random-access &Decoder, staged into_*, optional ordered() / .memoized() (decoder lanes) |
_decoder() returning (offset, len) for a second _slice() call | Var-data accessors return &'a [u8] / &'a str directly |
Generic Encoder<State> spelling | Named stage structs + H: HeaderState only for header mode (type-state note) |
encoded_length() as full-frame size | Use *_with_header when you need the frame |
nullify_optional_fields() on group and composite encoders | Group entries null their optional scalars on add(); only the message encoder gets apply_nulls() (above) |
| Always-on meta / Display noise | Opt-out size knobs: with_display_debug(false), with_meta_attributes(false), with_dispatch(false) |
Trust boundary
try_* returns Result on short buffers (and identity mismatches). Bare
wrap panics if short. Bare decode is a hybrid: panics if short, but
still returns Err on wrong template/schema. decode_unchecked is unchecked
extent + checked identity. After a safe constructor succeeds, fixed-field
accessors are branch-free. Full detail:
Trust Boundary.