Skip to content

Idea: New v3 opt-in serialization format to save space #366

Description

@TysonAndre
  • Use version \x00\x00\x00\x03 when this is stable
  • Avoid wasting space emitting key representations when array_is_list, i.e. when array keys are consecutive integers starting from 0 (have a fallback to start emitting unexpected keys with different values, if needed)
  • Allow more than one reference group to the same php value to be distinctly unserialized (e.g. $x = new stdClass; $y = $x; igbinary_serialize([&$x, &$x, &$y, &$y]);)
    (TODO: I forget the exact issue. Was that fixed already for the general case, or was the issue mixes of references and non-references)
  • Shorter representations for 1-character interned strings
  • Shorter representations for private properties of serialized class \x00$classname\x00$propname (and protected properties \x00*\x00$propname) (keys for parent classes and traits would be serialized normally)
  • Don't bother creating string ids for empty strings (EDIT: that's already part of the v2 design?) and 1-character strings that would already be interned in modern php versions (support reusing per-request in RINIT in igbinary itself if practical for php versions without ZSTR_CHAR guaranteed to exist)

Opt-in can be done on per-serializer settings, to allow any migrations (e.g. redis, memcached) to be done separately.

The unserializer should recognize v3 from the header

Related to phpredis/phpredis#2194

Updates (October 19):

Implementation is broken/incomplete and the v3 serialization format will be changed, found on the fork branch igbinary-v3-format

  • Shorter representations for very small non-negative integers (e.g. 0..7 can be X+(0..7)) is an idea I've seen in several other binary protocol formats, useful for ['key' => 1, 'x' => 0] where integers are used instead of booleans, or just from most integers being small (e.g. counts) when averaged out over common use cases
  • Still haven't implemented for serializing lists
  • Still haven't finished adding a way to represent creating a new reference group to the same php object for the general case
  • Haven't benchmarked this yet

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions