This release adds important new features, optimizations, bug fixes, and 8 new contributors. Fuzz tests passed 5+ billion execs for CBOR encoding and decoding functions.
Notable changes are split into 3 categories.
⭐ Notable Changes to Review Before Upgrading
These new features and bug fixes were cherry-picked to highlight for review (for projects using older version).
- PR 370: Add
SimpleValuetype to more fully support CBOR Simple Values, including values not assigned by IANA and... - PR 376: Add
ByteStringtype to support CBOR maps with byte string keys because Go doesn't allow []byte as map keys and... - PR 379: Make
Decoder.Decode()returnio.ErrUnexpectedEOFinstead ofio.EOFon EOF if current CBOR data item is incomplete. - PR 380: Make
Unmarshal()andValid()returncbor.ExtraneousDataError(instead of ignoring extraneous data if any remain). - PR 387: Retry in Decoder if io.Reader's Read() returns 0 bytes read with nil error. Add tests to get 100% code coverage for stream.go.
⭐ More Notable Changes
- PR 342: Add
DecOptions.UTF8to decode invalid UTF-8. Default is unchanged (reject invalid UTF-8 and return error). - PR 355 Allow
MaxNestedLevelssetting to be configured up to 65535. - PR 352, 377: Add
EncOptions.NilContainersModeto encode nil Go maps and slices as either CBOR nil (default) or empty container. - PR 381: Add
Decoder.Skip()to skip CBOR data item in CBOR Sequences (RFC 8742). - PR 386 Add functions for Extended Diagnostic Notation (RFC 8610 Appendix G).
- PR 398 Add
UnmarshalFirst()to decode CBOR data item and also return remaining bytes. - PR 400 Deprecate
Validand addWellformedto replace it. - PR 402 Add
UnmarshalFirst()toDecModeinterface. - PR 412 Add
Decoder.Bufferedto return remaining data in buffer.
🚀 Notable Changes Affecting Speed or Memory Use
- PR 335: Reuse underlying array if
RawMessagehas sufficient capacity. - PR 382: Return buffer to pool in
Encode(). It adds a bit of overhead toEncode()butNewEncoder().Encode()is faster and uses less memory.
Benchmark comparison for PR 382 shows significant improvement in speed and memory use.
:rocket: (click to expand) :rocket:
Benchmarks provided by @x448 for NewEncoder().Encode() optimized in PR 382.
$ benchstat bench-v2.4.0.log bench-f9e6291.log
goos: linux
goarch: amd64
pkg: github.com/fxamacker/cbor/v2
cpu: 12th Gen Intel(R) Core(TM) i7-12700H
│ bench-v2.4.0.log │ bench-f9e6291.log │
│ sec/op │ sec/op vs base │
NewEncoderEncode/Go_bool_to_CBOR_bool-20 236.70n ± 2% 58.04n ± 1% -75.48% (p=0.000 n=10)
NewEncoderEncode/Go_uint64_to_CBOR_positive_int-20 238.00n ± 2% 63.93n ± 1% -73.14% (p=0.000 n=10)
NewEncoderEncode/Go_int64_to_CBOR_negative_int-20 238.65n ± 2% 64.88n ± 1% -72.81% (p=0.000 n=10)
NewEncoderEncode/Go_float64_to_CBOR_float-20 242.00n ± 2% 63.00n ± 1% -73.97% (p=0.000 n=10)
NewEncoderEncode/Go_[]uint8_to_CBOR_bytes-20 245.60n ± 1% 68.55n ± 1% -72.09% (p=0.000 n=10)
NewEncoderEncode/Go_string_to_CBOR_text-20 243.20n ± 3% 68.39n ± 1% -71.88% (p=0.000 n=10)
NewEncoderEncode/Go_[]int_to_CBOR_array-20 563.0n ± 2% 378.3n ± 0% -32.81% (p=0.000 n=10)
NewEncoderEncode/Go_map[string]string_to_CBOR_map-20 2.043µ ± 2% 1.906µ ± 2% -6.75% (p=0.000 n=10)
geomean 349.7n 122.7n -64.92%
│ bench-v2.4.0.log │ bench-f9e6291.log │
│ B/op │ B/op vs base │
NewEncoderEncode/Go_bool_to_CBOR_bool-20 128.0 ± 0% 0.0 ± 0% -100.00% (p=0.000 n=10)
NewEncoderEncode/Go_uint64_to_CBOR_positive_int-20 128.0 ± 0% 0.0 ± 0% -100.00% (p=0.000 n=10)
NewEncoderEncode/Go_int64_to_CBOR_negative_int-20 128.0 ± 0% 0.0 ± 0% -100.00% (p=0.000 n=10)
NewEncoderEncode/Go_float64_to_CBOR_float-20 128.0 ± 0% 0.0 ± 0% -100.00% (p=0.000 n=10)
NewEncoderEncode/Go_[]uint8_to_CBOR_bytes-20 128.0 ± 0% 0.0 ± 0% -100.00% (p=0.000 n=10)
NewEncoderEncode/Go_string_to_CBOR_text-20 128.0 ± 0% 0.0 ± 0% -100.00% (p=0.000 n=10)
NewEncoderEncode/Go_[]int_to_CBOR_array-20 128.0 ± 0% 0.0 ± 0% -100.00% (p=0.000 n=10)
NewEncoderEncode/Go_map[string]string_to_CBOR_map-20 544.0 ± 0% 416.0 ± 0% -23.53% (p=0.000 n=10)
geomean 153.4 ? ¹ ²
¹ summaries must be >0 to compute geomean
² ratios must be >0 to compute geomean
│ bench-v2.4.0.log │ bench-f9e6291.log │
│ allocs/op │ allocs/op vs base │
NewEncoderEncode/Go_bool_to_CBOR_bool-20 2.000 ± 0% 0.000 ± 0% -100.00% (p=0.000 n=10)
NewEncoderEncode/Go_uint64_to_CBOR_positive_int-20 2.000 ± 0% 0.000 ± 0% -100.00% (p=0.000 n=10)
NewEncoderEncode/Go_int64_to_CBOR_negative_int-20 2.000 ± 0% 0.000 ± 0% -100.00% (p=0.000 n=10)
NewEncoderEncode/Go_float64_to_CBOR_float-20 2.000 ± 0% 0.000 ± 0% -100.00% (p=0.000 n=10)
NewEncoderEncode/Go_[]uint8_to_CBOR_bytes-20 2.000 ± 0% 0.000 ± 0% -100.00% (p=0.000 n=10)
NewEncoderEncode/Go_string_to_CBOR_text-20 2.000 ± 0% 0.000 ± 0% -100.00% (p=0.000 n=10)
NewEncoderEncode/Go_[]int_to_CBOR_array-20 2.000 ± 0% 0.000 ± 0% -100.00% (p=0.000 n=10)
NewEncoderEncode/Go_map[string]string_to_CBOR_map-20 28.00 ± 0% 26.00 ± 0% -7.14% (p=0.000 n=10)
geomean 2.782 ? ¹ ²
¹ summaries must be >0 to compute geomean
² ratios must be >0 to compute geomeanWhat's Changed
All merged pull requests are listed here (including the already highlighted ones).
(click to expand)
Changes to Code
Most coding changes here were already mentioned.
- Reuse underlying array if RawMessage has sufficient capacity by @zensh in https://github.com/fxamacker/cbor/pull/335
- Add decoding option to allow invalid UTF-8 by @fxamacker in https://github.com/fxamacker/cbor/pull/342
- Allow MaxNestedLevels to be up to 65535 by @immesys in https://github.com/fxamacker/cbor/pull/355
- Add support for unassigned/reserved CBOR simple values by @fxamacker in https://github.com/fxamacker/cbor/pull/370
- Add ByteString type to support any CBOR byte string by @fxamacker in https://github.com/fxamacker/cbor/pull/376
- add option to enforce nil container marshaling as empty containers by @dedefer in https://github.com/fxamacker/cbor/pull/352
- Refactor NilContainersMode option by @fxamacker in https://github.com/fxamacker/cbor/pull/377
- Make Decoder.Decode() return io.UnexpectedEOF instead of io.EOF if CBOR data item is truncated by @fxamacker in https://github.com/fxamacker/cbor/pull/379
- Fix handling of extra data in Unmarshal() & Valid() by @fxamacker in https://github.com/fxamacker/cbor/pull/380
- Add Decoder.Skip() to skip CBOR data item in CBOR Sequences (RFC 8742) by @fxamacker in https://github.com/fxamacker/cbor/pull/381
- Return buffer to pool when using Encoder by @fxamacker in https://github.com/fxamacker/cbor/pull/382
- Retry in decoder if Read() returns 0 bytes read with nil error by @fxamacker in https://github.com/fxamacker/cbor/pull/387
- Deprecate Valid() and add Wellformed() to replace it by @fxamacker in https://github.com/fxamacker/cbor/pull/400
- Add UnmarshalFirst by @immesys in https://github.com/fxamacker/cbor/pull/398
- Add UnmarshalFirst to DecMode interface by @fxamacker in https://github.com/fxamacker/cbor/pull/402
- Add functions for Extended Diagnostic Notation (RFC 8610 Appendix G) by @zensh in https://github.com/fxamacker/cbor/pull/386
- Add more tests for Diagnose and DiagnoseFirst by @fxamacker in https://github.com/fxamacker/cbor/pull/405
- Fix Diagnose to return io.EOF error on empty data by @fxamacker in https://github.com/fxamacker/cbor/pull/410
- Add
Decoder.Bufferedto return buffered data by @fxamacker in https://github.com/fxamacker/cbor/pull/412 - Fix
Diagnose()to returnio.EOFerror on empty data by @fxamacker in https://github.com/fxamacker/cbor/pull/410
Changes to CI, Comments, and Docs
- Remove trailing whitespaces in .golangci.yml by @CodingVoid in https://github.com/fxamacker/cbor/pull/333
- Update ci.yml by @fxamacker in https://github.com/fxamacker/cbor/pull/334
- Update ci.yml by @x448 in https://github.com/fxamacker/cbor/pull/339
- Update ci.yml by @x448 in https://github.com/fxamacker/cbor/pull/340
- Remove default permissions from GitHub Actions workflows by @x448 in https://github.com/fxamacker/cbor/pull/341
- Fix ci for stream mode branch by @fxamacker in https://github.com/fxamacker/cbor/pull/344
- Create dependabot.yml by @x448 in https://github.com/fxamacker/cbor/pull/346
- Bump github/codeql-action from 1 to 2 by @dependabot in https://github.com/fxamacker/cbor/pull/347
- Update pull_request_template.md by @x448 in https://github.com/fxamacker/cbor/pull/350
- Fix typos by @deeglaze in https://github.com/fxamacker/cbor/pull/353
- Mention security assessment by NCC Group by @fxamacker in https://github.com/fxamacker/cbor/pull/358
- godoc.org link in Readme should point to v2 by @jdharms in https://github.com/fxamacker/cbor/pull/361
- Add Go 1.19 to ci.yml test matrix by @fxamacker in https://github.com/fxamacker/cbor/pull/363
- bump safer-golangci-lint.yml to 1.46.2 by @x448 in https://github.com/fxamacker/cbor/pull/365
- Bump Go to 1.19 in ci-go-cover.yml by @x448 in https://github.com/fxamacker/cbor/pull/367
- Bump safer-golangci-lint.yml to 1.49.0.1 by @x448 in https://github.com/fxamacker/cbor/pull/368
- Mention 1276 repos depend on fxamacker/cbor/v2 by @x448 in https://github.com/fxamacker/cbor/pull/371
- docs: fix EncMode example in the package comment by @creachadair in https://github.com/fxamacker/cbor/pull/375
- Update fxamacker_cbor_banner.png for v2.5 by @x448 in https://github.com/fxamacker/cbor/pull/383
- bump safer-golangci-lint.yml to 1.51.1 by @x448 in https://github.com/fxamacker/cbor/pull/389
- Create govulncheck.yml by @fxamacker in https://github.com/fxamacker/cbor/pull/393
- Update README for cbor v2.5.0-beta2 by @fxamacker in https://github.com/fxamacker/cbor/pull/394
- Bump actions/setup-go from 3 to 4 by @dependabot in https://github.com/fxamacker/cbor/pull/395
- Update ci-go-cover.yml by @x448 in https://github.com/fxamacker/cbor/pull/404
- Update README for CBOR Sequences RFC 8742 by @fxamacker in https://github.com/fxamacker/cbor/pull/406
- Update README.md for v2.5.0-beta3 by @fxamacker in https://github.com/fxamacker/cbor/pull/407
- Bump safer-golangci-lint to 1.52.2 by @x448 in https://github.com/fxamacker/cbor/pull/408
- Update README.md for cbor v2.5.0-beta4 by @fxamacker in https://github.com/fxamacker/cbor/pull/411
- Update README.md for v2.5.0-beta5 by @fxamacker in https://github.com/fxamacker/cbor/pull/413
- Bump CI govulncheck to v1.0.0 by @fxamacker in https://github.com/fxamacker/cbor/pull/414
- Remove ignored UTF8 setting in diagMode() by @fxamacker in https://github.com/fxamacker/cbor/pull/417
- Update README for CBOR codec v2.5.0-rc1 by @fxamacker in https://github.com/fxamacker/cbor/pull/418
- Update README.md by @fxamacker in https://github.com/fxamacker/cbor/pull/419
- Update intro and cbor quick start in README by @fxamacker in https://github.com/fxamacker/cbor/pull/421
- Update README.md for v2.5.0 by @fxamacker in https://github.com/fxamacker/cbor/pull/422
- Update CONTRIBUTING.md by @fxamacker in https://github.com/fxamacker/cbor/pull/423
- Update README.md for v2.5.0 by @fxamacker in https://github.com/fxamacker/cbor/pull/424
New Contributors (alphabetically)
- @agaffney made their first contribution in https://github.com/fxamacker/cbor/pull/376
- @CodingVoid made their first contribution in https://github.com/fxamacker/cbor/pull/333
- @creachadair made their first contribution in https://github.com/fxamacker/cbor/pull/375
- @dedefer made their first contribution in https://github.com/fxamacker/cbor/pull/352
- @deeglaze made their first contribution in https://github.com/fxamacker/cbor/pull/353
- @immesys made their first contribution in https://github.com/fxamacker/cbor/pull/355
- @jdharms made their first contribution in https://github.com/fxamacker/cbor/pull/361
- @zensh made their first contribution in https://github.com/fxamacker/cbor/pull/335
Non-coding contibutions were made by opening notable issues that directly or indirectly improved this release.
- @burdiyan opened issue and followups that led to
Encode()returning buffer to pool as default behavior. - @espoal opened issue and followups that led to adding Skip() feature for CBOR Sequences (RFC 8742).
- @immesys opened multiple issues and provided helpful feedback in addition to merged PRs.
- @qmuntal opened issue and @x448 added feedback that led to improved support for CBOR Simple Values.
- @x448 opened multiple issues and provided helpful feedback in addition to merged PRs.
- @zensh opened multiple issues and provided helpful feedback in addition to merged PRs.
Full Changelog: https://github.com/fxamacker/cbor/compare/v2.4.0...v2.5.0