Chapter 4 Encoding and Evolution

4.1 Avro

4.2 Thrift

4.3 Protocol Buffers

Protocol Buffers Internals
https://www.safaribooksonline.com/videos/complete-introduction-to/9781789349344/9781789349344-video8_7

http://www.blogjava.net/DLevin/archive/2015/04/01/424012.html

除了序列化框架,protobuf还定义了一套简单的RPC框架.之所以说简单是因为它定义的Service层接口的协议,而没有具体和传输相关的实现,而只是将传输相关的逻辑抽象成RpcChannel和BlockingRpcChannel分别用于表示同步和一步方式的Service方法调用,而至于底层用什么样的协议和框架,由用户自己决定并实现.

Protocol buffers messages always use little-endian encoding. Implementations running on big-endian architectures should be doing the conversions automatically.

4.4 Flatbuffer

Zerocopy

Posted on Thursday, August 17, 2017 by Wouter van Oortmerssen The recent release of Flatbuffers version 1.7 introduced truly zero-copy support for gRPC out of the box.

Flatbuffers is a serialization library that allows you to access serialized data without first unpacking it or allocating any additional data structures. It was originally designed for games and other resource constrained applications, but is now finding more general use, both by teams within Google and in other companies such as Netflix and Facebook.

Flatbuffers enables maximum throughput by directly using gRPC’s slice buffers with zero-copy for common use cases. An incoming rpc can be processed directly from gRPCs internal buffers, and constructing a new message will write directly to these buffers without intermediate steps.

This is currently, fully supported in the C++ implementation of FlatBuffers, with more languages to come. There is also an implementation in Go, which is not entirely zero copy, but still very low on allocation cost (see below).

https://google.github.io/flatbuffers/flatbuffers_internals.html

Each scalar is also always represented in little-endian format, as this corresponds to all commonly used CPUs today. FlatBuffers will also work on big-endian machines, but will be slightly slower because of additional byte-swap intrinsics.

https://stackoverflow.com/questions/32759826/how-can-i-rewrite-the-protobuf-scheam-with-flatbuffer-schema