Cap'n Proto

Like most RPC frameworks dating as far back as Sun RPC and OSF DCE RPC (and their object-based descendants CORBA and DCOM), Cap'n Proto uses an Interface Description Language (IDL) to generate RPC libraries in a variety of programming languages - automating many low level details such as handling network requests, converting between data types, etc. The Cap'n Proto interface schema uses a C-like syntax and supports common primitives data types (booleans, integers, floats, etc.), compound types (structs, lists, enums), as well as generics and dynamic types.{{Cite web|last=Varda|first=Kenton|date=|title=Cap'n Proto Schema Language|url=https://capnproto.org/language.html|url-status=live|archive-url=https://web.archive.org/web/20150317184515/https://capnproto.org/language.html |archive-date=2015-03-17 |access-date=2020-09-05|website=}} Cap'n Proto also supports object-oriented features such as multiple inheritance, which has been criticized for its complexity.{{Cite web|last=Denhardt|first=Ian|date=June 2019|title=A Critique of the Cap'n Proto Schema Language.|url=https://zenhack.net/2019/06/25/a-critique-of-the-capnproto-schema-language.html|url-status=live|archive-url=https://web.archive.org/web/20190626022038/https://zenhack.net/2019/06/25/a-critique-of-the-capnproto-schema-language.html |archive-date=2019-06-26 |access-date=2020-10-10|website=zenhack.net}}

@0xa558ef006c0c123; # Unique identifiers are manually or automatically assigned to files and compound types

struct Date @0x5c5a558ef006c0c1 {

year @0 :Int16; # @n marks order values were added to the schema

month @1 :UInt8;

day @2 :UInt8;

}

struct Contact @0xf032a54bcb3667e0 {

name @0 :Text;

birthday @2 :Date; # fields can be added anywhere in the definition, but their numbering must reflect the order in which they were added

phones @1 :List(PhoneNumber);

struct PhoneNumber { # Compound types without a static ID cannot be renamed, as automatic IDs are deterministically generated

number @0 :Text;

type @1 :PhoneType = mobile; # Default value

enum PhoneType {

mobile @0;

landline @1;

}

}

}

Values in Cap'n Proto messages are represented in binary, as opposed to text encoding used by "human-readable" formats such as JSON or XML. Cap'n Proto tries to make the storage/network protocol appropriate as an in-memory format, so that no translation step is needed when reading data into memory or writing data out of memory.Unlike Apache Arrow, Cap'n Proto's in-memory values are [https://news.ycombinator.com/item?id=14248780 not suited for sharing mutable data] For example, the representation of numbers (endianness) was chosen to match the representation the most popular CPU architectures.{{Cite web|first=Kenton|last=Varda|date=|title=Cap'n Proto: Introduction|url=https://capnproto.org/index.html|url-status=live|archive-url=https://web.archive.org/web/20150317185148/https://capnproto.org/index.html |archive-date=2015-03-17 |access-date=2020-11-09|website=Cap'n Proto Homepage}} When the in-memory and wire-protocol representations match, Cap'n Proto can avoid copying and encoding data when creating or reading a message and instead point to the location of the value in memory. Cap'n Proto also supports random access to data, meaning that any field can be read without having to read the entire message.{{Cite web|last=Varda|first=Kenton|date=|title=Cap'n Proto: Encoding Spec|url=https://capnproto.org/encoding.html|url-status=live|archive-url=https://web.archive.org/web/20150317202732/https://capnproto.org/encoding.html |archive-date=2015-03-17 |access-date=|website=Cap'n Proto}}

Unlike other binary serialization protocols such as XMI, Cap'n Proto considers fine-grained data validation at the RPC level an anti-feature that limits a protocol's ability to evolve. This was informed by experiences at Google where simply changing a field from mandatory to optional would cause complex operational failures.{{Cite web|first=Kenton|last=Varda|date=|title=FAQ § How do I make a field "required", like in Protocol Buffers?|url=https://capnproto.org/faq.html#how-do-i-make-a-field-required-like-in-protocol-buffers|url-status=live|archive-url=https://web.archive.org/web/20150318011733/https://capnproto.org/faq.html |archive-date=2015-03-18 |access-date=2020-09-05|website=Cap'n Proto}}Marking a field as required was removed from [https://developers.google.com/protocol-buffers/docs/proto3 Protocol Buffers 3]. Cap'n Proto schemas are designed to be flexible as possible and pushes data validation to the application level, allowing arbitrary renaming of fields, adding new fields, and making concrete types generic.{{Cite web|title=Cap'n Proto: Schema Language|url=https://capnproto.org/language.html#evolving-your-protocol|access-date=2020-10-10|website=capnproto.org}} Cap'n Proto does, however, validate pointer bounds and type check individual values when they are first accessed.

Enforcing complex schema constraints would also incur significant overhead,Assuming the data has already been allocated (e.g. in network buffers, read from disk) access becomes O(1). Additional serialization/deserialization steps (as required to inspect values) would limit performance to O(n). negating the benefits of reusing in-memory data structures and preventing random access to data.{{Cite web|title=Cap'n Proto: Cap'n Proto, FlatBuffers, and SBE|url=https://capnproto.org/news/2014-06-17-capnproto-flatbuffers-sbe.html|access-date=2020-10-10|website=capnproto.org}} Cap'n Proto protocol is theoretically suitable{{Cite web|last=Richardson|first=Corey|date=October 2016|title=Robigalia: An Operating System for the Modern Era|url=https://robigalia.gitlab.io/book/rosme.html#_capabilities_in_phoma|url-status=live|archive-url=https://web.archive.org/web/20180915204543/https://robigalia.gitlab.io/book/rosme.html |archive-date=2018-09-15 |access-date=2020-10-10|website=robigalia.gitlab.io}} for very fast inter-process communication (IPC) via immutable shared memory, but as of October 2020 none of the implementations support data passing via shared memory.{{Cite web|last=Kenton|first=Varda|date=May 3, 2017|title=Why is not intended that in-memory state be in Cap'n Proto / Protobuf objects?|url=https://news.ycombinator.com/item?id=14252940|access-date=2020-10-10|website=Hacker News (news.ycombinator.com)}} However, Cap'n Proto is still generally considered faster than Protocol Buffers and similar RPC libraries.{{Cite web|last=Naughton|first=Chris|date=Aug 24, 2018|title=Protocol Benchmarks|url=https://github.com/ChrisMacNaughton/proto_benchmarks|url-status=live|archive-url=https://web.archive.org/web/20180830220546/https://github.com/ChrisMacNaughton/proto_benchmarks |archive-date=2018-08-30 |access-date=2020-09-05|website=Github}}{{Cite web|last=Parimi|first=Dinesh|date=2019|title=Datacenter Tax Cuts: Improving WSC Efficiency Through Protocol Buffer Acceleration|url=https://people.eecs.berkeley.edu/~kubitron/courses/cs262a-F19/projects/reports/project4_report_ver2.pdf|url-status=live|archive-url=https://archive.today/20200906032310/https://people.eecs.berkeley.edu/~kubitron/courses/cs262a-F19/projects/reports/project4_report_ver2.pdf|archive-date=2020-09-06|access-date=2020-09-05|website=}}

Cap'n Proto RPC is network aware: supporting both handling of disconnects and promise pipelining, wherein a server pipes the output of one function into another function. This saves a client a round trip per successive call to the server without having to provide a dedicated API for every possible call graph. Cap'n Proto can be layered on top of TLS{{Cite web|title=Cap'n Proto: Road Map|url=https://capnproto.org/roadmap.html|access-date=2020-10-10|website=capnproto.org}} and support for the Noise Protocol Framework is on the roadmap.{{Cite web|date=2021-03-13|title=Roadmap|url=https://capnproto.org/roadmap.html#rpc-protocol-features|url-status=live|website=Cap'n Proto|archive-url=https://web.archive.org/web/20150317192558/https://capnproto.org/roadmap.html |archive-date=2015-03-17 }} Cap'n Proto RPC is transport agnostic, with the mainline implementation supporting WebSockets, HTTP, TCP, and UDP.{{Cite web|title=Cap'n Proto: C++ RPC|url=https://capnproto.org/cxxrpc.html|access-date=2020-10-10|website=capnproto.org}}

The Cap'n Proto RPC standard has a rich capability security model based on the CapTP protocol used by the E programming language.{{Cite web|title=RPC Protocol|url=https://capnproto.org/rpc.html#protocol-features|url-status=live|website=Cap'n Proto|archive-url=https://web.archive.org/web/20150318011754/https://capnproto.org/rpc.html |archive-date=2015-03-18 }}

{{expand section|date=March 2021}}

As of October 2020, the reference implementation only supports level 2.

Cap'n Proto is often compared to other zero-copy serialization formats, such as Google's FlatBuffers and Simple Binary Encoding (SBE).{{cite web|title=Why flatbuffers instead of capnp?|url=https://github.com/google/flatbuffers/issues/2}}

Cap'n Proto was originally created for Sandstorm.io, a startup offering a web application hosting platform with capability-based security. After Sandstorm.io failed commercially, the development team was acqui-hired by Cloudflare,{{Cite web|last=Varda|first=Kenton|date=13 Mar 2017|title=The Sandstorm Team is joining Cloudflare|url=https://sandstorm.io/news/2017-03-13-joining-cloudflare|url-status=live|archive-url=https://web.archive.org/web/20170313180618/https://sandstorm.io/news/2017-03-13-joining-cloudflare |archive-date=2017-03-13 |access-date=2020-09-05|website=Sandstorm.io}} which uses Cap'n Proto internally.{{Cite web|last=Zhi|first=Jiale|date=2013|title=Introducing lua-capnproto: better serialization in Lua|url=https://blog.cloudflare.com/introducing-lua-capnproto-better-serialization-in-lua/|url-status=live|archive-url=https://web.archive.org/web/20140306184242/http://blog.cloudflare.com:80/introducing-lua-capnproto-better-serialization-in-lua |archive-date=2014-03-06 |access-date=2020-09-05|website=}}

{{reflist|group=note}}

{{reflist}}

{{Data Exchange}}

{{DEFAULTSORT:Capn Proto}}

Category:Data serialization formats

Category:Remote procedure call

Category:Inter-process communication