Robert Collins
http://lca2018.linux.org.au/schedule/presentation/77/
A war story: Developer re-implements a Google whitepaper in Rust using netmap for networking.
In 2016 Google released a whitepaper about their small packet wire rate – 40Gbps even, highly available IP load balancer. I happened to have some weeks shortly after that happened where I was able to do whatever I wanted… and I decided to have some fun and implement an open source look-alike.
Maglev has some distinctive design choices – such as the use of userspace networking to achieve zero-copy forwarding – which I knew about but had not actively used, as well as doing direct server return – sending packets to backend servers wrapped in GRE, but traffic in the other direction is routed directly, so the traffic pattern is asymmetric.
Since I was young and foolish then, I thought it would be good to also deepen my knowledge of the Rust language at the same time… and develop it on my not-as-Linux-friendly-as-I’d-like Surface : so my Linux
environment was actually a Hyper-V VM.
This combination led to discovering some truely fantastic facilities in Rust, bugs in the Linux kernel and the netmap project, and exploring some of the awkwardness where very clever C datastructures don’t map well to Rust’s ownership model.
The current status of the project is that the core use case works: traffic can be forwarded using network ring buffers, without needing a dedicated network card, routed with zerocopy while preserving GRE wrappers. Higher level facilities like dynamic reconfiguration and backend health signalling are not yet implemented (but are designed).
In the talk I will cover the choices I made and why – rust vs the world, netmap vs DPDK, things I found surprising about doing userspace networking, detail the gritty details of doing userspace networking with Netmap from Rust, using Rust to parse and create network packets, as well as driving netlink from within Rust. I’ll cover the bugs I had to fix in order to write the code I wanted to write.
This talk was given at Linux.conf.au 2018 (LCA2018) which was held on 22-26 January 2018 in Sydney Australia.
linux.conf.au is a conference about the Linux operating system, and all aspects of the thriving ecosystem of Free and Open Source Software that has grown up around it. Run since 1999, in a different Australian or New Zealand city each year, by a team of local volunteers, LCA invites more than 500 people to learn from the people who shape the future of Open Source. For more information on the conference see https://linux.conf.au/
#linux.conf.au #linux #foss #opensource
source
