Running the bidder inside the exchange’s cluster allows something that is not an option between two data centers. The exchange and the bidder can sit on the same machine and talk through a file instead of the network, and in my test that cut the time of a gRPC call by about a third.
Between two pods, even on the same machine, messages usually go over TCP: the bytes are cut into packets with headers and checksums, and the other side confirms what it received. In Kubernetes those packets also pass through the pod network, a virtual link for each pod plus the cluster’s routing and network policy rules. A Unix socket is a file that both programs connect to, and the bytes never touch the disk. The kernel, the core of the operating system, copies the bytes from one program to the other and skips all of that work. It only works inside one machine, so over the internet it was never an option.
Inside the exchange’s cluster it needs two things: the bidder pod on the same machine as the exchange pod, and a shared folder on that machine for the socket file. gRPC stays the same, with the same messages, and only the address changes, from bidder:9090 to unix:///sock/bid.sock.
I tested it on real Kubernetes nodes in GCP, with 20,000 real bid requests sent over gRPC and a real bid on 1 in 100 of them. The bidder sat in three places: on another machine, on the same machine over TCP, and on the same machine over a Unix socket. Every case ran three times at 10,000 requests per second. A call took 235 µs between machines, 204 µs on the same machine over TCP and 141 µs over the Unix socket. CPU work dropped by 9 µs per request across both sides, about 4%, most of it on the bidder’s side, which at a million requests per second is about 9 cores. Memory did not change.
It also makes placing pods and balancing traffic more challenging, since the exchange and bidder pods have to land on the same machines and Kubernetes does not spread requests across socket files.
In practice, sharing a machine with the exchange is only possible for a bidder that runs as a container inside the exchange’s cluster. From a separate data center the bidder stays at least one network away, however fast that network is. Next to optional encryption and compression and formats read in place, the socket is one more efficiency that the distance used to rule out, both for the exchange and the bidder.
