Running the bidder inside the exchange’s cluster opens options the internet never allowed. One of them makes saying no to a bid request almost free, and a bidder says no to most requests it gets.
Over the internet, bid requests are usually compressed with gzip, because every byte between exchange and bidder costs money. Inside the exchange’s cluster the bytes stay on a private network, so compression becomes optional. That allows a format the bidder can read in place without unpacking it first. The one I tested is FlatBuffers, built at Google in 2014 for games. Gzip would break it, since the whole message must be unpacked before anything can be read.
Today a bidder decodes every request before checking it. Decoding means reading the request from start to end and copying each of its fields into the bidder’s memory, and a real request in the traffic has about 95 fields. Later the garbage collector, the part of the program that frees unused memory, has to clean those copies up. All of this happens before the first check, and the checks use little of it: targeting reads a few fields (deal, country, device type, video size), the price check reads the floor, and the budget check reads none.
A zero-copy format lets the bidder read a field straight from the bytes it received. In FlatBuffers every object starts with a small table of positions, so reading the country means looking up its position and reading those bytes. Fields the bidder never asks for are never touched. Protobuf, the format behind gRPC, writes fields in a row, so a reader walks past everything before the field it wants.
I tested it on 20,000 real bid requests, with one schema for both formats, running the same checks against the four line items that bid in that sample. All formats made the same decisions, and all 225 requests we really bid on passed. Reading and checking a request cost 3.7 µs with JSON, 3.1 µs with protobuf and 0.26 µs with FlatBuffers. On the exchange, building the request took about the same in both, 2.2 µs in protobuf and 2.6 µs in FlatBuffers, so most of the gain is on the bidder’s side.
Multiplied out to a million requests per second, that is about 3 cores against 0.3 cores spent reading requests. Memory shows a bigger gap: protobuf decoding copied 4.4 KB per request into memory, FlatBuffers 1.1 KB, and that was in a test where one in five requests passed targeting and was read in full. The garbage collector’s work per request dropped about four times with it.
Exchanges often cap the requests a bidder gets because handling each one costs too much. When a no costs a fraction of a microsecond, a bidder can take more of the supply on the same machines. For buyers and advertisers, a deal can then be seen in full, without the impressions a cap drops before the bidder ever sees them. For publishers, every request a bidder can afford to look at is one more chance for an impression to find a buyer.
