A real bid request is about 2,000 bytes. The audience vector that embedding-based targeting would put inside it is 16,000 if not used properly. The vector alone outweighs everything else in the request, several times over.
Embeddings let you target by meaning instead of by segment ID. Let’s take a closer look at what that vector could weigh on the wire. An average bid request runs about 2 kilobytes, and since everything on a bid path is gzipped and a request is mostly text and repeated field names, it compresses to about a kilobyte over the wire.
A full Gemini-class vector is 3,072 numbers, each one a 32-bit float at 4 bytes, so about 12 kilobytes of raw data and around 16 kilobytes once it is packed into the request. Next to a 2-kilobyte request that is roughly eight times the size of everything else combined: one field outweighing the device, the user, the page, the impression and the supply chain all together.
Compression does not close the gap. The rest of the request is text and repeated field names, so gzip halves it, while a vector is close to random numbers with no repeating pattern, so gzip barely touches it. The full vector still weighs about 12 kilobytes compressed, against a kilobyte for the request it rides in, so on the wire it ends up around ten times everything else combined.
That is a movement problem before it is anything else. Take an exchange with fifty buyers at a hundred thousand requests a second each, five million a second, around 430 billion bid requests a day. The requests on their own are about 430 terabytes of egress a day. Attach a full vector to every one and you add roughly five petabytes a day, more than ten times that. On public cloud every one of those gigabytes is metered on the way out. If you run the exchange there and pull up the egress line for the vectors alone, you will want to sit down first.
This is also where running the bidder inside the exchange changes the arithmetic. Bedrock Platform’s bidder runs inside Index Exchange’s Index Cloud, so the request never crosses a network to reach it. To reach a bidder in another cloud the exchange pays egress on every byte of the fat request; to reach a co-located one it pays none, whatever the vector weighs. So the vector’s size stops being an egress cost on that hop, which makes it practical to send the bidder more signal than a remote one would justify: several vectors, richer context, things that were never worth shipping across a network.
The full, full-precision vector is more than the match actually needs on the wire. There is a lot of room between it and a vector that still matches well. I will take a closer look at it in the next posts.
Comparing two vectors is cheap, one multiply-and-add per match. Moving them outside the datacenter is the expensive part: a few thousand numbers per user, millions of times a second. That is what you can shrink.
