Targeting via embedding carries a vector inside the bid request’s segment object, and IAB Tech Lab is standardising a field for it. Producing that vector costs time, CPU and memory. I measured the time.
I launched two machines, one CPU only with eight cores, one with a GPU, an entry-level NVIDIA L4. Four open models between 33 and 600 million parameters, and eight realistic advertising texts (campaign briefs, targeting rules, a blog page, CTV metadata, a ninety-day user history) each written at roughly 100, 225 and 450 tokens. Every number below is a median of 160 timed runs.
On the GPU a full page took 14 milliseconds with the 33M model, 16 with 109M, 20 with 137M and 93 with the 600M one. Three of the four sit within 1.5x of each other, so which model you pick barely shows. On the CPU the same four span 61 milliseconds to 1.1 seconds, an eighteenfold spread.
Time splits into a fixed cost paid on every call plus a cost that grows with the text, and the two machines sit at opposite ends of that. The CPU’s per-token cost is high and tracks parameter count: 0.13 milliseconds for the 33M model, 0.31 for 109M, 0.41 for 137M, 2.32 for the 600M one. The GPU does that arithmetic 30 to 66 times faster, but pays a fixed 11 to 17 milliseconds on every call however short the text, which is why its numbers bunch up. Below roughly 95 tokens that overhead is bigger than the arithmetic it saves and the CPU is ahead.
These numbers tell you how big a job to plan for, not how fast you have to be. A vector gets made once and then reused, so what drives the volume is how fine you cut it. One vector per CTV episode is cheap. One per scene, so the ad matches what just happened on screen, is hundreds. Same on the web: a page once, or every section of a long article. Then it is how often you redo it: a news homepage a few times a day, an archived article once ever. Feeding texts through in groups speeds this up while the card has room to spare, up to 12 times on short text, though on my entry-level GPU that headroom ran out on full pages.
Live embedding is not off the table either. Some combinations of model, text length and hardware could land inside an auction window, others miss it by a wide margin, and the line moves every time you change one of the three. Making the vector in advance means never having to work out which side you are on.
Caveats: four models, one machine of each kind, one run per cell, so treat any difference under two percent as noise. The largest model I tested was 600M parameters and plenty of production systems run bigger, so the quick numbers here are a floor rather than a typical case. The models you would actually pick for quality are the slow ones, and the slower the model, the longer a vector has to stay in use to be worth making. Which turns this into a question about staleness: how out of date a vector may get before it stops describing what it belongs to.
What it costs to turn one page of text into a vector
