Embedding ten million pages costs ten dollars with a small model and about two hundred with a bigger one, on the same machine with the same pages.
On Tuesday I timed how long it takes to turn one page into a vector. Those timings turn into a bill, so here it is. I rented one entry-level graphics card at about 85 cents an hour and fed it pages in batches of sixteen, which is how you would run this if you were working through a catalogue rather than answering a live request. Ten million pages came to roughly $10 with the smallest model I tested, $30 and $45 with two mid-sized ones, and $220 with the largest. Small and large here means the number of values the model learned during training, from 33 million up to 600 million, and more of them generally means a better model and a slower one.
All four models are free to use commercially, so the machine rental is most of what you pay. Two other things move the bill as much as the model does. Sending sixteen pages through at once instead of one was worth up to 12 times, because the card sits mostly idle on a single page. Storing each number in the vector in less space is worth another two to three times, going by published results.
So is running your own cheaper than buying it in? Yes, as long as the card stays busy. Priced the same way, per million words of text, the biggest model I ran cost four and a half cents, against thirteen to fifteen for cloud providers. The smaller models came in under a cent. The exception is OpenAI’s small model at two cents, which undercuts everything I ran.
The catch is that a rented card is billed by the hour whether it does anything or not, and those figures assume it never stops working. Start a job overnight, leave the machine up all day, and the card works under a fifth of the time, so the real cost is five or six times higher and the hosted price wins. Run the job hard and hand the machine back and the figures hold. For the biggest model the line sits near a third: busier than that and running it yourself is cheaper than Google’s price, below it you are paying for an idle machine.
Where it starts to add up is refreshing user profiles rather than embedding a catalogue once. Re-embedding tens of millions of histories every day is a standing job rather than a one-off, so you are renting cards continuously, and the bill lands somewhere in the tens to hundreds of thousands a year depending on how many users you carry, how often you refresh, how long each history runs and which model you point at it. The direction that changes the picture is a bigger model. Cost tracked parameter count closely across the four I ran, so fine-tuning something several times larger multiplies all of this in step. Long histories push the same way, because past a few thousand words the cost climbs faster than the text does.
What it costs to embed ten million pages
