Hoppa till innehåll
VibekollenBETAVibekollen
BloggOpenRouter

LLM Quantization: What fp8 and fp4 Do to Quality and Price

Artikelbild eller återanvändbart omslag för OpenRouter

When you call an open-weight model through an API, several providers may serve it at different precisions, such as 16-bit bf16, 8-bit fp8 or 4-bit fp4.

The precision can affect both the answers and the price. LLM quantization creates these versions by storing a model's weights in fewer bits, so the model needs less memory and hardware, at some cost in accuracy.

Läs hela hos OpenRouter →

Texten är källans egen beskrivning av publiceringen. Innehållet tillhör OpenRouter.

Mer att läsa