Following up on #13966 (diffusers-cli polish for agents), it would be great to make the CLI more agent-optimized for quantization use cases. Exposing quantization through the CLI would reduce the number of tokens an agent has to try out (see the Is it agentic enough? study).
The quantization API surface is quite large with a lot of variation across quant configs, so the proposal is to expose it through PipelineQuantizationConfig via a --quantize flag that accepts a JSON config. For example:
diffusers-cli run \
--model black-forest-labs/FLUX.1-dev \
--quantize '{"quant_backend": "bitsandbytes_4bit", "quant_kwargs": {"bnb_4bit_compute_dtype": "bfloat16", "bnb_4bit_quant_type": "nf4"}, "components_to_quantize": ["transformer", "text_encoder_2"]}'
This doesn't have to be part of #13966 — it can be a separate follow-up.
We should also include a skill alongside it to cover all the variations in the quant configs.
Context: #13966 (discussion #13966 (comment)).
Requested by Sayak Paul - Slack thread - Agent trace
Following up on #13966 (
diffusers-clipolish for agents), it would be great to make the CLI more agent-optimized for quantization use cases. Exposing quantization through the CLI would reduce the number of tokens an agent has to try out (see the Is it agentic enough? study).The quantization API surface is quite large with a lot of variation across quant configs, so the proposal is to expose it through
PipelineQuantizationConfigvia a--quantizeflag that accepts a JSON config. For example:This doesn't have to be part of #13966 — it can be a separate follow-up.
We should also include a skill alongside it to cover all the variations in the quant configs.
Context: #13966 (discussion #13966 (comment)).
Requested by Sayak Paul - Slack thread - Agent trace