Hailuo MiniMax H3 training support - #3003
Merged
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds end-to-end SimpleTuner integration for the MiniMax H3 model family (including pipelines/blocks and model registry wiring), and updates docs/tests to reflect the new support. It also extends the conditioning-data pipeline for i2v_first_frame (video → first-frame PNG) so autogenerated conditioning datasets and their metadata can be generated, duplicated, and sampled consistently.
Changes:
- Register the
minimaxh3model family and add a full MiniMax H3 implementation (scheduler, packing, modular pipeline blocks, encoders/decoders/denoise, audio autoencoder, etc.). - Add/adjust i2v-first-frame conditioning behavior across metadata duplication, conditioning generation, and sampler path resolution, with expanded unit tests.
- Update READMEs and QUICKSTART docs (all language variants) to include MiniMax H3 in supported-model and compatibility matrices; add an example training config.
Reviewed changes
Copilot reviewed 52 out of 52 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tests/test_training_sample.py | Adds tests for explicit pairing metadata and i2v-first-frame metadata duplication behavior. |
| tests/test_sdr_sample_generator.py | Adds a test ensuring i2v-first-frame targets .png filenames. |
| tests/test_sampler.py | Adds a test validating conditioning sampling maps video paths to .png for i2v-first-frame. |
| tests/test_s2v_auto_split.py | Makes the audio-isolation test use temp dirs for better test hygiene. |
| tests/test_factory_edge_cases.py | Adds coverage ensuring i2v video datasets auto-generate first-frame conditioning when absent. |
| simpletuner/helpers/multiaspect/sampler.py | Adds i2v-first-frame conditioning path mapping (.mp4 → .png) during conditioning sample fetch. |
| simpletuner/helpers/models/model_metadata.json | Registers minimaxh3 metadata (class/module path, flavors). |
| simpletuner/helpers/models/minimaxh3/scheduler.py | Introduces a MiniMax-H3-specific rectified-flow Euler scheduler implementation. |
| simpletuner/helpers/models/minimaxh3/pipeline.py | Adds a conventional pipeline entry point for FL2VA/T2VA. |
| simpletuner/helpers/models/minimaxh3/pipeline_ref.py | Adds a conventional pipeline entry point for Ref2VA. |
| simpletuner/helpers/models/minimaxh3/packing.py | Implements packed-sequence geometry/constants and packing/unpacking utilities. |
| simpletuner/helpers/models/minimaxh3/modular_pipeline.py | Adds modular pipeline and LoRA loading/saving integration for MiniMax H3. |
| simpletuner/helpers/models/minimaxh3/modular_blocks_minimax_h3.py | Defines modular block graph(s) for FL2VA/T2VA and Ref2VA workflows. |
| simpletuner/helpers/models/minimaxh3/encoders.py | Adds text encoding and visual/audio reference encoding steps for MiniMax H3 workflows. |
| simpletuner/helpers/models/minimaxh3/denoise.py | Adds denoise loop with optional real CFG + skipped-layer guidance support. |
| simpletuner/helpers/models/minimaxh3/decoders.py | Adds video/audio decode steps to produce final outputs from denoised rows. |
| simpletuner/helpers/models/minimaxh3/before_encoder.py | Adds setup steps resolving canvas/frame counts and reference preparation. |
| simpletuner/helpers/models/minimaxh3/before_denoise.py | Adds layout building, noise preparation, and timestep-plan staging. |
| simpletuner/helpers/models/minimaxh3/autoencoder_audio.py | Adds MiniMax-H3 audio VAE (DAC/BigVGAN lineage) implementation. |
| simpletuner/helpers/models/minimaxh3/activations.py | Adds MiniMax-H3 MLP/SwiGLU activation helpers. |
| simpletuner/helpers/models/minimaxh3/init.py | Exposes MiniMax H3 components via package exports. |
| simpletuner/helpers/models/common.py | Adds minimaxh3 to supported families and upstream config sources. |
| simpletuner/helpers/metadata/utils/duplicator.py | Adds i2v-first-frame-aware path translation and metadata copying rules for conditioning datasets. |
| simpletuner/helpers/image_manipulation/training_sample.py | Allows explicit training_sample_path metadata to override training path resolution. |
| simpletuner/helpers/data_generation/sample_generator.py | Adds .png target extension for I2VFirstFrameSampleGenerator. |
| simpletuner/helpers/data_generation/conditioning.py | Supports generator-defined target extensions and refines bucket processing accounting/timeout logic. |
| simpletuner/helpers/data_backend/factory.py | Switches implicit i2v conditioning to first-frame generation and extends config versioning + text-embed caching hooks. |
| simpletuner/helpers/caching/vae.py | Includes minimaxh3 in video-latents preparation handling. |
| simpletuner/examples/minimaxh3-fl2va-convrot-int8.peft-lora/config.json | Adds an example config for training MiniMax H3 FL2VA convrot-int8 LoRA. |
| README.md | Adds MiniMax H3 to the supported model family list (license/conditions). |
| README.zh.md | Adds MiniMax H3 to the supported model family list (Chinese). |
| README.ja.md | Adds MiniMax H3 to the supported model family list (Japanese). |
| README.pt-BR.md | Adds MiniMax H3 to the supported model family list (pt-BR). |
| README.es.md | Adds MiniMax H3 to the supported model family list (Spanish). |
| README.hi.md | Adds MiniMax H3 to the supported model family list (Hindi). |
| documentation/quickstart/index.md | Adds MiniMax H3 to quickstart model overview tables and license footnotes. |
| documentation/quickstart/index.zh.md | Same as above (Chinese). |
| documentation/quickstart/index.ja.md | Same as above (Japanese). |
| documentation/quickstart/index.pt-BR.md | Same as above (pt-BR). |
| documentation/quickstart/index.es.md | Same as above (Spanish). |
| documentation/quickstart/index.hi.md | Same as above (Hindi). |
| documentation/QUICKSTART.md | Adds MiniMax H3 to compatibility matrices and model guide tables. |
| documentation/QUICKSTART.zh.md | Same as above (Chinese). |
| documentation/QUICKSTART.ja.md | Same as above (Japanese). |
| documentation/QUICKSTART.pt-BR.md | Same as above (pt-BR). |
| documentation/QUICKSTART.es.md | Same as above (Spanish). |
| documentation/QUICKSTART.hi.md | Same as above (Hindi). |
Suppressed comments (2)
simpletuner/helpers/data_generation/conditioning.py:518
- In CPU mode,
_check_completion_queue()only returns the number of successful items. If a worker completes a batch with partial failures (successful < total), those failures are never accounted for here, which later makesprocessed < dispatchedpermanently true and can force the end-of-bucket wait loop to run until the 300s timeout.
This issue also appears on line 552 of the same file.
if not self.gpu_mode:
done = self._check_completion_queue()
stats["processed"] += done
simpletuner/helpers/data_generation/conditioning.py:556
- The end-of-bucket wait loop compares
processed(successes) againstdispatched(attempted). If any dispatched items fail,processedcan never reachdispatched, so this loop will always run until the full 300s timeout even though all work has already finished (just with failures). Track failures and compare against(processed + errors)instead.
while stats["processed"] < dispatched and time.time() - start < 300:
completed = self._check_completion_queue()
if completed:
stats["processed"] += completed
time.sleep(0.1)
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
added 26 commits
August 5, 2026 16:22
When duplicating metadata, absolute conditioning paths outside the source dataset root now resolve to a basename under the target root instead of carrying relpath .. segments. Add coverage for inside-root paths, outside-root absolute paths, and i2v first-frame extension rewriting so future metadata copies cannot escape the target directory.
Add a shared SDNQ compile-mode helper and call it during argument parsing, trainer startup, and ConvRot SDNQ loader setup so eager versus torch.compile mode is applied before SDNQ modules are imported. Preserve the existing warning when SDNQ was imported too early, avoid duplicate warnings after a successful prior configuration, and cover the parse-time and loader-time paths in quantization config tests.
Register an h3_drift distiller that disables the active LoRA or LyCORIS adapter for a frozen parent prediction, then reduces video and optional audio prediction drift with token-balanced or modality-balanced weighting. Expose the method through the distillation factory, WebUI distillation selector, CLI option docs, and tests covering factory creation, adapter restoration after failures, video-only batches, and text-encoder training guards.
Add MiniMax-H3 target modality handling so the default path trains video-only, data backends can opt into joint audio-video rows, and image-latent batches drop stale cached audio instead of training an impossible soundtrack branch. Teach H3 validation and text encoding to handle CFG-distilled negative branches: caption tokens cap at 512 by default, empty negative prompts preserve positive prompt length with null token ids, and validation defers context-dependent negative prompt encoding per sample. Extend the H3 transformer/runtime with true one-frame image geometry, configurable SwiGLU gate ordering for Comfy-style checkpoints, reduced-precision AdaLN safeguards, cached static reference KV inference, musubi stream-out coverage, segmented checkpointing capability, field-registry entries, and regularisation parent audio targets. Update H3 docs, validation defaults, safety checks, collation/backend audio gating, and focused tests for image mode, target modes, reference cache, validation prompting, checkpoint controls, and audio regularisation targets.
Update the trainer test fixture to expect the default TorchDynamo mode when regional compilation is enabled, matching the current trainer path and avoiding max-autotune/cudagraph memory assumptions in that coverage.
Add separate 24G, 32G, 48G, and 80G MiniMax-H3 ConvRot INT8 LoRA example configs from the benchmark sweep, keeping the 24G preset on RamTorch and the larger presets on the measured segmented-checkpointing or no-checkpoint paths. Each preset keeps validation negative prompting disabled for the CFG-distilled default path, enables VAE slicing/tiling/temporal roll, and uses the shared 480p 39-frame FL2VA dataset config as the baseline workload.
When trainer initialization reads configuration from StateTracker, distinguish a genuinely missing sdnq_compile_mode attribute from an explicit invalid value. This avoids plain Mock-based config objects fabricating a child mock that fails SDNQ mode validation. Add regression coverage for config objects without a real sdnq_compile_mode while preserving validation for explicit compile/eager values.
Inspect init_lora checkpoints before constructing PEFT configs so mixed-rank checkpoints can populate rank_pattern and alpha_pattern instead of being forced through the global rank. Infer missing alpha values from per-module ranks when a checkpoint has mixed ranks but no explicit alpha tensors, and preserve explicit alpha tensors when present. Apply the same inference path for validation adapters and direct LoRA weight loading. Route vendored pipelines through SimpleTuner's get_peft_kwargs wrapper so Diffusers loader changes do not drop layerwise rank or alpha metadata.
Add a ModelFoundation flow-target contract for models whose public prediction direction is the inverse of the common noise-minus-latent convention. Use that contract in AnyFlow, Flow DPO, PerFlow, DMD/self-forcing wrappers, TwinFlow adversarial reconstruction, and diffusion-to-flow targets so teacher rollouts and student losses stay in the correct sign convention. Mark existing reverse-flow model families through the shared contract and cover inverse-flow behavior with distillation and OmniGen/TwinFlow regression tests.
Apply an SDNQ workaround before post-load quantization so an already-quantized SDNQTensor is dequantized before being requantized outside fake tensor mode. Keep Quanto text-encoder handling available for mixed SDNQ setups without sending the SDNQ base model through the Quanto branch a second time. Allow the existing Quanto include-path workaround to forward newer torch include_paths arguments unchanged.
Patch Diffusers' LoraBaseMixin.unfuse_lora so pipeline-level merged adapter tracking reflects the adapters that remain physically merged in any loadable component. This preserves fused_loras and num_fused_loras when only one component is unfused, and clears the tracking only after every component has actually been unmerged. Add a regression test covering a two-component PEFT pipeline with a partial unfuse followed by the final unfuse.
Patch Diffusers templated CUDNN and native flash attention backward wrappers so saved query, key, and value tensors are passed in the BHSD layout they were stored with. Only transpose grad_out into the backend layout before calling the ATen backward op, then transpose returned gradients back to the templated attention public layout. Add regression tests that mock the ATen backward operators and assert the saved key/value tensors are not transposed twice.
Compute LTX2 decoder up-block input channels from the current decoder width and project mismatched inputs to the upsampler width before upsampling. This keeps non-nominal constant-width decoder configurations from sending tensors with the wrong channel count into the 3D upsampler path. Add tests for projection, projection skipping, and a compact decoder shape that previously failed with mismatched widths.
Add AST-based regression coverage that both LTX Video 2 text-to-video and image-to-video pipelines pass the actual video_sequence_length into calculate_shift. This guards against reverting to shape assumptions that undercount the packed video sequence when dynamic timestep shifting is enabled.
Use a stable 16 FPS fallback when validation or checkpoint inference exports video and the config framerate is unset. Let models with special pipeline scheduler setup bypass distiller scheduler replacement, and centralize media extraction so validation accepts frames, images, videos, audio, and audio-only frame-less results consistently. Move MiniMax H3 token tag tensors to the inference device even though they are integer tensors, and switch checkpoint inference to the model hook that decides whether negative validation prompts are active.
Mark validation prompt encoding on the model while text embeds are produced so model-specific encoders can allow validation-only contextless prompts without relaxing normal dataset caption rules. When multiple cached prompt records are returned, give the model a chance to collate prompt embeddings before falling back to generic tensor concatenation, preserving auxiliary tensors such as MiniMax H3 token tags. Add regression tests for validation markers, contextless validation encoding, and model-owned collation of extra prompt-embed tensors.
Add a composed distillation requirement resolver so h3_drift includes the dataset/cache requirements of an optional inner distiller when validating data-backend and trainer setup. Teach H3DriftDistiller to construct a non-recursive inner SimpleTuner distiller, delegate batch preparation, scheduler/cache/caption hooks, generator/discriminator steps, and checkpoint lifecycle hooks, then add the frozen-base H3 drift loss on top. Preserve inner distiller logs with h3_drift_inner_total for the nested total and cover AnyFlow composition, recursive rejection, and composed requirement profiles in tests.
Explain how h3_drift can wrap another SimpleTuner distiller for step-distillation or preference objectives while still preserving the frozen MiniMax H3 base prediction. Update all H3 drift translations with the inner_distillation_method and inner_distillation_config settings, delegated hooks, compatibility behavior, and h3_drift_inner_total log. Enable h3_drift by default in the checked-in MiniMax H3 ConvRot INT8 example configs with a lighter drift weight for adapter training.
Add MiniMax H3 VAE single-file loading for raw, Diffusers, and Comfy ConvRot INT8 checkpoints, including fused QKV splitting, ConvRot metadata validation, and raw/Diffusers SwiGLU gate-order handling. Force the upstream H3 video VAE tiling and temporal chunking settings through check_user_config and load_vae so missing or false options do not produce unstable untiled decodes; document that behavior in the H3 quickstart translations. Support convrot-int8 VAE path defaults, Diffusers modular component registration, audio-optional scheduler paths, precomputed prompt embeds with token tags, validation negative-prompt gating for real CFG, and device-safe token-tag packing. Add MiniMax H3 context-parallel plan coverage plus regression tests for VAE single-file loading, ConvRot VAE wrapping, H3 LoRA conversion/init_lora handling, validation prompt context, and video-only modular pipeline paths.
Normalize malformed webshart:/ sample IDs before path handling so cache keys and backend lookups resolve to the same logical sample. Honor webshart_shard_cache_gb=0 by disabling whole-shard caching instead of falling back to the default cache size, and reject negative cache sizes. Add video-aware metadata and sample loading for Webshart shards, including ffprobe-based bucket derivation when indexed metadata is incomplete. Reset compressed torch streams before fallback loading so cache reads do not depend on the caller's stream position. Document the zero shard-cache behavior and cover URI normalization, cache keys, builder settings, metadata, and video-loading paths with tests.
Correct H3's dataward timestep plumbing so training, AnyFlow teacher rollouts, FlowMap conditioning, and validation paths use the model-specific 1 - sigma convention instead of raw sigma values. Rework the H3 drift distiller to keep a normal H3 SFT anchor when an inner distiller rewrites timestep conditioning, while still applying the frozen-base prediction drift loss and logging the SFT component separately. Fix MiniMax H3 VAE single-file loading for Comfy head-interleaved QKV tensors, including ConvRot INT8 weights and scales, and cover the loader behavior with regression tests. Batch H3 Qwen3-VL prompt and image encoding, trim cached prompt embeddings by true token length, and refresh validation VAE/audio VAE components after unload/reload transitions. Add the audio_flow_schedule_shift option through CLI docs, SDK field metadata, H3 defaults, quickstart translations, and MiniMax H3 example configs. Add an H3 objective geometry diagnostic CLI with tests so failed drift/AnyFlow adapters can be compared against base, normal-target, and FlowMap objective directions.
Add sparse range probing for remote Webshart video samples so ffprobe can read container metadata without downloading full TAR members when offset and length metadata are available. Prepare Webshart video bucket metadata in bounded parallel chunks using the backend download parallelism setting, while preserving the existing periodic cache flush behavior. Make sample limiting and shuffled distributed bucket splitting independent of discovery order so every rank partitions the same serialized cache contents. Reload refreshed bucket caches on every rank before splitting and prevent rank-local prompt records from being split a second time during text embed cache preparation. Cover range reads, sparse probing, deterministic sample limiting, deterministic bucket shuffling, bucket cache reloads, and rank-local text embedding cache behavior.
Pass the active model through prompt-embedding fallback paths so empty-prompt dropout cache keys use the model being trained rather than stale state attached to the default cache backend. Cover filename-keyed caches where the active model disables the dropout sentinel and the original dataset sample key must be preserved.
Avoid running the frozen-base reference prediction when h3_drift loss_weight is zero, while still preserving the configured SFT or inner-distiller objective. Return zero-valued drift metrics in this mode so training logs remain structurally stable without spending memory or compute on an unused reference pass. Cover the zero drift-weight path with an H3 drift distiller regression test.
added 28 commits
August 10, 2026 09:02
Honor max_num_samples when the sampler builds model-card overview statistics for multi-process jobs so the displayed sample total does not exceed the configured cap. Read nested video frame and FPS settings when legacy flat fields are absent, and show unknown values explicitly instead of rendering None.
Allow optimizer_beta1 and optimizer_beta2 to add a betas tuple even when optimizer_config also supplies optimizer-specific keyword overrides. Add unittest coverage for mixed optimizer_config and generic beta override parsing.
Let EMAModel copy live weights through an initial warmup window, then switch directly to the configured fixed EMA decay. Persist the warmup setting in EMA state, pass it from Trainer, expose it in the field registry and options documentation, and cover the copy-through behavior in tests.
Detect TensorBoard video writer support before passing 5D validation tensors to add_video, because installs without the legacy moviepy.editor module reject video media. When video logging is unavailable, log the first frame as an image so validation summaries still render instead of failing the tracker path.
Add repo-subfolder support to Hub model, LoRA, EMA, and validation-asset uploads so intermediary checkpoints can be preserved under checkpoint-specific paths. When checkpoint pushing is enabled, upload both the checkpoint subfolder copy and the refreshed top-level artifacts, and return the checkpoint URL separately from the latest model URL.
Use model-owned scheduler labels, identify video families with text-to-video and image-to-video tags, and emit optional license_name/license_link metadata when a model provides it. Keep widget asset URLs repo-relative, avoid double-prefixing already namespaced LoRA repo IDs, and cover MiniMax H3 and AnyFlow scheduler card output in tests.
Treat an empty benchmark directory as missing so startup and base validation are not skipped after a partial or failed benchmark write. Add scheduler-cache coverage for special-scheduler models that either reject distiller scheduler replacement or explicitly allow a compatible wrapper for intermediary validation.
Register find_unused_parameters as an advanced CUDA hardware field so users can opt into DistributedDataParallel unused-parameter detection from generated configs. Add command-line parsing coverage for explicit true, explicit false, and unset values.
Add a CPU bouncing FP8 linear module that keeps frozen float8 weights and per-row scales on CPU while transferring and dequantizing only for the forward/backward matmul. Teach RamTorch replacement to detect scaled FP8 linears, add tests for output and gradient parity, and allow distributed runs to disable shared CPU parameter attachment when that path is unsuitable.
Move trainable parameters to the accelerator before DDP wrapping while marking frozen streamed parameters and buffers to be ignored by DDP. Enable gradient_as_bucket_view when block swapping is active, preserve existing unused-parameter behavior through a shared DDP kwargs path, and add tests for frozen-state ignore handling.
Broadcast sampled batches only within the model-replica group and run batch preparation under a replica-local RNG seed so context-parallel shards share stochastic conditioning. Scale standalone context-parallel losses before DDP averaging, including activation-offload probes, and add runtime tests for replica grouping, synchronized RNG, and FSDP exclusion.
Read SIMPLETUNER_PROCESS_GROUP_TIMEOUT_SECONDS when constructing Accelerate process group kwargs so long-running distributed startup paths can extend the default timeout without changing config files. Validate that the override is positive before passing it to InitProcessGroupKwargs.
Allow distributed runs to place TorchInductor cache files in rank-specific directories by setting SIMPLETUNER_RANK_LOCAL_INDUCTOR_CACHE_ROOT. Add optional faulthandler logging controlled by SIMPLETUNER_FAULTHANDLER_DIR and SIMPLETUNER_FAULTHANDLER_TIMEOUT_SECONDS so hung or stalled runs can emit per-rank stack traces.
Add a Trainer helper that invokes intermediary validation at the restored global step when validation_on_startup is enabled. Call the helper from both training entrypoints after trackers are initialized and cover enabled, disabled, and cleanup-path behavior in tests.
Switch the trained component to eval mode and use accelerator autocast while generating the base-model benchmark, then restore training mode after success or failure. Add tests for normal benchmark execution and exception paths so validation failures cannot leave the model in eval mode unexpectedly.
Apply the existing constant-scheduler learning-rate restoration logic to constant_with_warmup as well so resumed runs keep the configured learning rate after scheduler state load.
Call init_distillation_adapter_modules in the script entrypoint before creating trainable PEFT adapters so distillers can register adapter-side modules in time for LoRA wrapping. Update the train entrypoint cleanup test double with the new initialization method.
Move ComfyUI LoRA import/export conversion behind ModelFoundation methods so model families can provide native conversion behavior without branching inside save_lora_weights. Preserve transformer prefixes through model-level constants, keep Flux2 and SD1/SDXL specialized exporters as overrides, and fail loudly when a LoRA load rejects every denoiser key.
Add schedule-shift overrides, central-difference boundary handling, non-diffusion sigma caps, CP-aware branch partitioning, and per-branch geometry/adaptive-loss logging for AnyFlow training. Move cached negative prompt embeddings onto the training device for real-CFG distillation, keep fused guidance separate from the raw flow target, and add coverage for nested H3 drift, collate unconditional embeddings, FlowMap gates, sidecar delta tables, and scheduler behavior.
Rename the existing Anima AnyFlow example to stage1 forward distillation, add a stage2 on-policy example that resumes from the stage1 LoRA, and keep the shared dataset and prompt library files with each example directory. Expose Anima segmented checkpointing interval and stride methods, allow those modes in safety checks, and update segmented-checkpointing documentation and translations to point at the stage1 example.
Add localized documentation for the EMA copy-through warmup option in the existing options references. Keep the translated option tables aligned with the English EMA warmup entry so the feature branch can be reviewed without missing documentation coverage.
Add experimental MoBA-style 3D sparse attention routing for H3 target-video tokens, with FlexAttention and Ulysses helpers, block-shape validation, padding/layout utilities, benchmark scripts, and distributed verification coverage. Wire sparse-attention config through the H3 model registry, transformer forward path, denoiser lattice metadata, documentation, and translations. Extend H3 training and validation support with native scheduler labels, reference/audio row interval handling, AnyFlow/FlowMap diagnostics, native ComfyUI LoRA import/export support, AdaLN delta table handling, and focused tests.
Let activation-offload prefetch autotune treat a missing context-parallel topology as standalone training before scaling the probe loss. This keeps the real model_predict/backward autotune probe usable in lightweight trainer initialization paths while preserving the context-parallel loss scaling path when topology metadata is present.
Pass a missing trainer seed through to the context-parallel batch RNG synchronizer instead of assuming every lightweight config namespace defines one. This preserves deterministic CP sampling when a seed is configured while keeping trainer test scaffolds and startup paths compatible with standalone defaults.
Use a model-specific flow sigma-to-timestep converter when one is available, and otherwise apply the standard flow-matching convention from the base model foundation. This keeps H3's reversed transformer-time handling intact while preserving custom validation timesteps for generic flow-matching models and trainer test scaffolds.
Update the localized dataloader guides to match the Webshart shard-cache and bucket oversubscription behavior documented in the English guide. Document that shard_cache_gb=0 disables whole-shard caching while preserving indexed range reads, and clarify that automatic oversubscription pads only undersized aspect buckets instead of increasing repeats globally.
Resolve the H3 branch conflicts after the mainline support fixes landed. Keep the shared SDNQ compile configuration, model-card scheduler metadata, runtime cleanup, validation benchmark detection, EMA stepping, FSDP argument handling, and webshart caption filtering updates from main. Preserve the MiniMax H3 conditioning metadata duplication, AnyFlow unconditional text-cache metadata, H3 drift/AnyFlow requirement plumbing, startup validation ordering, and H3 test coverage. Restore StateTracker monkeypatches in TrainingSample tests so quantization tests are not order-dependent.
Bring in the resume LR scheduler fix from main after the latest dependent PR landed. Resolve the trainer resume conflict by preserving parameter-group-specific learning rates for constant schedulers while keeping the H3 startup and distillation flow intact. Adjust the H3 drift wrapper test to compare the composed float32 loss with an explicit tolerance so it remains stable across local and CI math paths.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request adds support and documentation for the "MiniMax H3" model across all supported languages. It updates the compatibility matrices, model lists, and quickstart guides to include MiniMax H3, specifying its license, conditions, and technical details.
MiniMax H3 Model Support
READMEfiles, including license details and usage conditions. [1] [2] [3] [4] [5] [6]Compatibility and Feature Matrix Updates
These changes ensure that users are informed about the availability, requirements, and technical specifics of MiniMax H3 in all documentation and compatibility resources.