MoE Communicatie Overlap

MoE expert-parallel communicatieoverlap in Megatron Bridge. Behandelt dispatch/combine overlap, flex dispatcher backends, en expert wgrad scheduling.

Bron: github.com/NVIDIA/skills/tree/HEAD/skills/nemo-mbridge-perf-moe-comm-overlap · Licentie: Other