Expert capacity
In a mixture-of-experts layer, the most tokens one expert may take in a batch. It keeps the work balanced between experts; tokens sent to a full expert skip the layer.
Introduced by
GShard, 2020
In a mixture-of-experts layer, the most tokens one expert may take in a batch. It keeps the work balanced between experts; tokens sent to a full expert skip the layer.
GShard, 2020