Mixture of experts

A network made of several smaller networks, called experts, plus a gating network that decides how much each expert contributes for a given input. Different experts can specialise in different kinds of input.