Quiz · Mixture of experts1 / 5
RightSignalThu, Oct 22 · today
1 of 6
Quiz · Mixture of experts2 / 5
Two chosen experts have router weights 0.6 and 0.4 and outputs 5 and 10. What is the layer's output?
Quiz · Mixture of experts3 / 5
Why does a mixture-of-experts model still need a lot of GPU memory?
Quiz · Mixture of experts4 / 5
Why do mixture-of-experts models add a load-balancing penalty during training?
Quiz · Mixture of experts5 / 5
What decides which experts handle a given token?
You got 0 of 5.
5 left unanswered. Swipe back to finish them.
SwipeScroll or use arrow keys