Nvidia is building a trillion-parameter open model, and it would still be smaller than China’s
Nvidia released Nemotron 3.5 Lightning on Tuesday. It is a 30 billion parameter mixture-of-experts model with three billion active at any moment. The architecture is a hybrid of Mamba-2, MoE and attention layers, with a one million token context window. The weights are on Hugging Face and…
No discussion yet. Be the first to share your thoughts!