How to add custom models in NeMo?

[gagangayari]

I wanted to play around with different model components. Eg Plugging in MLA instead of MHA. Or change the forward method somewhat. I guess we can specify it via MCoreSpec. Can somebody please help me with the complete flow on how to add such a customized model and use it for training. If there is any recipe that would be a great help. Thanks

[ashors1]

Hi, thanks for your question! Many adjustments to the architecture can be made by tweaking the model's GPTConfig. Using MLA requires swapping out MCore's TransformerConfig for the MLATransformerConfig. An example of a config that uses MLA in NeMo can be found here.