Imo it’s better to have “too much” units instead of too less. A big cast gives you two things in particular.
First of all, you have more chances if you’re playing Permadeath which is an important thing for me since I love playing with it. However a huge number of character doesn’t automatically mean that it’s suitable for Ironmans and such (looking at you, Radiant Dawn).
The second big thing about it is that while we’re all having our favs and characters that we bring with us every time there are always other characters to take with you for another run. Especially in combination with Permadeath you never know how your final cast could end up. Which is always a cool thing to me. And those who want to reset and try to bring everyone to the goal can also profit from it that way.
Also it is still possible to make every character memorable even though they have just a few lines of dialog when being recruited… Or in their Supports. Generally the latter is perfectly suited for fleshing out their character. The Main Cast needs to be present and grow in the story all the time. The sidecharacters can have their own stories via supports and talk events. Maybe add in some secrets.
The worry that a character will never be used is not nessecary. Look at characters like Marty, Arden, Lot, Cecilia, Isadora, etc. etc. Some consider two of them good, some bad. Two of them are VERY bad statistically speaking… Yet, those units all have fans. And it’s the same with every unit out there. Everyone has someone out there who likes to use them in battle and that comes down to multiple things. Some like their stats, some their looks, some their class… But in the end everyone is a part of your game and will be used eventually. It’s the same with secrets. Someone will find them - It’s just a matter of time. And sometimes characters you don’t expect to be favorites turn out to be the heroes of your players.
Also while not every unit will or can be present in the main story - The story they write ingame will be the one that you will see. Maybe they grow pretty good. Maybe they become the heroes of your party. Maybe they die a tragical death. Or while saving multiple of your other units. Even if a unit has 0 dialog, if it is a playable character it has the chance to write all these things with actions. And while I love a mixture of both gameplay and story - The gameplay is unskipable and the thing that you create while playing.
As Mr Green said in his comment, the con of a large cast is that it can be harder to remember specific characters. And of course if a cast is huuuuge you can’t use every single character. But then there is the question, are characters like Forsyth, Zeke, Nomah, Ewan, Marisa, Saleh etc. more memorable just because their cast is smaller? Is it possible to use everyone there just because the cast is smaller? I mean here you even are encouraged to use the same units multiple times due to split promotions. Nah, I think even a big cast can be memorable if you try it. This negative thing can be fixed by adding talk events, more dialog, supports, etc. And it isn’t too bad compared to the missing replacements you have in a small cast which in the end harm the gameplay - which is more fatal imo. The easiest fix there would be to disable permadeath or adding things like revival options (Valkyre/Ohm/Bifröst Staff, Springs from SoV, etc.) and Generic Replacement Units. But especially the first would be a thing that I’d miss in a Fire Emblem.