Over the course of an entire game, I usually don't have much of an issue with random variance of stat gains. However, I usually don't have the time to sink to repeatedly play an FE game over and over and over again, so when I do play through one, I usually have the group of units I want to use set in my mind, and random variance can royally screw with that plan.
Also, the randomness of growth rates during early portions of the game can be very frustrating and can further sabotage plans and progress - consider FE9 earlygame if you're not roflstomping the game with Titania: if, say, Oscar gets shit-all for Strength and Speed early on, he goes from a unit generally perceived as great to one that's, at that moment, mediocre if not a liability.
And, of course, as a player, don't you instinctively want to see the units that you're fielding reach their potential? I know that I always get a "bleh" feeling when a unit begins to drop off and noticeably lags behind the rest of the group as the game reaches its finale.
While it has other flaws, I almost like Valkyria Chronicles' way of levelling up classes instead of individuals so that you can rotate units without an immediate drop in effectiveness. Of course, for FE, this really wouldn't make sense, since you have characters like your Jagen, Est, etc. that for story reasons have very different levels of experience in combat, but you get less deviation and (hopefully) the option to use the characters that you might want to use but could have to push to the side because of how the RNG rolled.