(Long ramble to follow, apologies, but I get rather annoyed at how lackluster magic is handled in official FE…)
One of magic’s main problems is that, for the most part, it doesn’t do anything special. Sure, Dark magic often will have secondary effects beyond damage, and you’ll occasionally get other magic that also has it, but a good chunk of what has made up the core of magic’s existence in Fire Emblem is just weapons with differing stats or something relatively benign like effective damage versus some group of units.
For something so mystical that only certain types of units or certain characters can use, it sure doesn’t feel all that mystical or involved outside of the lore. I know people love the simplicity of Fire Emblem (and I’m not entirely saying to go crazy complex, just give me a moment…), but when regular weapons have more interesting and varied properties to them, I think you’ve got a problem. Even something as simple as giving, say, Elfire an innate Seal Defense skill would be an improvement to what the developers have routinely put out!
I don’t particularly agree with the homogenized Tome ranks that the series has shifted towards, especially when they haven’t really made the tomes do anything as part of it. Like, if you want to homogenize tome ranks together to cut out the bloat (which Awakening probably should have done), that’s one thing, but at least make the smaller selection have more variety amongst what they can do. Sure, it’s nice that a magic user can potentially use everything and not be way too overly specialized (see Tellius especially where weapon ranks are a pain to gain, leaving the mages often stuck to their own element or weak magic from another), but is having 18 tomes (that still don’t do much other than changing stats) that anyone can use better than having 30 tomes, only 6 of which each type can use?
I like the concept of units having their own lists of magic that they can learn, though I would like it better if it was used in addition to having standard tomes along side them. I’ll give Shadows of Valentia/Gaiden a pass, since there are lore reasons for not having tomes, but I personally think that the Learned Magic system should be something that any unit can use in any class with a restriction on the number of uses per map (and, if you really want to do something like what Three Houses did with limiting it, just either reduce the number of uses if you’re not in a magic class or give them low uses to begin with and have magic classes multiply the number of uses), and by having Tomes as standard weapons, it lets the unit use a spell that they might have learned early because of their Level Up list that might be higher rank than what they could use in Tome form or to allow them to have extra tools and not have to carry a specific weapon in their actual inventory (edit - especially if you would only be bringing it to handle a small number of specific threats or targets across the entire map).
For Staves, I know Warp and such can be fun to use (see Thracia), but if they’re going to be included then the games need to be designed where they can’t just be (ab)used to render entire chapters pointless and able to be cleared in one turn with them, or just remove them. (Personally, I opt to permit Rewarp but not Warp since the mage is potentially putting themself into danger. Alternatively, inflict a debuff on the unit Warped/Rescued or prevent them from moving (if they haven’t yet) after the staff is used on them (including preventing being refreshed).) Outside of that and egregious things like the Hexing Rod, etc. that are just there to troll the player, I think Staves are relatively well balanced since their entire point is to be used and do an effect when that happens.
As for balancing and comparing to/vs other weapon types, variety is key. Sure, you could just make an equivalent Killer/Brave/Reaver/etc. tome that is comparable to what physical weapon types get, but that’s honestly pretty boring. Consistent, yes, but still boring. Give magic a variety of ranges (1-only, 2-only, 1-2, 3-5, 2-4, 5-8, 3-15, etc.) and adjust the power and effects of what those tomes do versus their range. Give a wind tome 3-15 range but make it weak or less accurate to compensate. Make Nosferatu locked to Range 2 to prevent tanking with it a la Awakening instead of virtually everything else that IS has tried in order to balance it.
The magic triangle is mostly pointless for its WTA/WTD bonuses, but I do personally like having different schools of magic and having them do different things. Even the typical Power/Accuracy/Crit trends are fine as general stylings that are an overarching theme across all of the tomes of the type, just as long as the tomes (usually) do something beyond their damage. Split promotions will always vary even for non-mages in what the options are and what they can do. Staves are really only so powerful in promoted Sage/Druid/etc. hands because they usually have higher Magic stats or come in with really good Staff ranks that make them so good. If you adjust the staff formulas to use, for example, Res as a driving factor or you nerf the amount the staves heal by themselves and require Healtouch for more normal healing amounts (and lock it to Cleric-type classes) or just don’t give a promoted character that joins trivial access to the entire list of Staves by giving them basically the highest rank they would ever need and invalidating other units in the process, then I theorize that other things might fall into place along the way.
In addition to the rest, I think there’s room for more wide-reaching magic effects, i.e. AoE magic. Give them limited uses and make them work like Staves where they don’t equip (see the Lex Talionis engine’s Spells) and give them other interesting things that, again, aren’t just being straight damage amounts over a wide area. (For example, Lex Talionis has a Dark Spell that targets 1 enemy from a long range and reduces their stats if it hits and has another spell that makes a target act like they’re a flying unit for a time - while these would likely be staff effects in a traditional game, you could easily slap damage onto the first spell or give staff-like effects to other weapon types and restrict more powerful staff effects (i.e. Warp) to traditionally healing classes based on what weapon types you assign to classes.)