I would like a two-slot system, one for actual armor, the other purely cosmetic. This is something I remember fondly from my days in DDO.
Then, using "Farmer" as an example, we would gain the Farmer perks by simply collecting the complete Farmer outfit. The reward would be added to our stats, much like what happens currently when completing a book set.
Those collected pieces could then be applied to the cosmetic slots if we wanted to sport a certain look for MP or role-playing, but we wouldn't be required to wear them or switch sets to benefit from the associated perks.
For those worried about masking armor strength in PvP, simply add an option in server settings for the cosmetic slots. That would be similar to the Persistent Profile thing we have now, where servers have some control over how a player's appearance changes.
Best of both worlds?
Just spit-balling here. I am sure there are things I have not considered, but I want to nip in the bud the "not worth dev's time" argument. The UI needs some love anyway, and any change to the armor system is already likely to involve some UI changes, so why not something like this?