Exactly this. My vision is arguably marginally worse with contacts than with glasses, particularly under floodlights or if I've had to wear them all day ahead of a match - but there's not a chance I'd turn up to a match and try to defend wearing glasses.And why should referees follow the same rules? Are you taking off your watch, too? I think this is a grossly overused mantra.
While I agree it is better for refs who need vision help to wear contacts (which is what you see at the top levels--usually the same with players, too), I think it is absurd to say no one can ref if they need to wear spectacles (not everyone can use contacts). While it makes sense to say it is better for players to wear sports goggles if they need them to see, the same doesn't apply to referees who are not engaged in a contact sport while they officiate and are not using their head on the ball. The referee choice on spectacles needs to be based on the role--the glasses that provide the best ability to see.
(Personally, I need glasses. My eyes cannot tolerate contacts on a full time basis. But I almost always wear my contacts to officiate. In high pollen season, I'm not always able to wear them.)
Only once have I even worn them to walk in to the changing rooms. I put my contacts in while getting changed and of course immediately as soon as the match starts, the manager is claiming I can't see because I was wearing glasses earlier!