Different levels, different "expectations" (I say that as my personal opinion). L7, doing Sunday morning kids etc., you just wear your kit and turn up shortly before the game and ref it. As you move up, while there's no need to have a specific badged top, you'll generally find the need to warm up (usually while doing a pitch inspection) and having a training zip-jacket or sweatshirt just looks a bit better and keeps everything clean and tidy.
As you move up the levels, so say you are doing Step 7 football next season, you'll have the chance to get changed, do a warm up, then come back in to your changing room and get into your ref kit, so having a training kit makes sense. Some people, myself included, like to have all the gear and no/little idea and I generally feel I am more professional looking warming up in my badged referee kit and trackie bottoms, but it's not compulsory.
As an L4 this coming season, I would prefer my assistants to have some training kit when we do the warm-up, but I won't make a fuss if they don't as long as when we officiate the match all of us look clean and presentable.