You make a good point, and one which I constantly preach but usually, it falls on deaf ears until the referee is experienced enough to absorb the meaning of my ramblings. Refereeing is not about giving a free kick, or not giving a free kick
Its about managing the game as a whole. Its entirely possible to have a great game without detecting every instance of foul play. When we are all new, we are too eager to spot every foul, thinking it makes us good. Its not. Its about knowing what fouls to penalise, when to do it, where to do it etc
Brych is the best referee around ( that I can see on tv anyway), at the moment