Really not have to think about it much
. There has been occasions when I have called this and there has been occasions when I have not. You need to asses the 'benefit' case by case. There are other factors too. Like the temperature of the game and if you want to slow it down.
The common one i do stop is when the ball is kicked out/cleared near the attacking corners and off the fence ends up 30 yards out. Now all attackers and defenders were in the PA waiting for a cross before it went out. If the are taking it quick from 30 yards out to an unmarked team-mate to whip it back in, that is a clear benefit and I am not allowing it. There are also times that they have done it because it is easier to find an unmarked team mate without too much overall impact to the game. No point being pedantic there.
Talking about other factors, a good tip I had from a national league referee, If you are refereeing two new teams and you want to establish authority, use this very early in the game (and only very early) to do that. You are highly unlikely to get any dissent or disagreements because it is very early and trivial. But you have established that even if 'you as player think' it is not important/right, I ask for something, you just do it and get on with it.