My advice is give more free kicks, if remotely looks a foul give it, tendency as ex players is to let the game flow but losing match control is really easy.
A better scenario is ‘come on ref that was soft’ than an escalation to players confronting each other, you can then let the game flow later,