It all comes down to whether you find the comments offensive, etc. You might think " How much you being paid?" insulting but someone else might brush it off. If you are insulted, send em off. No doubt you will be criticised but each persons threshold is different. If you think a comment is insulting but a soft send off (e.g. what colour shirt you wearing ref?) it might be an idea to call the captain over, tell him what you think and that the next comment could result in a note direct word from you to the player. If you get another soft but insulting comment call the offender and captain over, explain to the player that you found the comment insulting and send him away. Remind the captain that you earned him and it is up to him to get a firmer grip of his players. Some comments may seem soft but are still insulting and still undermine your authority.