I completely disagree with the overall idea that if you didn't hear it, you can't send him off. In a case as heated and emotional as this, when you have a player saying "yeah I said that" and you can be reasonably sure that he actually did, then you send him off.