If you genuinely believe that he has deliberately moved his arm toward the ball (post 15) then of course this should be given. If, however, you believe what you say in your first sentence above, then you absolutely can’t give this as handball in accordance with the laws. It is a fundamental principle of the law, as it’s currently written, that the ONLY advantage you cannot get from an accidental handball is for that player to immediately score a goal.
The fact that many players / commentators etc believe it’s unfair to gain ANY advantage from the ball accidentally striking the arm is irrelevant