Well, here's what the law says:
There are different circumstances when a player must be cautioned for unsporting behaviour, e.g. if a player:
• verbally distracts an opponent during play or at a restart
I don't know about you but I don't see anything there about intent. It's the same principle as a careless tackle that ends up tripping an opponent. We're not supposed to judge whether the player intended to trip the opponent, only if the opponent was actually tripped.
Now having said that, I would make a couple of other points. In my experience this is a very rare offence and I never have actually cautioned a player for doing this. In many cases where it might be a possible consideration, I reckon it could very easily fall into the category of a "doubtful or trivial" offence and so I would be inclined to warn a player first and only if they continued with the same behaviour after that, think about moving on to a caution.
I am reminded of one occasion when I did warn a player for what was a possible case of unintentional distraction. It was a girls U16 game and there was one player who had a habit of giving a little shriek every time she went in for what looked like it might be a tough challenge. It seemed to be almost involuntary on her part and I couldn't say absolutely 100% for sure that it was distracting her opponents but it looked to me like it might cause some of them to hesitate slightly. I warned her about it and she stopped doing it. Had she continued to do it after being warned and if at that point I judged that it actually had distracted an opponent, I would eventually have cautioned her, even though as I said, she seemed to be doing it as more of a nervous reaction and not with an intent to distract.
So yes, I would certainly consider cautioning a player for verbally distracting, judged according to its effect on the opponent alone, with intent not entering into the equation.