Uh oh, this one again?
First off, for those referees who say 'nah, never ever ever any QFK around the penalty area, it's just too hard for my match control hey'. Go and throw your badge in the bin and take up knitting if doing the job right is too hard for you. While there's room in some events to make the decision that's 'easier', if you don't have the courage to make the decision that's correct but more difficult then you shouldn't be out there.
Why do we have free kicks? It's (largely) to restore an advantage lost by a foul. If you're going to deny the QFK for no reason other than it's too hard for YOU the referee then that's just an abuse of power and a lack of understanding of the laws and spirit of the game. You're taking away the advantage of the kick and instead using the foul to give the advantage to the defence. Which is just daft.
The defence didn't have 9-10 players behind the ball in a fully organised and structured defence before the foul - why go out of your way to ensure they have that now to the detriment of the attack? A FK is about the attacker, not the defender.
So, when to allow it?
First off, in the scenario presented in the OP where he had to blow the whistle to take it back, I think it's justified to say that the referee has now intervened so it's ceremonial. They tried to take the QFK, did it wrong, they've lost that.
A QFK is a right unless the referee needs to intervene for some reason (injury, card, sub, other issue).
If you, as the referee, have started engaging in conversation with players, then I think that constitutes you intervening. Having said that, it always brings me great joy when the goalkeeper runs 20 yards to argue with the referee and the attacking team says 'sweet, I'll take it now then!'. Perfectly fine - keeper's own fault, and if you're just making quick comments to shut it down it's okay. Start having a conversation? You're now intervening because YOU are now holding the player out of position.
If you award a FK and you happen to be very close to it, ask them what they want then move away if they want it quick.
There's some disagreement about what happens if the referee starts to manage the wall. IMO you can still remind players '10 yards!' without it being intervening, but once you're coaching them back 'keep going, keep going, another step', I'd suggest that's the point of no return for losing the QFK - and if the attack are standing around while you're doing that, then they didn't want it quick (look at it this way - they only wanted the kick AFTER you started to intervene. Nuh-uh, you don't get it both ways. On the whistle!)
Aside from that, you can use your judgement. If the attacking team are milling around and not looking to be doing anything, then you can operate on the assumption of it being ceremonial (and same if the ball needs to be fetched from the other end of the field or the creek). Aside from that? QFK is a right. That right doesn't disappear because it's in the attacking third - heck, that's when it's the most important to protect that right.