In the UK, the basis of 'there's no right to privacy in a public place' tends to hold true in most situations of appropriate behaviour by someone with a camera.
As a pro photographer, indeed, as an amateur photographer, I would have every right to photograph whatever I saw whilst in a public place. As long as I wasn't going to 'extraordinary lengths' to photograph it - such as climbing a tree, or hiding in a bush.
Neither do the police, or any other 'authority' have the right to delete, or ask you to delete, any images legally captured as they are your property. Police can confiscate a memory card (or roll of film to show my age) to be held as evidence, however a full receipt of the item along with it's contents would be required.
I have gone toe-to-toe with a couple of officers over this on more than one occasion ;-)
If you are in a public place, and are being filmed or photographed there is not much you can do about it. If your child is being filmed or photographed whilst in a public place there is not much you can do about it either. If you intend to do something you don't want to be seen, don't do it in a public place - common sense eh?