I'd just like to accentuate the point that others have already made. When a manager says this, what he usually means is, "You didn't give my team a foul every time I wanted you to." In general, you can safely ignore such comments.
I largely agree, but it is not always as simple as wanting calls for my team.
As referees we have to read a game and find where the foul bar needs to be for a fair game. Different players and teams have different tolerance for contact. When two teams have similar levels, drawing the line is relatively easy. When one team wants to play through everything and the other wants every little thing called, well, that's when we really earn our keep in finding a line that can make the game work.
Going back to the OP, you'll get a lot more valuable feedback about your line drawing from the run of play than from coaches or managers after a game. Take anything you hear after the game (both good and bad) with a hefty dose of salt. But if during the game you see players getting regular frustrated with non calls on contact, that can be a clue that you have set the foul bar too high for
this game. And if both the fouler and the victim seem surprised by foul calls, that suggests your foul bar is too low for
this game. Don't fall into the delusion that a foul is a foul is a foul--there are 100% fouls and 100% fair plays and a bunch of space in between where our greatest skill is figuring out which plays--in today's game--should be called as fouls and which should be passed on as trifling.
The only way to get good at this is experience out on the field. But you can get a little bit of help by watching experienced referees and seeing what they are calling and not calling, and thinking through why they are making the choices they are.