What should we do about new users when they break the rules? What shouldn't we do?
Communicating with new users can be troublesome, first of all because they don't know what ways are available to us for that purpose. They comment in answers, they include parts of the answers in their questions, etc. There are also some more intentional reasons that make some users troublesome but luckily they're not all like that. In any case, I'll try to cover all cases. This is more intended as a guideline, rather than hard rules, but I suggest we all try to follow them.
First of all, read: Stack Exchange is not a forum: the role of “niceness” on a Q&A site.
Be nice and friendly
Always, always be nice. Well this is actually a rule in the FAQ so, you might say "hey there's nothing new there!". But being nice (or rather, nicer) to new users is even more important. We don't really want to scare them away, do we?
But how can we be friendly? Well, you don't need acrobatic wording, but a "Hello xxx and welcome to our site!" is the best way to start. It's not enough, but it's a start.
Never attack the user
By this, I mean never say things like "What is this crappy question?" et similia. It's not going to help. Even the most well-behaved person will feel bothered by that. Use the Golden Rule that is valid in all life situations:
One should treat others as one would like others to treat oneself.
Other than being extenuating, communicating with new users can be frustrating if they refuse to cooperate. Be patient with them. This doesn't mean "let them walk over you". Don't be submissive, but certainly don't attack back: that's not going to help. Ignore personal attacks (i.e. don't answer, but flag them) and always answer with facts.
A: You are all against me, this site sucks!
B: No, look, we are not targeting you on purpose. The FAQ says that...
If they're serious users, they will apologize and start cooperating. If they're trolls (that is to be decided case by case, there rarely is a clear-cut situation), they will end up being suspended in order to let the community live in peace. Don't feed the trolls.
Has the matter being treated in the FAQ? Did we talk about it in a Meta question? Link them to it.
Suggest fixes or edit it yourself
Voting to close is your right and also your duty. Moderators, when applicable, leave the new user time to fix the question before closing it. In some high-visited sites this is technically impossible, but in small sites, it's possible.
As high-rep users, you can vote to close, but don't hesitate to leave a comment suggesting the OP fixes to improve the question and make it fit. Close votes expire after a while, so even if you voted to close and the OP fixed the question, all is well.
If the question can be easily fixed by you, do it. You have the privilege to edit for a reason. Even if low rep, do it, someone else will revise your edit and approve it if it actually improves the question.
If their answer is short, ask for some explanation
Asking "Please make it longer or we are going to delete it" is kind of intimidating and might put off people, but it's the truth. So there are two ways to do this. You could use a pro-forma comment (this is mine):
Welcome to (site)! Usually we prefer longer and more elaborated answers on short answers. If you can improve your answer by adding detail, context, examples, and backing up with references, this would increase your answer's quality. Poor and/or repetitive answers risk being down-voted and subsequently removed.
It says the same thing, but look at the wording! :D Another way is ask for an explanation on something. Like
Welcome to (site)! You didn't treat enough about this point and it looked interesting, mind to expand on it?
Or similar wordings. But you get the point.