Just kick them out of the server, or mute them, or block them.