another option is IP Trace they know from reports where player account originate so at that point do a total ban on all address stemming from those dns address ..... and even if they try an ip mask it wont work to hide behind

A decent filtering system will work just fine. Yes it's true Bad-boy only stopped spam you saw, but that is the point. If the players server wide stop seeing the spam, then the advertising still stops.

Hell, just steal a dev from google, have them design some learning chat filter that you can then sell to other companies. A filter that learns based on several factors, some can offer fast short term relief for players and others long term curbing of spammers in general.
Examples would be:

• When a spammer says something in chat, and dozens of players all click an easy to reach "Report Spammer" button within several seconds the system is flagged that a potential spammer is detected. It can then block that specific message immediately, so it can no longer be seen in chat.
• For a longer term solution, the system all keeps a database of all these spam messages and begins to learn what combinations of symbols, spaces are used to make words, and it can then begin to decipher whole words on it's own, and automatically block a website even if they change how it is presented(through symbols and spaces etc).
• Much of this can be programmed in at the beginning, as already it is safe to assume no one is spelling any words using a ton of symbols and spaces and random gibberish.
• If you want to get really inventive, it can also find suspicious character names (like gyuhdgyhn) that get created by checking for certain combinations of letters that generally do not fit together (more then three or four consonants in a row tend to be suspicious). These players will then be required a few more steps of verification to create a character like this, slowing down the bot creation of characters used to just spam for a few minutes and deleted.

Supplement the time and costs to create this by licensing out the system to other developers. Make it cheap enough that it will not be an unnecessary burden on them of course, the goal is to spread it to as many games as possible.