FAQ FAQ   Search Search   Memberlist Memberlist   Usergroups Usergroups   Register Register 
PhpBB- & WordPress- antispam.com
Home of the Antispam for all fields mod ! 
  Profile Profile   Log in to check your private messages Log in to check your private messages  Log inLog in 

 PhpBB- & WordPress- antispam.com Forum Index » [PhpBB2] Feature requests » highlighting of forbidden words
 Goto page 1, 2  Next The time now is Tue Oct 17, 2017 11:15 pm 

 Post new topic   Reply to topic View previous topic :: View next topic  
Author Message
PostPosted: Thu Dec 14, 2006 10:29 pm    Post subject: highlighting of forbidden words  Reply with quote  

Lord Raiden
Experienced user


Joined: 11 Dec 2006
Posts: 123

Not sure how good an idea this is, but it *is* a suggestion, and not nessisarily a feature request. The idea is to have in the admincp under the spam listings below the info you already show, a summary of what words hit and put it in a tabled form something like this:

Text Rule Triggered
----------------------------------------------------------------------------------------------
.....and the Carolina Panthers ran the...... | *panthers*

I'd still keep the option you already to show the entire message, but maybe also slice out the message into another table that shows each area that hit and what rule it triggered. I'd use a truncating rule that removes anything beyond 30 charecters to either side of the word and adds tripple dots to indicate the truncation. I suggest this because not all of the items trapped have obvious clues why it got caught. I thought this would be a nice addition that would add some clarity to that.

TOP View user's profileSend private message  BOTTOM
PostPosted: Tue Dec 19, 2006 7:00 pm    Post subject:  Reply with quote  

WebSnail
Phpbb coder


Joined: 14 Dec 2006
Posts: 38

I second this request..

I've had a couple of posts recently fall foul of the *rdto* rule but I'm damned if I can find where it was used..

Quote:
Hello I started at St Mary''''s hospice last week and have been told I''''m responsible for the Sunflower Scramble - I''''ve never organised anything like this before so basically need all the help I can get! It would be much appreciated!! I''''d love to beat Emo''''s total from this year. I''''m going to be at the conference in Leeds - looking forward to it, should be a good excuse to return to my student days Wink Charlie


I'm pretty sure this is due to the 733t word translator side of things but can't be sure so in that respect a highlight system would make life a LOT easier, even if it's in the test system only..

TOP View user's profileSend private messageVisit poster's website  BOTTOM
PostPosted: Wed Dec 20, 2006 3:55 pm    Post subject:  Reply with quote  

Top_Speed
Experienced user


Joined: 19 Dec 2006
Posts: 1
Location: Geneva, NY

WebSnail wrote:
I second this request..

I've had a couple of posts recently fall foul of the *rdto* rule but I'm damned if I can find where it was used..

Quote:
Hello I started at St Mary''''s hospice last week and have been told I''''m responsible for the Sunflower Scramble - I''''ve never organised anything like this before so basically need all the help I can get! It would be much appreciated!! I''''d love to beat Emo''''s total from this year. I''''m going to be at the conference in Leeds - looking forward to it, should be a good excuse to return to my student days Wink Charlie


I'm pretty sure this is due to the 733t word translator side of things but can't be sure so in that respect a highlight system would make life a LOT easier, even if it's in the test system only..


There is a bug... If your user typed the 2 words "Forward to" the word filter isn't taking into consideration the space between the words.

I have notified Ramon of this and I'm sure it will be fixed in the next release.
_________________
Kill the Spamalator B-tards!

TOP View user's profileSend private messageVisit poster's website  BOTTOM
PostPosted: Tue Feb 06, 2007 7:10 pm    Post subject:  Reply with quote  

ramon fincken
Site Admin


Joined: 11 Dec 2006
Posts: 269
Location: A'dam/Diemen, The Netherlands

1)

I implemented storage of the word_ids in this version.

2)
Quote:
There is a bug... If your user typed the 2 words "Forward to" the word filter isn't taking into consideration the space between the words.

Thats not a bug ! its a feature !! Razz
_________________
Phpbbantispam founder, available for freelance WordPress coding.

PhpBB2 mod: http://www.phpbbantispam.com/viewtopic.php?t=1
WordPress plugin: http://wordpress.org/extend/plugins/antispam-for-all-fields/
Project files & mailinglist: https://sourceforge.net/projects/phpbbantispam/

TOP View user's profileSend private messageVisit poster's website  BOTTOM
PostPosted: Wed Feb 07, 2007 1:35 am    Post subject:  Reply with quote  

Lord Raiden
Experienced user


Joined: 11 Dec 2006
Posts: 123

Hmm, I wonder if there's a way to "unleet" the code without getting false positives on legitimate word combinations and still catching spam.

TOP View user's profileSend private message  BOTTOM
PostPosted: Fri May 11, 2007 2:37 pm    Post subject:  Reply with quote  

WebSnail
Phpbb coder


Joined: 14 Dec 2006
Posts: 38

Lord Raiden wrote:
Hmm, I wonder if there's a way to "unleet" the code without getting false positives on legitimate word combinations and still catching spam.

Shouldn't be a problem.. I think it'd actually be useful to put in options as to which tests you want to run or not..


Anyway, the feature originally requested is fast becoming vital as I'm constantly hitting triggered posts that I can't for the life of me work out what got it flagged as spam.

I think I may start looking at this seriously myself soon.


I'm also getting rather desperate for a trigger score to be added as well now.. something like "Number of triggers required to ID as spam: 1, 2, 3, etc..."

I'm reaching the point where most false positives only have one or two trigger words while the spam has multiple triggers... it'd certainly help to reduce the problems we've been having.

TOP View user's profileSend private messageVisit poster's website  BOTTOM
PostPosted: Fri May 11, 2007 6:17 pm    Post subject:  Reply with quote  

Lord Raiden
Experienced user


Joined: 11 Dec 2006
Posts: 123

Yeah, that sounds like a good idea. I've been using Spam Assassin for years and they've been using spam scores for quite a while. So I think it'd be a good idea to try with this mod as well. It would definitely allow me to add more words to the spam list without worrying about false positives as much.

TOP View user's profileSend private message  BOTTOM
PostPosted: Fri May 25, 2007 4:15 pm    Post subject:  Reply with quote  

WebSnail
Phpbb coder


Joined: 14 Dec 2006
Posts: 38

Ok... I managed to get the original request sorted myself...

Because the current code has 5 different tests it runs I've gone for the easy option.

Basically what my mod does is provide output from each of the 5 tests in the test center. Put simply it highlights the output in red where the spam term was/is found.

This basically allows you to isolate where the spam term occured and why at least it might have triggered.


Open: admin_antispam_testcenter.php
Find:
Code:
      // Spamwords testing report phase
      $test_result_antispam = validate_spam_testcenter($test_value);
      if(strlen($test_result_antispam) > 0)
      {
         $test_result = '<font color="red"><b>»BANNED</b /></font>';
         $test_result .= $test_result_antispam;
      }
      else
      {
         $test_result = '»Pass';
      }
      $test = 'Test type : <b>'.$test_type.', antispam wordlist</b />';
      $test .= '<br />Test value : <b>'.$test_value.'</b />';
      write_block('TestResultSet',$test,$test_result);
            unset($input);


Replace with:
Code:
//MOD'd to tag the spamwords
      // Spamwords testing report phase
 
      define('SPAM_TAG_OPEN', "[!!<]");
      define('SPAM_TAG_CLOSE', "[>!!]");
     
      $test_result_antispam = validate_spam_tc_tagged($test_value);
     
      if(strlen($test_result_antispam['banned_words']) > 0)
      {
         $test_result = '<font color="red"><b>»BANNED</b /></font>';
         $test_result .= $test_result_antispam['banned_words'];
      }
      else
      {
         $test_result = '»Pass';
      }
      for ($i=1; $i <= 5; $i++) {

            $test_txt = preg_replace("#".preg_quote(SPAM_TAG_OPEN)."#", "<font color='red'>", $test_result_antispam[$i]);
            $test_txt = preg_replace("#".preg_quote(SPAM_TAG_CLOSE)."#", "</font>", $test_txt);
         
         $test = 'Test type : <b>'.$test_type.'['.$i.'], antispam wordlist</b />';
         $test .= '<br />Test value : <b>'. $test_txt.'</b />';
         write_block('TestResultSet',$test,$test_result);
            unset($input);
      }
//MOD END tag spamwords



Open: /admin/antispam_functions/admin_antispam_functions.php

Find:
Code:
// -------------------------------------------------------------------------------------
// FUNCTIONS
?>


Before, Add:
Code:
//MOD Tagged testcenter function
function validate_spam_tc_tagged($txt)
{
   global $db;

    $test = array();

   // 1st round
   $test[1]['txt'] = change_txt($txt, 1);
   // 2nd round
   $test[2]['txt'] = change_txt($txt, 2);
   // 3rd round
   $test[3]['txt'] = change_txt($test[2]['txt'], 3);
   // 4rd round
   $test[4]['txt'] = change_txt($test[3]['txt'], 4);
   // 5rd round
   $test[5]['txt'] = change_txt($test[4]['txt'], 5);



   $sql = "SELECT word
                FROM  " . ANTISPAMWORDLIST_TABLE;
   $banned_words = '';
   if ($result = $db->sql_query($sql))
   {
      if ($row = $db->sql_fetchrow($result))
      {
         do
         {
               for ($j=1; $j<=5; $j++) {
                 $test_tmp = word_has_spamstatus_tag($row['word'],$test[$j]['txt']);
                 $test[$j] = $test_tmp;
                  }
            if ( $test[1]['spam'] || $test[2]['spam'] || $test[3]['spam'] || $test[4]['spam'] || $test[5]['spam'])
            {
                  $banned_words .= (strlen($banned_words) > 0) ? ', '.$row[word] : '<br />Found words: '.$row[word];
            }

         }
         while ($row = $db->sql_fetchrow($result));
      }
   }
   $db->sql_freeresult($result);
   
   $text_test['banned_words'] = $banned_words;
   $text_test[1] = $test[1]['txt'];
   $text_test[2] = $test[2]['txt'];
   $text_test[3] = $test[3]['txt'];
   $text_test[4] = $test[4]['txt'];
   $text_test[5] = $test[5]['txt'];
   
   return $text_test;
}

// One of the core functions ..
function word_has_spamstatus_tag($word,$txt)
{
   // 1.1.2
   $test_return['spam'] = false;
   $test_return['txt'] = $txt;
   if(preg_match("#\b(" . str_replace("\*", ".*?", preg_quote($word, '#')) . ")\b#i", $txt))
   {
         $search = "#(" . str_replace("\*", "", preg_quote($word, '#')) . ")#i";
         $test_return['txt'] = preg_replace( $search ,(SPAM_TAG_OPEN."$1".SPAM_TAG_CLOSE), $txt);
         
         $test_return['spam'] = true;
   }
   return $test_return;
}



I thinlk that's most of it and as you can see I've created seperate functions based on originals rather than editing them..

TOP View user's profileSend private messageVisit poster's website  BOTTOM
PostPosted: Fri May 25, 2007 4:29 pm    Post subject:  Reply with quote  

WebSnail
Phpbb coder


Joined: 14 Dec 2006
Posts: 38

Forgot the template file:

templates/subSilver/admin/admin_testcenter_tag_body.tpl

Code:
<table class="forumline" width="100%" cellspacing="1" cellpadding="4" border="0">
   <tr>
      <th class="thHead" height="25" valign="middle">Antispam {MODE}</th>
   </tr>
   <tr>
      <td class="row1" align="center">
      <table width="100%" cellpadding="4" cellspacing="1" border="0" class="forumline" align="center" id="table1">
         <tr>
            <th class="thHead" colspan="2">Information</th>
         </tr>
         <!-- BEGIN Informationset -->
         <tr>
            <td class="row1" valign="top" width="50%">{Informationset.info}</td>
            <td class="row2" width="50%" valign="top">{Informationset.value}</td>
         </tr>
         <!-- END Informationset -->
         </table>
      </td>
   </tr>
   <form method="POST" action="admin_antispam_testcenter2.php?{SID}&mode=test" name="antispamform">
      <input type="hidden" name="action" value="test">
      <tr>
         <td class="row1" align="center">
         <table width="100%" cellpadding="4" cellspacing="1" border="0" class="forumline" align="center" id="table5">
            <tr>
               <th class="thHead" colspan="2">Tests</th>
            </tr>
            <!-- BEGIN TestSet -->
            <tr>
               <td class="row1" valign="top" width="50%">{TestSet.info}</td>
               <td class="row2" width="50%" valign="top">{TestSet.value}</td>
            </tr>
            <!-- END TestSet -->
         </table>
         </td>
      </tr>
   </form>
   <tr>
         <td class="row1" align="center">
         <table width="100%" cellpadding="4" cellspacing="1" border="0" class="forumline" align="center" id="table5">
            <tr>
               <th class="thHead" colspan="2">Test results</th>
            </tr>
            <!-- BEGIN TestResultSet -->
            <tr>
               <td class="row1" valign="top" width="50%">{TestResultSet.info}</td>
               <td class="row2" width="50%" valign="top">{TestResultSet.value}</td>
            </tr>
            <!-- END TestResultSet -->
         </table>
         </td>
      </tr>
</table>
</form>
</tr>
</table>
<br clear="all" />

TOP View user's profileSend private messageVisit poster's website  BOTTOM
PostPosted: Fri May 25, 2007 10:17 pm    Post subject:  Reply with quote  

Lord Raiden
Experienced user


Joined: 11 Dec 2006
Posts: 123

Thanks! Smile

TOP View user's profileSend private message  BOTTOM
PostPosted: Tue Jun 12, 2007 10:38 pm    Post subject:  Reply with quote  

Lord Raiden
Experienced user


Joined: 11 Dec 2006
Posts: 123

Websnail, here's an add-on that highlights the text right in the admin_antispam_database.php file so that when you're viewing a post to determine if it's spam or not, you can see what words got it flagged. It's still got a few quirks, but I figured to get at least the test idea out there for people to chew on and then polish it from there. Here's what I left at phpbb.com to get feedback on this as well:

----------------------------------------------------------------------------

WEE!! Ok, I've got highlighting of spam words figured out. Sorta. Need some people to plug this into their forums and test this for any possible improvements. Also, I need someone who has better knowledge of php classes to fix the sql statement on my function below. It's written for mysql since that's what I know and run and I couldn't figure out how to do the db class right. So this is sort of a patch while we work out the bugs in this.

Ok, open admin/admin_antispam_database.php and go to line 806. You should see this:

Code:
   while($row = $db->sql_fetchrow($result))
   {   


Replace everything in there from line 806 to 838 with this:

Code:
   while($row = $db->sql_fetchrow($result))
   {   
        $result2= grab_db_data($row['Id']);
        $vartype = ''; $all = ''; $varvalue= '';

        $all .= '<table border="1" style="border-collapse: collapse" width="100%" id="table1">';
        $all .= '<tr><th align="left">Field</th>';
        $all .= '<th align="left">Value</th><th align="left" width="155">Type(post,get)</th></tr>';
        $i_action = 0;
   // Outputs list of fields trapped by spam filter
         while($row2 = $db->sql_fetchrow($result2))
         {
      // Test for and highlight spam words
      $test_varkey = $row2['varkey'];
      if ($test_varkey == "subject" || $test_varkey == "message")
      {
         $test_message = $row2['varvalue'];
         $message_out = highlight_spam($test_message);
      } else {
         $message_out = $row2['varvalue'];
      }

      $i_action++; if($i_action >2) $i_action= 1;
      $all .= '<tr><td class="row'.$i_action.'" valign="top"  align="left">'.$row2['varkey'].'</td>';
      // if this is a website, treat it special, otherwise use default template.
      if($row2['varkey'] === 'website')
      {
         $all .= '<td class="row'.$i_action.'" valign="top"  align="left"><a href="'.$row2['varvalue'].'" target="_blank">'.$message_out.'<a/></td>';
      } else {
         $all .= '<td class="row'.$i_action.'" valign="top"  align="left">'.$message_out.'</td>';
      }
      $all .= '<td class="row'.$i_action.'" valign="top"  align="left">'.$row2['vartype'].'</td></tr>';
         }


It cleans up Ramon's code a bit and adds in the changes for highlighting of spam words.

Next, put this at the bottom of the file right before "?>".
Code:
// Function for highlighting spam words for the admin.
function highlight_spam($test_message)
{
   // Grab our wordlist and throw it into arrays.
   $query  = "SELECT word FROM phpbb_antispamdb_words";
   $result = mysql_query($query);
   $counter = 0;
   while (list($spam_word[$counter]) = mysql_fetch_row($result))
   {
      $counter++;
   }

   // Clear result
   $total = $counter;
   $result = "";
   $counter = 0;

   // Filter for spam and highlight accordingly.
   while($total > $counter)
   {
      $word = $spam_word[$counter];
      $word = str_replace("\*", "", preg_quote($word, '#'));
      $test_message = str_replace("$word", "<font color=\"red\"><b>$word</b></font>", $test_message);
      $counter++;
   }

   return $test_message;
}


That'll give you full highlighting of spam words in red bolded letters so you can see what someone spammed you with. It won't catch the "leeted" code and highlight it, mostly because I couldn't get that to work right. If someone's got some ideas on how to pull that off, I'm game. I tried it and it just made the script run forever it seemed like. So I left it out. I was probubly doing something wrong, so if someone's got any ideas, I'm open to them.

TOP View user's profileSend private message  BOTTOM
PostPosted: Wed Jun 13, 2007 9:37 am    Post subject:  Reply with quote  

WebSnail
Phpbb coder


Joined: 14 Dec 2006
Posts: 38

Unfortunately I'm not convinced that code will pick out the words that have been tagged because they combine badly (ie: with spaces removed).

The whole thing needs a very complex regex which I don't have the skills to resolve yet but when I have the time... oooh yes Smile

TOP View user's profileSend private messageVisit poster's website  BOTTOM
PostPosted: Wed Jun 13, 2007 1:43 pm    Post subject:  Reply with quote  

Lord Raiden
Experienced user


Joined: 11 Dec 2006
Posts: 123

Well, I'm still kind of a noob php programmer myself, but I've tested this and it *does* highlight a good percentage of the words, but not all of them yet. For now the base code is in place. The rest is just sorting out what needs to be tweaked to make this work. Getting it working in the first place was all that I was bothered with right now. I figured that if I at least did this much and threw it out here, you guys could pick it up and run with it further.

TOP View user's profileSend private message  BOTTOM
PostPosted: Wed Jun 13, 2007 2:03 pm    Post subject:  Reply with quote  

WebSnail
Phpbb coder


Joined: 14 Dec 2006
Posts: 38

Lord Raiden wrote:
Well, I'm still kind of a noob php programmer myself, but I've tested this and it *does* highlight a good percentage of the words, but not all of them yet. For now the base code is in place. The rest is just sorting out what needs to be tweaked to make this work. Getting it working in the first place was all that I was bothered with right now. I figured that if I at least did this much and threw it out here, you guys could pick it up and run with it further.

Oh hey... not knocking your efforts.. it all helps.. For what it's worth I start to think the problem through and then my brain sort of freezes up until someone hands me a beer or something Smile

I suspect it'll be something that comes together while I'm asleep and trying to focus on something else.

TOP View user's profileSend private messageVisit poster's website  BOTTOM
PostPosted: Sat Jun 16, 2007 4:27 pm    Post subject:  Reply with quote  

ramon fincken
Site Admin


Joined: 11 Dec 2006
Posts: 269
Location: A'dam/Diemen, The Netherlands

now this looks pretty cool websnail !

I'll def. take a look at this one Smile
_________________
Phpbbantispam founder, available for freelance WordPress coding.

PhpBB2 mod: http://www.phpbbantispam.com/viewtopic.php?t=1
WordPress plugin: http://wordpress.org/extend/plugins/antispam-for-all-fields/
Project files & mailinglist: https://sourceforge.net/projects/phpbbantispam/

TOP View user's profileSend private messageVisit poster's website  BOTTOM
 Post new topic   Reply to topic All times are GMT + 1 Hour

Display posts from previous:    Goto page 1, 2  Next
 PhpBB- & WordPress- antispam.com Forum Index » [PhpBB2] Feature requests » highlighting of forbidden words
 
 Page 1 of 2
 
 
 
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Jump to:  


     Powered by phpBB © 2001, 2005 phpBB Group | Template Neon | Snelle Managed WordPress webhosting