In: , , ,
On: 2012 / 11 / 21 Viewed: 393 times
Shorter URL for this post: http://ozh.in/vk

I've been speaking lately with folks from Spamhaus about anti spam measure in YOURLS and a YOURLS plugin for this. Currently the #1 result in Google for "spamhaus PHP" is a post on Lockergnome which gets it totally wrong and provides a script that does not work, so here is a PHP script that does work.

This script checks a URL (its domain part, in fact) against the 3 major black lists: Spamhaus, SURBL and URIBL.

The script:

  1. /**
  2.  * Check a URL against the 3 major blacklists
  3.  *
  4.  * @param string $url The URL to check
  5.  * @return mixed true if blacklisted, false if not blacklisted, 'malformed' if URL looks weird
  6.  */
  7. function ozh_is_blacklisted( $url ) {
  8.  
  9.     $parsed = parse_url( $url );
  10.  
  11.     if( !isset( $parsed['host'] ) )
  12.         return 'malformed';
  13.        
  14.     // Remove www. from domain (but not from www.com)
  15.     $parsed['host'] = preg_replace( '/^www\.(.+\.)/i', '$1', $parsed['host'] );
  16.  
  17.     // The 3 major blacklists
  18.     $blacklists = array(
  19.         'zen.spamhaus.org',
  20.         'multi.surbl.org',
  21.         'black.uribl.com',
  22.     );
  23.    
  24.     // Check against each black list, exit if blacklisted
  25.     foreach( $blacklists as $blacklist ) {
  26.         $domain = $parsed['host'] . '.' . $blacklist . '.';
  27.         $record = dns_get_record( $domain );
  28.        
  29.         if( count( $record ) > 0 )
  30.             return true;
  31.     }
  32.    
  33.     // All clear, probably not spam
  34.     return false;
  35. }

Usage:

  1. if( ozh_is_blacklisted( $url ) ) {
  2.     // do something brutal (eg die() your script, yell at user, etc...)
  3. }
  4.  
  5. // all is fine *for today*, do your regular stuff.
  6. // This said, it'd be nice to recheck every couple of days

Feel free to steal.

Shorter URL

Want to share or tweet this post? Please use this short URL: http://ozh.in/vk

Metastuff

This entry "Checking Domain Blacklists from Spamhaus, SURBL and URIBL in PHP" was posted on 21/11/2012 at 10:24 pm and is tagged with , , ,
Watch this discussion : Comments RSS 2.0.

11 Blablas

  1. 1
    Daniel Johnson, Jr. United States »
    thought, on 24/Nov/12 at 6:50 pm # :

    So, is this plugin available? I just got YOURLS set up last night. LOVE it.

  2. 2
    Ozh »
    thought, on 25/Nov/12 at 10:21 pm # :

    Daniel Johnson, Jr. » It is. Check the official plugin list

  3. 3
    JP United States »
    wrote, on 27/Dec/12 at 7:28 pm # :

    Great work. But Spamhaus says you need to reverse the IP, I don't see where your code does that. (FYI: http://www.spamhaus.org/faq/section/DNSBL%20Usage#252) Also, ZEN checks the PBL which should not be used to determine if an IP address belongs to a spammer. The query result needs to be parsed to see if the IP is on the SBL or XBL only.

  4. 4
    Ozh »
    wrote, on 27/Dec/12 at 9:11 pm # :

    JP » Yep you're right. I should update the code. Fancy doing it? :)

  5. 5
    JP United States »
    said, on 27/Dec/12 at 10:19 pm # :

    I'm not familiar with SURBL and URIBL, but I'll try. I'll focus on Spamhaus because I was already working on a MyBB plugin for it. It will take a few days, in the meantime here's how to take the IP and reverse it for Spamhaus:

    1. $revIP = implode(".", array_reverse(explode(".", "192.168.2.1"))) . ".zen.spamhaus.org";
  6. 6
    JP United States »
    said, on 28/Dec/12 at 4:55 pm # :

    OK, I might be wrong but it may have been easier than I thought. Spamhaus and URIBL both return 127.0.0.2 if the IP address is on their blacklist. I'm not sure about SURBL but it may be the same. I was not able to test this. Could you test and let me know?

    1. function is_blacklisted($ip) {
    2.   $blacklists = getblacklistproviders();
    3.  
    4.   foreach($blacklists as $blacklist) {
    5.     $url = buildurl($ip, $blacklist);
    6.     $record = dns_get_record($url);
    7.     if ($record === "127.0.0.2") {
    8.       return true;
    9.     }
    10.   }
    11.   return false;
    12. }
    13. function getblacklistproviders() {
    14.   // add providers here
    15.   return array(
    16.     'zen.spamhaus.org',
    17.     'multi.surbl.org',
    18.     'multi.uribl.com'
    19.   );
    20. }
    21. function buildurl($ip, $blacklistprovider) {
    22.   return implode(".", array_reverse(explode(".", $ip))) . $blacklistprovider;
    23. }
  7. 7
    Ozh »
    thought, on 28/Dec/12 at 5:14 pm # :

    JP » The thing is, except Spamhaus, I don't think you're supposed to reverse IPs, yet your code does it for all providers :)

  8. 8
    JP United States »
    said, on 28/Dec/12 at 6:10 pm # :

    Ozh, you have to reverse IP for all three.

    http://www.uribl.com/about.shtml#implementation
    http://www.surbl.org/guidelines
    http://www.spamhaus.org/faq/section/DNSBL%20Usage#252

  9. 9
    Ozh »
    thought, on 28/Dec/12 at 7:47 pm # :

    JP » Oh yeah, sorry, I mixed up things. Your script is very fine for checking IPs (like, before accepting mail for instance) but this doesn't work well for web spam since a domain can be blacklisted (evil.com) but the IP it's hosted on can be clear (201.202.203.204 shared hosting with lots of clean sites), or the other way round

  10. 10
    Crazy-Jake United States »
    said, on 15/Jul/13 at 4:26 am # :

    So, i've reinstalled my yourls site, and found i needed a spam filter after only a day. HOWEVER, when i try to use this plugin, it blocks EVERYTHING.
    i've not been able to post a single site that would get shortened. I leave it on and only turn it off when I want to post a link, but it prevents any legitimate posts from going though in the meantime. Any word on this?

  11. 11
    Ozh »
    replied, on 07/Aug/13 at 8:38 pm # :

    Crazy-Jake: check https://github.com/YOURLS/antispam/issues/2

Leave a Reply

Comment Guidelines or Die

  • HTML: You can use these tags: <a href=""> <em> <i> <b> <strong> <blockquote>
  • Posting code: Post raw code (no <> &lt; etc) within appropriate tags : [php][/php], [css][/css], [html][/html], [js][/js], [sql][/sql], [xml][/xml], or generic [code][code]
  • Gravatars: Curious about the little images next to each commenter's name ? Go to Gravatar.
  • Spam: Various spam plugins on patrol. I'll put pins in a Voodoo doll if you spam me.
  • I will mark as Spam test comments, all comments with SEO names (ie "My Cool Online Shop" instead of "Joe") or containing forum-like signatures.

Read more ?