Greylisting anti spam solution for Qmail

Greylisting je zajímavá metoda filtrování spamu. Je založena na tom, že spamovací automaty nemají ve většině případů plně implementované veškeré RFC. Jak to funguje a jak jej implementovat do Qmailu?Greylisting is an interesting method to fight against spam. It’s based on the fact most spam is not sent out by fully compliant MTA’s. How it works and how this can be implemented into Qmail?

Určitě znáte metodu whitelistů a blacklistů. Kdo/co je na whitelistu – tak projde filtry vždycky, kdo je na blacklistu – neprojde nikdy. Greylist funguje na úrovni SMTP spojení a říká neznámým serverům “teď od tebe nemohu přijmout poštu, zkus to za chvíli”. Správně implementované poštovní servery (sendmail, qmail, postfix, atd.) zařadí poštu do fronty a zkusí to znovu za nějakou dobu. Spammeři se většinou nikdy nevrátí a už to nezkusí, protože nemají tak chytré nástroje na spamování. Většinou spamují napadené počítače, které se bezhlavě připojují na servery a rozesílají spamy. A kdyby přeci jen spammer přšel znovu, získali jsme nějaký čas, během kterého se již ip adresa útočníka může dostat na blacklist rbl filtrů. Více se o metodě můžete dozvědět na www.greylisting.org.Při implementaci greylistingu pro Qmail jsem vycházel z doporučení identifikovat příchozí poštu podle trojice indentifikátorů. A to je adresa odesílatele, adresa příjemce a IP adresa příchozího serveru. Pokud se tato trojice objeví při SMTP spojení poprvé, odmítneme spojení z dočasných důvodů (kód 421). Správně fungující server se vrátí za několik minut a zkusí to znovu. V tu dobu už odesílatele známe a pošta je přijata.
Společně s RBL filtrem (sbl-xbl.spamhaus.org) se jedná o velmi účinnou obranu. Takto vypadají statistiky za několik málo dní na jednom z administrovaných poštovních serverů:

Odmítnuté spojení na základě RBL: 6712
Počet pokusů o odeslání, které už nebylo opakováno: 5009
Počet doručených zpráv: 811

Mnoho implementací využívá databázi (ať už MySQL nebo PostgreSQL) pro evidenci pokusů o doručení, což se mi zdálo jako kanon na vrabce, proto jsem použil obyčejné soubory pro uchování těchto informací. Celý program je napsán v perlu (rychlost je pro operaci se soubory více než dostačující) a je volán ze smtp daemonu (qmail-smtpd), ktrerý se na základě návratového kódu rozhoduje, zda zprávu přijme nebo ne. Celý vtip je v tom, že vytváříme prázdné soubory. Všechny informace jsou uloženy buď v názvu (trojice indentifikátorů) a v čase vytvoření a změny souboru (ctime a mtime).

Ze všeho nejdříve tedy musíme patchnout qmail-smtpd. Použil jsem metodu Guenthera Maira (který pro greylisting použil PostgreSQL a C):

You surely know the method of whitelists and blackists. Who/what is on a whitelist will pass a filter without any restriction. Who is on blacklist will never pass. Greylist method works on SMTP layer during server communication. It says to unknown servers “I’m unable to receive a message, try again later”. Correctly implemented MTA’s (sendmail, qmail, postfix, etc) spool this message for later delivering. Spammers ussualy never return because their mail agents are too stupid to do so. Usually, they are spamming from compromited computers using very small scripts which are connecting SMTP servers like headless chicken. Yes, some spammers have more intelligent MTA, but they are able to be identified by other antispam tools during the time when waiting for 2nd attempt. You can find more info about greylisting method on www.greylisting.org.I have used a recommendation to identify all incoming mails on unique triplet for identifying a mail during my Qmail greylisting implementation. This unique triplet consists of the envelope sender address, the envelope recipient and the IP address of incoming server. If this triplet occurs for the first time, we will temporary reject the e-mail (reply code 421). All correctly working MTA’s will return in several minutes and will try again. In this time, we already know the triplet and the mail is accepted.

Together with RBL filtering (sbl-xbl.spamhaus.org) it is a very effective solution. Here are some statistic data during last several days from one of Qmail servers:

RBL denied connections: 6712
Greylist never repeated rejected connections: 5009
Accepted messages: 811

Many greylist implementations use a database (MySQL or PostgreSQL) for storing the data, but it looks as an overhead for me. Therefore I used simple files for storing all those info. The script is written in perl (the performance is fast enough for the file processing) and it is called within the smtp daemon (qmail-smtpd). It will decide if the mail is accepted or not on a return code of the called script. The point is that we will create empty files, all the info is stored in the names of files (unique triplet) and in the creation and modification time (ctime and mtime).

The very first step is to patch the qmail-smtpd.c. I have used the Guenther Mair idea (he used PostgreSQL and C for greylisting):


--- qmail-1.03.orig/qmail-smtpd.c 1998-06-15 12:53:16.000000000 +0200
+++ qmail-1.03/qmail-smtpd.c 2006-09-07 11:09:24.184352312 +0200
@@ -19,6 +19,9 @@
#include "env.h"
#include "now.h"
#include "exit.h"
+#include "open.h"
+#include "fork.h"
+#include "wait.h"
#include "rcpthosts.h"
#include "timeoutread.h"
#include "timeoutwrite.h"
@@ -28,6 +31,9 @@
unsigned int databytes = 0;
int timeout = 1200;
+#define GREYLIST_STATFILE “/var/qmail/control/greylist”
+void die() { _exit(111); }
+
int safewrite(fd,buf,len) int fd; char *buf; int len;
{
int r;
@@ -49,6 +55,8 @@
void die_ipme() { out(“421 unable to figure out my IP addresses (#4.3.0)rn”); flush(); _exit(1); }
void straynewline() { out(“451 See http://pobox.com/~djb/docs/smtplf.html.rn”); flush(); _exit(1); }

+void err_tempfail() { out(“421 temporary envelope failure (#4.3.0)rn”); }
+void err_permfail() { out(“553 sorry, permanent envelope failure or blacklist entry (#5.7.1)rn”); }
void err_bmf() { out(“553 sorry, your envelope sender is in my badmailfrom list (#5.7.1)rn”); }
void err_nogateway() { out(“553 sorry, that domain isn’t in my list of allowed rcpthosts (#5.7.1)rn”); }
void err_unimpl() { out(“502 unimplemented (#5.5.1)rn”); }
@@ -222,6 +230,41 @@
stralloc mailfrom = {0};
stralloc rcptto = {0};

+int envelope_scanner()
+{
+ int child;
+ int wstat;
+ static char *envelope_scannerarg[2] = { “bin/greylist”, 0 };
+
+ switch(child = vfork()) {
+ case -1:
+ return 1;
+ case 0:
+ if (!env_put2(“MAILFROM”, mailfrom.s)) die();
+ if (!env_put2(“RCPTTO”, addr.s)) die();
+ execv(*envelope_scannerarg,envelope_scannerarg);
+ env_unset(“MAILFROM”);
+ env_unset(“RCPTTO”);
+ _exit(111);
+ }
+
+ wait_pid(&wstat,child);
+ if (wait_crashed(wstat)) {
+ return 1;
+ }
+
+ switch(wait_exitcode(wstat)) {
+ case 101:
+ err_tempfail();
+ return 0;
+ case 102:
+ err_permfail();
+ return 0;
+ default:
+ return 1;
+ }
+}
+
void smtp_helo(arg) char *arg;
{
smtp_greet(“250 “); out(“rn”);
@@ -248,6 +291,7 @@
out(“250 okrn”);
}
void smtp_rcpt(arg) char *arg; {
+ int fdgrey;
if (!seenmail) { err_wantmail(); return; }
if (!addrparse(arg)) { err_syntax(); return; }
if (flagbarf) { err_bmf(); return; }
@@ -256,8 +300,14 @@
if (!stralloc_cats(&addr,relayclient)) die_nomem();
if (!stralloc_0(&addr)) die_nomem();
}
– else
+ else {
if (!addrallowed()) { err_nogateway(); return; }
+ fdgrey = open_read(GREYLIST_STATFILE);
+ if (fdgrey != -1) {
+ close(fdgrey);
+ if (!envelope_scanner()) return;
+ }
+ }
if (!stralloc_cats(&rcptto,”T”)) die_nomem();
if (!stralloc_cats(&rcptto,addr.s)) die_nomem();
if (!stralloc_0(&rcptto)) die_nomem();

Greylisting (volani sciptu bin/greylist) aktivujeme vytvorenim (prazdneho) souboru /var/qmail/control/greylist. Vlastní perlovský program má v sobě ještě proceduru pro čištění adresáře pro kontrolní soubory. Tento adresář (/var/spool/greylist) musí existovat a uživatel, pod kterým běží qmail-smtpd (qmail, vpopmail), do něj musí mít právo zapisovat. V programu je také implementován zakladní whitelist a blacklist na IP adresy.Greylisting (calling the script bin/greylist from smtp daemon) is activated by touching file /var/qmail/control/greylist. The perl script itself has a procedure for cleaning up the directory where the control files are stored. This directory (/var/spool/greylist) has to exist and the user the qmail-smtpd is running under has to have permission to write there. There is also a simple blacklist and whitelist implemented there.

#!/usr/bin/perl
#
# /var/qmail/bin/greylist
# Greylist spam filtering implementation
#
# Copyright (C) 2007 Vaclav Vobornik
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA
# or see http://ww.gnu.org/licenses/gpl.html.
#
#
# constants
our $LOGING = 2; # loging level: 0:nothing, 1:basic, 2:verbose, 3:debug
our $GREYDIR = “/var/spool/greylist”;
our $CLEAN = $GREYDIR .”/clean”;
our @WHITELIST = (“127.0.0.1″,”88.10.175.111”);
our @BLACKLIST = (“62.18.116.86”);

# mail info
my $m_ip = $ENV{TCPREMOTEIP};
my $m_to = $ENV{RCPTTO};
my $m_from = $ENV{MAILFROM};

# time related variables
our $t_block = 5*60; # temporary block expires (5 minutes)
our $t_timeout = 12*60*60; # unused open window expires (12 hours)
our $t_expiry = 31*24*60*60; # valid open window expires (31 days)
our $t_cleanup = 20*60; # cleanup every (20 minutes)
our $t_now = time();
our $t_clean = $t_now – (stat $CLEAN)[9];

# exit codes
my $e_accept = 0;
my $e_tempreject = 101;
my $e_reject = 102;

#######################################
# == Main program == #
# Don’t edit anything below this line #
#######################################

# add a local address into the @WHITELIST
use Sys::Hostname;
use Socket;
push(@WHITELIST,inet_ntoa(scalar gethostbyname(hostname())));

# and index lists
for (@WHITELIST) {$is_wlisted{$_} = 1}
for (@BLACKLIST) {$is_blisted{$_} = 1}

# functions

sub debug {
my($level,$msg) = @_;
if ($LOGING >= $level) {
print STDERR “greylist: LOG$level: $msgn”;
}
}

sub touch {
my $file = shift;
debug(3,”Touching $file”);
open(FILE,”>$file”);
close(FILE);
}

sub _exit {
my $exit = shift;
debug(1,”Exiting with code $exit”);
exit $exit;
}

sub cleanup {
if ( $t_clean > $t_cleanup) {
debug(2,”Time to clean up”);
open(CLEAN,”>$CLEAN”) or return;
foreach $file (< $GREYDIR/grey:*>) {
if (-f $file) {
debug(3,”Checking $file”);
my $ctime=(stat $file)[8]; # create time
debug(3,”Create time: $ctime”);
my $mtime=(stat $file)[9]; # modify time
debug(3,”Modify time: $mtime”);
if ($ctime == $mtime) {
# only 1 attempt long time ago or
if ($t_now – $ctime > $t_timeout) {
debug(2,”Deleting $file – unused time window”);
unlink($file);
}
}else{
# it’s long time ago when an email came
if ($t_now – $mtime > $t_expiry) {
debug(2,”Deleting $file – inactive sender”);
unlink($file);
}
}
}

}
close(CLEAN);
}
return;
}

# first, do a clean up
cleanup();

debug(1,”Checking $m_from -> $m_to ($m_ip)”);

# isn’t it blacklisted?
if ($is_blisted{$m_ip}) {
debug(2,”$m_ip is blacklisted”);
_exit $e_reject;
}

# isn’t it whitelisted?
if ($is_wlisted{$m_ip}) {
debug(2,”$m_ip is whitelisted”);
_exit $e_accept;
}

# compose a file name

my $filename = “$GREYDIR/grey:$m_to:$m_from:$m_ip”;
debug(3,”File name: $filename”);

if (-f $filename) {
debug(2,”File $filename exists, checking the times”);
my $f_ctime=(stat $filename)[8]; # create time
debug(3,”Create time: $f_ctime”);
my $f_mtime=(stat $filename)[9]; # modify time
debug(3,”Modify time: $f_mtime”);

# non-patient sender:
if ($t_now – $f_ctime < $t_block) {
debug(2,”Non-patient sender $filename, recreating the file”);
unlink($filename);
touch($filename);
_exit $e_tempreject;
}else{
debug(2,”Valid sender $filename, updating the file”);
touch($filename);
_exit $e_accept;
}

}else{
debug(2,”File $filename doesn’t exist”);
touch ($filename);
_exit $e_tempreject;
}

 

Komentáře vítányComments are welcome

Leave a Reply

Your email address will not be published. Required fields are marked *