rblcheck

rblcheck Manual About rblcheck rblcheck is a very basic interface to RBL-style DNS listings such as those operated by the MAPS and ORBS projects. The general idea behind RBL-style listings is rapid lookup of IP addresses using DNS (for example, for blacklisting IP addresses because of abuse). Each IP address is reversed and has a domain name attached to it; for example, the IP address 127.0.0.2 would become 2.0.0.127, and then a domain such as "relays.visi.com" would be added to it. You would then try to resolve the result (ie. 2.0.0.127.relays.visi.com); if you receive a positive reply, then you know that the address is listed. Further information can also be queried, such as text descriptions of why the address was listed. rblcheck is licensed under the terms of the GNU General Public License, a free software license. Building

Supported Platforms I currently perform a manual build and test of rblcheck on the following platforms prior to each release: Red Hat Linux 7.0/Intel Debian GNU/Linux 2.2r2/Intel Sun Solaris 7/SPARC I speculate that most releases will compile on the following platforms: Sun SunOS 4.x Sun Solaris 2.x/7/8 IBM AIX 4.3.3 BSDI BSD/OS 2.x/3.x/4.x FreeBSD 2.x/3.x/4.x Compaq Digital UNIX/Tru64 4.0x Hewlett-Packard HP/UX 10.x/11.x Most versions of Linux (libc5- and glibc-based) Almost any relatively POSIX platform with a resolver library. Please send success reports (and what, if anything, you had to change to make rblcheck build correctly) for your platform of choice to rblcheck-users@lists.sourceforge.net.

Compiling Before building rblcheck, edit the file sites.h for the listings you wish to check against (to un-comment a listing, remove the /* and */ around the SITE(...) line). To compile rblcheck, just run ./configure in the main directory, followed by make when the first command is complete. To make sure that what you now have works correctly, run make check; make sure that you have an Internet connection handy when you do, or a large number of the tests will produce (non-fatal) failures. When you are done, you can type make install to install the software (by default, everything will be installed in /usr/local. For more details on how the configure command works, please see the file INSTALL in the main directory, which has a complete breakdown of all of your options.

Note to developers You may have received the version of rblcheck that you're currently working from via a nightly CVS snapshot (the file you downloaded would likely have been called rblcheck-20020101.tar.gz or something similar), or you might have downloaded a version of the source directly from the CVS tree. The most distinguishing feature of these developer-only versions is the lack of a configure script in the main distribution directory, and the presense of a bootstrap script instead. These releases are not for the faint of heart, and you may very well be unable to build them without a bit of work. As a matter of fact, the documentation you're reading right now may not even apply to one of these experimental versions. If you are not building a development release, you do NOT need to read this section, as none of these instructions apply to you. Please read for instructions on how to build a normal rblcheck release. And with that out of the way, welcome to rblcheck development! You will need a number of utilities to build a development release, above and beyond the usual build requirements: Autoconf 2.13 Automake 1.4 docbook-tools 0.6.8 (the Red Hat RPM package identifies itself as docbook-utils, other distributions may name the package differently, if they include it at all) Autoconf and automake are tools which make cross-platform development easier by automatically checking for issues related to particular platforms. The basic idea is to ensure that even though the primary developers of a piece of software may not have access to a wide range of platforms, the chances of a successful "out of the box" build will still work. The reality is a little different, of course, but it works relatively well. If this sounds like your cup of tea, you might be interested in reading the definitive book on GNU Autotools: GNU Autoconf, Automake, and Libtool. The docbook-tools package is required for building multiple formats of the documentation included with rblcheck. The documentation is written entirely in SGML (Docbook 4.1), and is located in the file docs/rblcheck.sgml. Once you've verified that you have these installed, you'll need to run ./bootstrap from the main rblcheck source directory, which will create a number of the files you need to complete the build. At this point, you should follow the usual instructions for building the application (see ).

Note to distributors and packagers If you're distributing a pre-built version of rblcheck for your users, I strongly recommend leaving everything in sites.h commented out. The ability to add sites there was only added as a convenience feature for people building a personal copy for their own use, and was never intended to be used to pre-set policy for people using pre-packaged versions. Over the years, there has been a great deal of volatility in the RBL listing "market"; even the venerable MAPS RBL is no longer publically available without making special arrangements. Therefore, along with those pre-defined lists of RBL services is built-in obsolescence. As a side-note, I don't make new releases just to update the sites.h file, and at some point in the future, I'll probably stop seeding it with suggestions. As the package maintainer, you're taking on the burden of keeping that up-to-date. Do you really want that hassle, or would it just be easier to point your users at a list of RBL services that is up-to-date (see the rblcheck homepage for links to several current listings)? You're also making life a more difficult for the user; users will come to depend on that pre-built list of sites. What happens when, by necessity, you need to change that built-in list (because some listings are no longer available, or because new ones are now online)? Your users will very likely complain about the change in behavior, and rightly so. Even worse, what does the user do when they disagree with your selection of services? Recompile from source (which defeats the purpose of providing them with a pre-compiled binary in the first place)? Let them pick their own listings. You and your users will be much happier. If you feel that you just can't release rblcheck without a built-in list, may I suggest using the global rblcheckrc which the rbl script uses? At least that way, the end user can change your pre-selections on their own, without having to build from source.

Using

rblcheck basics The program has several command line options: Be quiet about whether the IP address was matched or not. Handy for automatic processing scripts which just check the return code of rblcheck. Display, if available, a textual description of why the site was originally placed in a particular listing. Stop checking listings after the first successful match of any IP address against any listing. List the currently defined RBL listings. By default, rblcheck has no pre-defined listings; you can change this behavior by editing the file sites.h as described in . Clear the list of defined RBL listings. This is handy if you don't want to use the default listings added by the installation of rblcheck on your system. <service> Toggle a service in the list of defined RBL listings. , Get help about rblcheck. Display the version of rblcheck. You can check multiple addresses at once, to make life a little easier with scripts; just add additional addresses on the command line after all of the options. By specifying an address of "-", rblcheck will read from stdin and perform a lookup on every line of input (if it is a valid IP address).

Return codes When invoked, rblcheck returns either 0 (to indicated error, or that the address was not in any of the listings), or a positive number (indicating the number of listings that the IP address was found in).

Verifying your installation To verify that the program is working after you've compiled it, try the following test: $ ./rblcheck -s relays.ordb.org 127.0.0.1 127.0.0.1 not RBL filtered by relays.ordb.org $ ./rblcheck -s relays.ordb.org 127.0.0.2 127.0.0.2 RBL filtered by relays.ordb.org If you see any other result than the above, then something has gone wrong; please see below about reporting bugs and finding help.

The rbl wrapper A recent addition to the rblcheck package is the rbl shell script, which is a simple wrapper around rblcheck, with one special feature: it will read a global rblcheckrc (usually in /etc or /usr/etc), and then a .rblcheckrc from the current user's home directory. These files can contain any of the usual rblcheck command-line arguments (see ), but are most useful for setting the most common RBL-style listings you use day-to-day. For example, your personal .rblcheckrc might contain: -t -m -c -s relays.ordb.org -s relays.visi.com -s some.future.rbl.listing Please note that for mail filtering, or any performance-sensitive use, you should use rblcheck directly, to avoid adding additional latency to the process. rbl is mainly intended as a convenience tool for interactive use.

Third-Party Software rblcheck works well as a command-line tool for performing lookups, but it also serves an important role working with other software, such as mail servers and email filtering programs.

Procmail rblcheck was really designed to be used with procmail, as long as you have access to the IP address of the system sending you email. Surprisingly, most MTAs make obtaining this information more difficult than it needs to be. The procmail rule I present here assumes you've found some way to put the IP address of the sender in the variable TCPREMOTEIP. See the sections below on Sendmail and QMail for ideas on how you can get ahold of this value. The following procmail rule will, once you have TCPREMOTEIP, use rblcheck to look up the IP address in the built-in filters: :0 * ! ? if [ -n "$TCPREMOTEIP" ]; then rblcheck -q "$TCPREMOTEIP"; fi { EXITCODE=100 LOGABSTRACT=all LOG="Filter: RBL-filtered address: \"$TCPREMOTEIP\" " :0: $FILTER_FOLDER } FILTERFOLDER is assumed to have been set up ahead of time as the place to put email that you don't want to see (either another incoming folder, /dev/null, or a 'formail' invokation that rewrites the message and tacks on an extra header or munges the subject so you can easily identify it. Note that EXITCODE above is for QMail, and indicates a permanent error. Under Sendmail, 77 is more appropriate. Under anything else, it's hard to say; your best bet is to refer to the documentation regarding execution of programs. One more thing: procmail has a nasty habit of munging the environment, but most MTAs already do this for you. Hence, you should add the '-p' flag to the invokation of procmail (either from a local delivery rule in Sendmail, or from a .forward or .qmail file). This will ensure that procmail doesn't clean out the value of TCPREMOTEIP. To test the procmail recipe: Save any mail message, with full headers, to a file. Run procmail with the environment variable TCPREMOTEIP set to an offending address, and with the message you just saved as input: cat message | env - TCPREMOTEIP=127.0.0.2 procmail -p Check your procmail log and mailbox. If the message went through, you have a problem. If you have a message in your logfile stating that the message was bounced for being in the RBL, you're fine. Run procmail with the environment variable TCPREMOTEIP set to a non-filtered address, such as 127.0.0.1, and with the message as input: cat message | env - TCPREMOTEIP=127.0.0.1 procmail -p Check your procmail log and mailbox. If the message didn't go through, you have a problem. If you have a copy of the message in your mailbox, and no errors in your log file, you're fine.

Sendmail This solution for obtaining the IP address of the connecting host could be considered to be a bit of a hack, but it works quite reliably. If you're an ordinary user on your system, you won't be able to use this; talk to your system administrator about the possibility of installing the sendmail.cf patch below. Point them at this file as a source of information. Currently, in your sendmail.cf file, you'll probably have something like: Mlocal, P=/bin/mail, F=lsDFMAw5:/|@SnE, S=10/30, R=20/40, T=DNS/RFC822/X-Unix, A=mail -f $g -d $u Or, if you're using procmail as the local delivery agent: Mlocal, P=/usr/bin/procmail, F=lsDFMAw5:/|@ShPfn, S=10/30, R=20/40, T=DNS/RFC822/X-Unix, A=procmail -a $h -d $u This is the local delivery rule used to execute .forward scripts. Your system might use something like rsh or another restricted shell instead of sh for running programs. Don't let that scare you; they all basically work the same. Change the above lines to look like this (there will also be Mprog lines which look similar; you can modify them in exactly the same manner): Mlocal, P=/usr/bin/env, F=lsDFMAw5:/|@SnE, S=10/30, R=20/40, T=DNS/RFC822/X-Unix, A=env TCPREMOTEIP="${client_addr}" mail -f $g -d $u (replacing "mail -f $g -d $u" with "procmail -a $h -d $u" appropriately.) ONLY change the P=... and A=... entries. Most certainly do not mess with F=... unless you know what you are doing. This will create an environment variable TCPREMOTEIP, which you can now use with rblcheck to determine if the address has been blocked. To test this, set up an alias like: foo: |mailx -s "$TCPREMOTEIP" user@domain.com Then, send email to the alias "foo" (or whatever). You should immediately get a piece of email with the IP address which sent the message in the subject line. (Replace mailx with mail on some systems.) This is about the most efficient means of getting this information to executed programs that I can see with sendmail. What would -really- be nice here would be a way to program how Sendmail sets up the environment before executing an external program, at the point of execution. Bug the Sendmail developers if you agree with me. ;-)

QMail Getting this going under QMail turns out to be a real challenge, since QMail doesn't have the same level of programmability that Sendmail has. Hence, we need to employ an additional script to grab the IP address from the headers. (Thanks to Russell Nelson for confirming QMail's behavior here.) QMail has a very specific means of adding Received: lines to messages, making them relatively easy to parse. For example, the following headers are typical: Return-Path: <emarshal@xnet.com> Delivered-To: emarshal@LOGIC.NET Received: (qmail 26029 invoked from network); 13 Oct 1997 15:04:13 -0000 Received: from quake.xnet.com (HELO mail.xnet.com) (198.147.221.35) by labyrinth.logic.net with SMTP; 13 Oct 1997 15:04:13 -0000 We can disregard the Return-Path: and Delivered-To: lines; they're unimportant to us. The Received: headers are the most interesting. The first Received: line we'll see is the local delivery of the mail; hence, the "qmail 2609 invoked from network". The second Received: line is the most important to us; it's the one which contains the IP address of the sender...in this case, 198.147.221.35. To complicate things, the "(HELO mail.xnet.com)" section may not exist, and the IP address might have ident information prepended to it (like "qmailr@198.147.221.35"). Two programs are provided to help you retrieve this information automatically from the headers, both with the same semantics. "origip.c" compiles into "origip", and for those who have trouble compiling it (if you do, please email me with any errors), "origip.awk" is provided which behaves the same way. Essentially, you pass either of these programs an email message, and they in turn extract the sending address and either print it back to you, or exit with a non-zero return value. To use this in procmail, just use: TCPREMOTEIP=`origip || echo 127.0.0.1` This will pipe the message through origip (replace origip with "origip.awk" in the case of using the awk script), and will capture the address. If there is an error, we'll default to 127.0.0.1, which will allow the mail through. (If you're undecided which program of the two you want to use, consider that the C version is much faster, and will be maintained more than the awk script. However, the C version is probably more prone to bugs. ;-) Once you have that line in place, go ahead and use the procmail recipe supplied above in good health.

inetd and smtpd If you use an smtp server which runs from inetd (sendmail can operate this way, as can QMail and a number of other MTAs), here's a good way to do site-wide filtering using rblcheck; add the following to /etc/hosts.allow: smtpd: ALL: spawn /usr/local/bin/rblcheck -q %a && \ exec /usr/local/bin/smtpd || /bin/echo \ "469 Connection refused. See http://maps.vix.com/rbl/\r\b\r\n" This gives you RBL support on a site-wide basis, even if native support doesn't yet exist for your MTA of choice. This assumes that your inetd has support for tcp_wrappers checks; many Linux variants fall into this category, but other platforms may differ. A word of warning, however: there are a number of very good reasons to not run a production mail server from inetd, which I won't elaborate on here. You might want to investigate more secure and reliable alternatives, such as tcpserver from D. J. Bernstein's excellent daemontools package.

Other Software Don't ask me. If you figure out a way to make this work under another setup, let me know how you did it, and I'll add it here. If you find better ways of doing this than the ones I'm using above, let me know too, and you'll see your idea show up in here in the next release.

Future

To Do There are a number of enhancements to rblcheck that are up for consideration, in no particular order. Patches for any of the below will earn you fame, fortune, and a warm fuzzy feeling for having made the world a better place. ;-) Follow CNAMEs from rblcheck(). This will let us get at the eventual TXT RRs for a domain name. Optional support for adns. Timeout option. Should default to no timeout, but apparently there is an issue with rblcheck hanging in an offline environment where the nameserver in use is located off-site. This would compensate for the misconfiguration, and provide a 'quick-fail' mode for people to play with. I'm still not entirely convinced that this should be rblcheck's problem, though; using a local cache like dnscache from djbdns seems like a much more reliable solution to what is a general misconfiguration. (from Aaron Schrab aaron@schrab.com, era eriksson era@iki.fi, and torben fjerdingstad tfj@fjerdingstad.dk) Add the ability to only list successful matches, and add a debug mode which outputs what is happening internally, and any errors received. This could be best implemented with a '-v X' option, where 'X' is a number between 0 and 9 (0 being silent, 4 being default/ordinary output, 9 displaying heavy debugging info). Add an option to display the PTR address returned by the original query, prior to following it. (from Craig Callender craig@mosquitonet.com) Add the ability to specify a particular PTR address; for example, when performing a lookup in ORBS, allow for matching against only a specific returned address (such as 127.0.0.2). Perhaps allow a list of these matches. The question is: is this better implemented here, or in the caller via a filter (after the display of PTR addresses is added)? Send ideas to rblcheck-users@lists.sourceforge.net.

...or Not To Do? These are the features that you'll never see in the version of rblcheck that I produce (rblcheck is GPL; you're free to produce your own version if this doesn't suit you well), and the rationale behind why not. I'm hoping that the explainations will make sense, but I'm always open to criticism if you feel my logic is flawed. Support for doing lookups of FQDNs This is a futile method of checking whether a particular system is blackholed; if the originating system has control of reverse DNS for their IP space, they can make their IP address resolve to anything they want, such as system.network.com. If rblcheck were to do a lookup on that, "system.network.com" might resolve to anything (such as "127.0.0.1, which will always pass with every currently known RBL-based blacklist). A compromise that I've considered would be similar to TCP Wrapper's system of multiple lookups: take the IP address, look up the reverse, then resolve the reverse to an IP address. If both lookups match, then proceed, otherwise exit with a failure. I'd consider integrating code from someone who implemented this behavior. (A note for the confused: some people might mistake this for my saying that a tool for looking up hosts in DNSBL listings (a means of looking up originating domains in a blacklist, much like the RBL) isn't a good idea. On the contrary, I'd like to see a complimentary dnsblcheck to go along with rblcheck. The problem I have is specifically with resolving the name to an IP address for use by an RBL lookup tool.) Return codes that identify a particular blacklist This took a while to convince me it was wrong. Here are the questions I went over when deciding against this: What do you do with your scripts when the blacklist order changes, or new blacklists appear (assuming a dynamic assignment of identifiers)? If you hard-code a number for each and every blacklist, who becomes responsible for being a registry for those numerical identifiers? What happens to already-assigned values when blacklists shutdown or change incompatibly? How do you scale beyond 256 RBL services? With the advent of software such as rbldns (a part of the djbdns package which allows anyone to host an RBL-style service), and with nearly a dozen OsiruSoft domains alone, does this limitation seem reasonable? How do you report multiple matches with a return code, when the return code can only hold one number? Some of these can be solved, but not cleanly; you'll need no end of special cases to accomodate them. Integrated scanning of email and similar features This is a question of UNIX philosophy: design one tool for one task, and do it well. There are plenty of far more powerful text parsing tools available which are more appropriate for the job (email parsing is easily handled by tools such as formail, general text parsing is better suited by tools such as sed, awk, perl, python, etc), and they can do a much better job than something that I could integrate into rblcheck while still maintaining a small footprint.

Notes on origip Not convinced that you should use the C version of origip? Here's some data (using test_origip.sh as a testbed) which definitely speaks volumes: With origip.awk: 18.60user 22.46system 0:43.59elapsed 94%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (102172major+51041minor)pagefaults 0swaps With origip.c: 13.16user 19.41system 0:33.73elapsed 96%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (71381major+46044minor)pagefaults 0swaps 10 seconds faster, 3 seconds less system time used, 5 seconds less user time, and over 30,000 less page faults. In other words: a LOT easier on the box you'll be running this on. This test was run on a little under 600 messages, from various mailing lists and private messages, on a Linux 2.0.31 system. Reporting Problems If you're in need of assistance, a (very low volume) mailing list for rblcheck users has been set up; to subscribe, simply send an email to rblcheck-users-subscribe@lists.sourceforge.net, answer the confirmation email, and you'll be able to post your questions. (Archives) All software has bugs, and rblcheck has a place for you to report them if you believe you've found one. (Bug tracker) If you have a SourceForge account already, please log in before reporting a bug so that you can be notified of changes. Credits Thanks, kudos, and other good stuff go to: Paul Vixie paul@vix.com For the original RBL, a fantastic tool for the prevention of spam. Often duplicated, never repeated, and all that jazz. The MAPS Project A great gathering point for protection services regarding email. Everyone there should be given a pat on the back for the thankless hours they put into maintaining an excellent resource for everyone. Stephen R. van den Berg srb@cuci.nl For writing Procmail, a nifty little mail filtering tool. While certainly not the only place to use rblcheck, it's certainly one of the more useful. Dire Straits For writing good music to write code to. ;-) I wonder if spam qualifies as a symptom of industrial disease? ...and a cast of thousands: Russell Nelson nelson@crynwr.com Frank Tegtmeyer fte@iqweb.de Anders Eriksson aer@cdt.luth.se Kevin Kadow kadokev@ripco.com Ronald F. Guilmette rfg@monkeys.com era eriksson era@iki.fi Ophir Ronen ophir@internap.com Brian Willoughby brianw@sounds.wa.com Jeff A. Earickson jaearick@colby.edu Gerald Pfeifer pfeifer@dbai.tuwien.ac.at Pat Myrto pat@rwing.COM William Yang wyang@freenet.columbus.oh.us Dougal Campbell dougal@advicom.net Jason Gunthorpe jgg@gpu.srv.ualberta.ca Timothy J Luoma luomat@peak.org Adam Shand larry@alaska.net Andrew Kelley kelleya@dimensional.com Thomas Meyer tmeyer@ifi.unizh.ch Jonathan Bradshaw jonathan@NrgUp.Com Jacques Distler distler@golem.ph.utexas.edu Harlan Stenn Harlan.Stenn@pfcs.com Marco d'Itri md@Linux.IT