$Id: RELEASE.NOTES,v 1.65 2006/05/14 15:27:31 jonz Exp $ RELEASE 3.6.8-STABLE 3.6.8 is a maintenance release BUGFIX: PostgreSQL fails to connect (always) BUGFIX: LDAP client will not build due to header missing from makefile RELEASE 3.6.7-STABLE 3.6.7 is a maintenance release BUGFIX: Using UIDInSignature, wrong UID is written to message when using groups BUGFIX: PostgreSQL driver does not reconnect on failure in daemon mode BUGFIX: X-DSPAM-Probability sometimes misreported when multiple algorithms used BUGFIX: Agent segfaults when DeliveryHost or ClientHost not specified, --client BUGFIX: Agent segfaults on some systems when syslog is used BUGFIX: Agent segfaults when dlopen() to storage library fails BUGFIX: Infinite loop created when deleting preference, not using extensions BUGFIX: ATX (agent context) does not hold enough bits for 'flags' variable 3.6.6 is a maintenance release MAINT: Phased out deprecated Berkeley DB drivers MAINT: Phased out legacy tools (dspam_corpus, dspam_genaliases) BUGFIX: When using logfile, write errors result in segfault BUGFIX: Compiler warnings with sqlite_drv and sqlite3_drv BUGFIX: MySQLUIDInSignature causes segfault on retrain BUGFIX: trainPristine preference "off" does not override default RELEASE 3.6.5-STABLE 3.6.5 is a maintenance release MAINT: PgSQL SQL tuning MAINT: WebUI aesthetic and functional fixes MAINT: Added --disable-syslog and --with-logfile= configuration flags MAINT: Added -t flag for dspam_stats to total stats MAINT: Markov result used as X-DSPAM-Confidence when Markov used MAINT: Support for separate read/write servers to be used with mysql_drv BUGFIX: Spam are quarantined when --deliver=summary BUGFIX: Admin graphs malformatted when subject contains newline character BUGFIX: WebUI does not use MAX_COL_LEN BUGFIX: Output for dspam_admin aggr pref incorrect BUGFIX: Flat-file preference writes fail on some systems BUGFIX: Failure to connect to ClamAV causes segmentation fault BUGFIX: NULL username in system causes segmentation fault BUGFIX: ClamAV processing and cleanup issues BUGFIX: Fragment files overwritten on retrain BUGFIX: Miscellaneous invalid read / segmentation fault bugs BUGFIX: If TrainingMode not specified in dspam.conf or passed in, segmentation fault BUGFIX: No output returned when using --deliver=summary with dspamc RELEASE 3.6.4-STABLE 3.6.4 is a maintenance release DOC: Documented user preferences in README MAINT: Added dspam_train tool, replacing most functions of dspam_corpus MAINT: Code cleanup and performance improvements MAINT: Significant improvements in accuracy, specifically reduced false pos. MAINT: Removed experimental neural collaboration functions MAINT: Added ClassAlias configuration directive to dspam.conf MAINT: Added undo option for retraining via WebUI MAINT: Added storeFragments support to WebUI MAINT: Added mass-retraining support to WebUI BUGFIX: DSPAM segfaults when invalid UID specified using UIDInSignature BUGFIX: No output when using --classify with --client BUGFIX: dspam_corpus overrides default dspam.conf settings BUGFIX: Multi-driver builds fail when preferences-extension is not supported RELEASE 3.6.3-STABLE 3.6.3 is a maintenance release MAINT: Modest performance improvements MAINT: Significant improvements in accuracy (from scratch-testing anyway) BUGFIX: "No TrustedDeliveryAgent" error occurs erroneously in some cases BUGFIX: ChangeUserOnParse can format incorrectly BUGFIX: The dspam_logrotate tool does not support multiple mailboxes BUGFIX: Memory leak in Bayesian Noise Reduction BUGFIX: LDAP calls fail to close connection to server BUGFIX: Bayesian database is created even on LDAP user verification failure RELEASE 3.6.2-STABLE 3.6.2 is a maintenance release BUGFIX: Token values incorrectly calculated, cause false positives BUGFIX: Daemon mode + hash_drv results in crashes without HashConcurrentUser BUGFIX: Invalid call to calloc() in hash_drv can lead to crashes RELEASE 3.6.1-STABLE 3.6.1 is a maintenance release BUGFIX: ParseToHeaders can segfault on malformatted To header BUGFIX: TrackSources does not correctly parse messages when used with QMail BUGFIX: Missing check for existing strcasestr BUGFIX: X-DSPAM-Reclassified heading appears blank BUGFIX: Plused-Detail support does not work with domains BUGFIX: Hash databases are not 8-byte aligned (do not work with 64-bit systems) BUGFIX: Certain daemon-mode operations append a trailing period BUGFIX: DataSource 'document' option causes improper processing BUGFIX: Segfaults occur on malformed Content-Type header BUGFIX: History appears blank in dspam.cgi RELEASE 3.6.0-STABLE ENHANCEMENTS Automatic Whitelisting Improvements Automatic whitelisting has been converted into more of a trusted sender reputation type system. Instead of senders having to send you zero spam in order to be whitelisted, they will have to send you less than one spam for every fifteen legitimate messages. This means a user with whom you've conversed much and exchanged 45 innocuous messages with will be allowed to spam you up to three times before they'll lose their whitelisting status. Dynamically Loadable Storage Driver Support DSPAM now supports a dynamically loadable storage driver plugin codebase, which allows third party driver authors to more easily build new drivers, and packagers to build and distribute dspam with multiple storage drivers. PostgreSQL Performance Enhancements PostgreSQL enhancements have been made for installations of v8.0+, speeding up processing up as much as three times. LDAP Verification LDAP Verification can be used with some storage drivers (such as MySQL and PostgreSQL) and allows DSPAM to verify that a user exists in LDAP before adding them to the system locally. This is ideal for virtual user scenarios where DSPAM is left to validate users. Unified Spam Alias / UID in Signature Support The MySQL and PgSQL storage drivers now support embedded user id's in the message's signature identifier. This means you can now configure a single system-wide spam alias and have DSPAM set the user based on that specified in the signature. No more long alias lists to maintain. Domain Based Delivery Hosts Support for using different delivery hosts for each domain has been added. Each domain and its delivery host can be configured in dspam.conf. Integrated Clam A/V Support DSPAM can now communicate directly with clamd (using streaming mode) to check all messages for viruses. Upon detection, DSPAM can act one of three ways: reject the message with a permanent error, accept the message and quietly drop, or treat the message as spam. Markovian Discrimination Added support for Markovian weighting, enabling DSPAM to perform as a Markovian classifier. These functions were based on Bill Yerazunis' published research describing the techniques used in CRM114. Markovian weighting is a lexically verbose approach to filtering spam, and provides rather impressive results. See doc/markov.txt for more information Self-Contained Hash Driver Inspired by Bill Yerazunis' CRM Sparse Spectra (CSS) storage algorithm, the hash driver (hash_drv) is a very fast, self-contained storage driver using a combination of random access files and hash overflow. It requires no dependencies and outperforms all other storage drivers. Support for auto- extending files is included, which allows a database to grow as needed. Other management tools, such as cssstat and csscompress, are also included. Fallback Domain-Based User Support Fallback domain support can be used by systems where a user's full email address is provided as the username, and allows nonexistent users to fall back to @domain.tld as the username. Ideal for configuring pass-thru. Improved, Unified Logging All logging functions have been consolidated into two functions - one for debugging-only messages, and one for system messages. All system messages are now logged to syslog, stderr, and debug (if debug is active). This should improve the quality of what is logged to the system log. Activity logging has also been improved and the retrain log has been eliminated entirely. Instead, events are logged with different status codes and info text to the user's primary log (and system.log). Plused Detail Support Support for mailboxes using plused detail (e.g. username+mailbox) has been added. DSPAM will ignore everything after + when setting the username, and attach it upon delivery. Message Fragments Support DSPAM can now be configured via a user preference to store 1K of each message processed on the server. This can be used by some GUIs (such as the DSPAM Appliance UI) to display message fragments from history. Domain Blocklisting Support Added support for per-user blocking of certain messages coming from undesirable domains. Blocklisted messages are quarantined, but not learned as spam. Other Enhancements: User preferences made case-insensitive LMTP error codes now include messages from relay host RBLInoculate option allows RBL'd emails to be quarantined or learned as spam MySQL and PgSQL object creation scripts updated to improve performance Added 'date' to quarantine display and sorted most recent at top Significant code cleanup and audit Domain-based groups changed from @domain.tld to *@domain.tld X-DSPAM-Result expanded to support Virus, Blacklisted, and Blocklisted results DSPAM WebUI now supports history paging Rewritten documentation for ease of installation Jonathan Zdziarski