Amavisd-new is a high-performance interface between mailer (MTA) and content checkers: virus scanners, and/or SpamAssassin.
It is written in Perl for maintainability, without paying a significant price for speed. It talks to MTA via (E)SMTP or LMTP, or by using helper programs.
Best with Postfix, fine with dual-sendmail setup and Exim v4, works with sendmail/milter, or with any MTA as a SMTP relay. For Courier and qmail MTA integration there is a patch in the distributed package.
amavisd-new is a high-performance and reliable interface between mailer (MTA) and one or more content checkers: virus scanners, and/or Mail::SpamAssassin Perl module. It is written in Perl, ensuring high reliability, portability and maintainability. It talks to MTA via (E)SMTP or LMTP protocols, or by using helper programs. No timing gaps exist in the design, which could cause a mail loss.
It is normally positioned at or near a central mailer, not necessarily where users' mailboxes and final delivery takes place. If looking for a per-user and low-message-rate solution to be placed at the final stage of mail delivery (e.g. called from procmail or in place of a local delivery agent), there may be other solutions more appropriate.
When calling of Mail::SpamAssassin (SA) is enabled, it calls SA only once per message regardless of the number of recipients, and tries very hard to correctly honour per-recipient preferences, such as pass/reject, check/nocheck, spam levels, and inserting spam-related mail header fields.
amavisd-new benefits from the use of Perl module Net::Server, which offers a fast pre-forked multichild process control. amavisd-new provides rfc2821-compliant SMTP server and client, a rfc2033-compliant LMTP server and client, and generates rfc3462/rfc3464-compliant (ex rfc1892/rfc1894) delivery (and non-delivery) status notifications.
This makes it suitable for mail anti-virus and/or anti-spam checking on a busy mail gateways that care for reliability and standards compliance.
amavisd-new grew out of amavisd(-snapshot) (which in turn is a daemonized version of amavis-perl), but through five years of development turned into a separate product, hardly resembling its origin. The code is several times the size of its predecessor, yet faster in throughput, richer in features, compliant to standards, includes optional support for spam detection, and makes virus scanning optional and easier to adjust/extend. Compatibility with helper programs from amavisd(-snapshot) is retained.
All modifications since the original amavisd done by Mark Martinec, with contribution of ideas, patches and reports from the amavis-user mailing list community and individuals.
Features
-- general
* written in Perl for better maintainability, portability, no risk of buffer overruns or invalid pointers; critical code paths are well optimized;
* a daemonized pre-forking server, saving on startup time; designed with high throughput in mind;
* security conscious, runs under a dedicated user id (not root and not mail), can run in a chroot jail (including SpamAssassin and external programs);
* thorough error checking, informative error reporting, fail-safe failure modes;
* does not lose or corrupt mail when unpredictable things happen, including I/O errors, disk full conditions (to the extent of underlying Perl/file-system/OS ability to detect errors), or a machine crash; the responsibility for mail always stays with MTA;
* does not let mail pass unchecked when unpredictable things happen or when mail is too big (with the exception that a mail size limit for spam checking can be specified); such conditions cause temporary failure to be reported to MTA, mail stays in MTA queue;
* does not load entire mail into memory, so there are no arbitrary size limitations and out-of-memory conditions while transfering, decomposing, virus scanning, quarantining (including SQL quarantining of arbitrary size mail); an exception is mail passed to SpamAssassin, which, due to the way SA works, needs to be in memory, but a size limit can be specified above which SA is not called;
* DKIM and DomainKeys friendly - does not break signed mail, including keeping its integrity in a copy passed to SpamAssassin, so that SA plugins Mail::SpamAssassin::Plugin::DKIM and Mail::SpamAssassin::Plugin::DomainKeys can be reliably used without causing false positives;
* optionally calls one or more anti-virus scanners - the current list includes entries for more than 40 AV scanners and is easily extended, see amavisd.conf file for the list;
* optionally checks MIME types, file names and content types of decoded mail parts (content classifications are provided by a file(1) utility) against a list of banned names and content types;
* optionally checks mail header for invalid characters and some other common violations of rfc2822, and produce informative bounces or notifications;
* optionally calls Perl module Mail::SpamAssassin to check for spam, with optional hard or soft blacklisting/whitelist (regarding spam) of envelope sender address;
* pen pals soft-whitelisting feature (since 2.4.2) reduces spam score on replies to previous correspondence sent from a local user (requires logging to SQL to be enabled);
* may modify mail headers, but it doesn't modify mail body, even if configured to call SpamAssassin (with an exception that the 2.0 defanging feature can wrap original mail into a MIME wrapper). Starting with version of 2.5.0 there is an interface to external utilities altermime or Anomy Sanitizer, which allows for per-recipient sanitation of mail body or adding disclaimers.
* versatile lookup mechanisms (ACL, hash, regexp lists, SQL, LDAP);
* true per-recipient handling of most settings, even for multi-recipient messages;
* modular (package namespaces), yet contained in a single file; only the required modules are compiled;
* adheres tightly to a bunch of RFC specifications - see release notes for details;
* works best with MTAs which interfaces a content filter via SMTP (or LMTP), such as Postfix, Exim v4, or dual (any) MTA setup (all features available, fast), but can also interface with sendmail/milter, Courier, qmail, or other MTAs using helper programs;
* supports client-side LMTP since version 2.5.0, which allows setting up amavisd as a LMTP-to-LMTP filter, feeding a local delivery agent such as Cyrus; note that such a setup does not check outgoing mail and does not allow Pen pals feature to be used, and is only useful for sites with specific needs;
* supports Delivery Status Notifications (DSN) extension to the SMTP protocol (RFC 3461) and notification messages as required by RFC 3462 and RFC 3464 (since 2.4.0);
* quarantining to SQL, to files, or to mailboxes;
* logging to syslog (or a file), and optionally to SQL (where the database is up-to-date and consistent at any time);
* does not duplicate features that a decent MTA or SpamAssassin already provides;
* does not rely on private data structures or private interfaces of a MTA.
Performance
* pre-forked reusable children managed by Net::Server Perl module, saving on process creations and startup latency;
* significant speedups (25% with fast AV scanner compared to amavisd-snapshot-20020300 with the same AV scanner) in SMTP relay configuration; directory and temporary file reuse;
* persistent connections to certain AV scanners, e.g. Sophie and Trophie; or directly callable AV scanner via Perl module to access Sophos engine: SAVI-Perl, or via Mail::ClamAV for ClamAV access;
* cache of the recent body message digests (MD5) -- significant speedup for mailing list bursts, or equal-contents bursts of spam or viruses;
* calls spam scanner (and virus scanners) once per message regardless of the number of recipients, gaining 50% speedup on SA calls on the average for free (depending on the average number of recipients per message), while still obeying most per-recipient settings, splitting mail if necessary;
* when configured to call Mail::SpamAssassin (this is optional), it orders SA to pre-load its config files and to precompile the patterns, so performance is at least as good as with spamc/spamd setup. All Perl modules are pre-loaded by a parent process at startup time, so forked children need not re-compile the code, and can hopefully share some memory for compiled code;
* detailed timing breakdown report for each passed message (with log level 2 or higher);
* tunable number of content filtering processes to operate at peak aggregate mail throughput and without wasting more memory than is useful (must be supported by MTA as well, e.g. with Postfix filter setup or dual-sendmail, not with milter or Postfix proxy);
* supports SMTP and LMTP server-side and client-side service extension PIPELINING (client side since 2.5.0);
* provided utility amavisd-nanny and an associated BerkeleyDB database for a quick-overview monitoring in real time of amavisd child processes;
* provided utility amavisd-agent and an associated BerkeleyDB database to provide access to numerous SNMP-like counters and gauges in real time;
Interfaces to MTA
* accepts mail from MTA either via SMTP (fully rfc2821-compliant), or via LMTP (rfc2033), or through a Unix pipe or inet socket from a helper program;
* forwards mail via SMTP or LMTP over a Unix or INET or INET6 socket, or by a pipe to some external command (sendmail mail submission program, or its look-alike), or by telling a helper program the mail is to be accepted or not (e.g. with sendmail milter); an optional forwarding method can produce BSMTP files;
* in a SMTP or a LMTP relay configuration the amavisd-new can be located on a separate host from the MTA host, and several MTAs may share a single amavisd-new service;
* similarly the sendmail/milter allows amavis_milter + amavisd-new to be separated from the MTA host (in sendmail.mc use: INPUT_MAIL_FILTER(`milter-amavis',`S=inet:port@hostname,F=T,T=S:10m;R:10m;E:10m')dnl and start amavis-milter with -p inet:port@0.0.0.0 instead of -p path_to_unix_socket; thanks to Dibo for the tip);
* pass reject reason from the MTA on the output side back to MTA on the input side;
* proper (non-)delivery status notifications (DSN), compliant with rfc3462 and rfc3464 (ex rfc1892/rfc1894);
* informative MTA log entries, especially in the most common SMTP relay setup;
* amavis internal log id (am_id) reported in log entries, passed to MTA in SMTP response, and included in a DSN notification (since 2.5.0), making easier to match MTA and amavisd log entries;
* supported MTA configurations:
o Postfix supported and thoroughly tested (advanced content filtering model);
o dual-sendmail and other dual-MTA configurations (any MTA type including qmail) with amavisd-new relaying between them (SMTP) is the recommended setup (for speed and flexibility) with other mailers;
o sendmail libmilter interface supported and tested (header rewriting or adding address extensions is only available when using Petr Rehor's amavisd-milter helper program in place of the one provided in the package;
o Exim v4 supported;
o Exim v3 via helper program mostly works, but is not recommended due to deficiencies in the amavis.c helper program or its interfacing with Exim;
o sendmail in the traditional setup with amavis helper program (amavis.c) called as a local delivery agent is still available for compatibility. Please set $forward_method = undef in amavisd.conf if this method is used; the dual-MTA setup with sendmail is preferred;
o since amavisd-new-2.0 a Postfix SMTP extension command XFORWARD is supported on input and output, making available extra information about original SMTP client to amavisd;
o Courier is supported by applying a patch provided in the package;
o qmail interface (qmqpqq) is provided by applying a patch provided in the package;
o Postfix SMTP transparent proxy mode may work for low-traffic sites, but is not a recommended or supported interface method;
User individuality, quarantine
* can specify subgroups of users who want to receive viruses or spam (with alert header fields added, contents optionally pushed to an attachment, notifications can still be generated);
* optionally add address extensions to recipient addresses of mail carrying viruses or spam, facilitating placement of such mail in separate folder at the final delivery stage: e.g. user@domain -> user+spam@domain
* can split a message (by clustering to recipient groups of equal treatment) if not all recipients in a multi-recipient message require the same header insertions/deletions/edits, or the same mail body modifications by external sanitation programs or external programs for adding disclaimers (such external programs are supported since 2.5.0);
* quarantine options: save to individual files, save to a local mailbox file, save to BSMTP files, save to SQL (no arbitrary size limitations or large memory requirements), pass to MTA for delivery to a quarantine mailbox (possibly remote [not advisable with sendmail/milter unless steps are taken to avoid loop]);
* added headers in quarantined messages preserve envelope addresses and quarantine id and facilitate releasing quarantined messages; added headers also identify the reason for quarantining (virus name or spam report);
* provided utility amavisd-release to request (from an amavisd daemon) releasing a message from a quarantine; (or use third-party MailZu for managing a quarantine);
* can suppress quarantining if spam score is above a configured level;
* quarantining can also be specified for passed clean messages, providing an archival
Anti-virus
* optionally decodes/unpacks/decompresses the following formats: MIME, uuencode, xxencode, BinHex, compress, gzip, bzip, bzip2, zip, 7-zip, freeze, lzop, tar, cpio, rpm, deb, rar, arc, arj, zoo, lha(lzh), tnef, ole, cab;
* includes support for approx. 40 AV scanners off-the shelf (see file amavisd.conf, variable @av_scanners, for the list);
* easy support for command-line virus scanners (new entries are easily described and added to the list @av_scanners, file amavisd.conf);
* virus scanners can be split in two lists: @av_scanners and @av_scanners_backup, the second list is only consulted if all primary scanners fail;
* supports daemonized virus scanners (e.g. clamd, Sophie, Trophie, DrWebD, FRISK F-Prot, Panda pavcl -tsr) and scanners accessible via Perl modules (SAVI-Perl, Mail::ClamAV);
* supports specialized scanners (like a jpeg-scanner), which can check for just one aspect of mail validity and can report malware, but do not vouch for other aspects of a message, so as not to preclude other virus scanners from running;
* can specify recipient for whom virus checks need not be performed, fully per-user configurable even with multi-recipient mail;
* can specify recipients who want to receive viruses (with alert header field added, contents optionally pushed to an attachment), fully per-user configurable even with multi-recipient mail;
* optionally add address extensions: e.g. user@domain -> user+virus@domain, if virus detected and desired to be passed; fully per-user configurable even with multi-recipient mail
* can specify final_virus_destiny: reject, bounce, discard, pass (defaults to discard, with optional notification);
* bounce suppression: does not accuse innocent users of sending viruses if sender address is believed to be faked;
Anti-spam
* can favour (soft-whitelist) envelope senders (since 2.0) by lowering computed spam score according to recipient's individual wishes;
* pen pals soft-whitelist: can favour envelope senders (since 2.4.2) by lowering computed spam score if incoming message can be associated (through SQL logging database) with a previous outgoing message from a local user to the current sender; starting with 2.5.0 recognizes also a reference to a Message-ID of a previous mail;
* can blacklist (or soft-blacklist) envelope senders or domains whose messages should be considered spam;
* bypass_spam_checks: can specify users or subdomains (envelope recipients) for whom spam checks need not be performed; with proper per-recipient handling of multi-recipient mail;
* spam_lovers: can specify users or subdomains (envelope recipients) who want to receive spam, with alert header line added, contents optionally pushed to an attachment; with proper per-recipient handling of multi-recipient mail;
* optionally add address extensions (known as plus-addressing): e.g. user@domain -> user+spam@domain, if spam is detected and is desired to be passed; with proper per-recipient handling of multi-recipient mail;
* can specify final_spam_destiny: reject, bounce, discard, pass (defaults to bounce, but DSN is suppressed above a cutoff level);
* can send spam notifications to spam admin, which include the full Mail::SpamAssassin report; these notifications can be restricted to reporting only spam from internal hosts, through the use of policy banks;
* spam headers are inserted on a per-user basis according to their tag/tag2 level settings; this means that a multi-recipient message is split into clusters of recipients with same settings if needed (not available with milter interface). This permits per-recipient individual settings, while still being efficient for multi-recipient messages;
* SpamAssassin check is called only once per message regardless of the number of recipients, all header editing and actions taken is then done by amavisd-new for each recipient individually, based on its settings;
* bounce suppression for high-scoring spam, or disabled completely.
Other
* customizable notification messages based on simple macro expander in the spirit of m4 -- see README.customize;
* versatile lookup mechanisms, described in README.lookups, to be used with virus_lovers, spam_lovers, bypass_checks, local_domains, inet_acl, ... The lookup tables supported are: Perl hash, access control list, Perl regular expressions list, SQL lookups, LDAP lookups;
* non-delivery notification are not sent to mailing lists (mail with Precedence: bulk or list, or with a List-Id (rfc2919) header field);
* meticulous error checking and handling;
* well-commented code;
* policy banks -- since 2.0 the whole set of configuration settings may be switched/overlaid, based on incoming TCP port number or original SMTP client IP address (obtained by XFORWARD Postfix SMTP extension command).
Security considerations
Security considerations for the host running amavisd-new
amavisd-new accepts mail from MTA, may call external Perl modules and may fork external programs to decompress and decode message, classify its content, then the checked mail is either passed to MTA for delivery, or rejected and quarantined.
Any component of a program that comes in contact with unpredictable and possibly malicious mail/document content, must be careful not to let the content have any uncontrolled effect on the operation of the program, or its environment.
amavisd-new is written entirely in Perl, with taint mode Perl checking enabled. This in itself is a strong argument that the processing within amavisd-new (and Perl modules it calls) is not likely to be subject to buffer overruns, stack smashing, and other problems that are common source of security problems in programs written in languages like C.
Information coming from external world like SMTP envelope information, mail header and body contents, suggested MIME file names, etc., is only used by amavisd-new for operations that do not influence the program environment. For example, names of created temporary files are internally generated and do not depend on suggested file names from MIME header. Mail addresses or other tainted information is never passed through shell to an external program.
The external Perl modules called by amavisd-new have not been thoroughly screened for possible security implications. They still benefit from the Perl environment, and the Perl taint mode checking is not turned off even when other Perl modules are executing, including SpamAssassin if enabled (which is a relatively complex piece of software). Perl modules that deal with decoding and checking of mail contents may be targets of malicious mail content, especially if they include code written in C, like decoding and uncompressing libraries, e.g. zlib and uulib/uudeview (Convert::UUlib).
External programs that get forked from amavisd-new to perform some decoding/uncompressing or classifying task, are the greatest potential threat to the safe operation of the host running amavisd-new. Some of these programs that are used to decode certain archive formats are quite complex, are old or poorly maintained, and/or written by less security conscious authors. E.g. a vulnerability is present in Unix utility file(1) version 3.41 or older. Generally it is advised that external programs are kept up-to-date and that crashes of such programs are reported immediately to their maintainers (after verifying first the version is recent).
There is a tradeoff in deciding whether to call some external decoder: calling it may open a vulnerability at the host running amavisd-new; not calling it (and not decoding certain types of document) may cause virus checker to miss a malicious mail contents, increasing danger for the mail recipient, while reducing risk for the host running checks.
While it may be true that only a powered-down computer, locked in a basement and disconnected from the network is completely secure computer, this is not practical to get any job done. Besides choosing a content filtering program to be written in Perl and using taint mode checks, there are other things one may do to reduce security threats to the computer running content filter:
* Never run amavisd-new as root or with other elevated privileges. Use a dedicated username (UID) and group (GID) for the purpose (for example: don't use usernames mail or nobody). Start amavisd-new daemon with su(1) (best), or use command line options -u (and -g) to specify username/group when starting amavisd, or at least specify $daemon_user and $daemon_group in amavisd.conf . Later versions of amavisd-new perform certain checks on its environment and fail to start up if some obvious security flaw is found in current UID of the process or in ownership and protections of the most critical files and directories.
* Protect account under which amavisd-new will run, don't let it have a valid password and disallow interactive logins and remote access.
* Check settings for external programs like $arc, $unarj, $unrar, $zoo, $lha, $lzop, $unfreeze, $uncompress, $gzip, $bzip2, $cpio, $rpm2cpio, $cabextract, and disable those not trusted (amavisd.conf, or just remove them). Make sure external programs, their configuration files (e.g. /usr/share/magic) or directories where they reside, are not writable by a non-privileged (non-root) user. Normally such files should be owned by root. This also applies to external virus checking programs and their database, and external programs that may be called by SpamAssassin (if enabled).
* The same applies to configuration files used by amavisd-new and SpamAssassin, e.g. /etc/amavisd.conf, /etc/mail/spamassassin/*, /usr/share/spamassassin/* . Most importantly these files should not be writable by user (UID or GID) under which amavisd-new is running, they should preferably be owned by root and not (world or group)-writable.
* The directories /var/amavis/tmp, /var/virusmail, /var/amavis/db, /var/amavis/.spamassassin, /var/amavis/.razor ($TEMPBASE, $QUARANTINEDIR, $db_home) must be writable for amavisd-new process, and are commonly owned by user and group under which amavisd-new is running. These directories should not be writable by other non-privileged users. It is advised to keep $TEMPBASE in a separate subdirectory (e.g. /var/amavis/tmp) and not letting temporary files be created in the top-level /var/amavis directory.
* chroot mode of operation is a very powerful security concept in Unix. amavisd-new can work in a chroot environment since amavisd-new-20021116. This requires that all external programs called by amavisd-new can operate in a chroot file system subtree. Preparing a chroot environment including all required programs with their shared libraries, is highly system-dependent. Using only sockets to communicate with the external world (e.g. SMTP, daemonized virus scanners) simplifies the setup. Unfortunately setting up chroot environment for amavisd-new is not for inexperienced Unix administrators. Besides documentation in README.chroot, recommended reading for setting up chrooted environment for amavisd-new and other components is also: Fairly-Secure Anti-SPAM Gateway Using OpenBSD, Postfix, Amavisd-new, SpamAssassin, Razor and DCC, by Scott Vintinner.
* Keep all software up-to-date, especially the external decoders and virus scanners.
* Do regular backups, keep file system signatures (e.g. with Tripwire).
* Choosing a platform less widespread and popular may help: Alpha or Sparc CPU instead of Intel, *BSD or Tru64 or Solaris instead of Linux (not to mention Windows) may help. This may be considered security through obscurity, but any additional obstacle can help.
Security considerations for mail clients being protected
Running a virus checking content filter for each mail before it reaches the mail reader is an important line of defense against virus outbreaks and in protecting the (possibly not security conscious) recipients, or their mail reader programs or computer environment.
Not all malware is passed by e-mail. Several viruses or worms use multiple mechanisms to propagate, including WWW, sharing disks or through peer-to-peer 'contents' sharing, social engineering, or even a memory key or a CD brought-in in a pocket or distributed by magazines and software publishing houses may bring in a virus;
Content filtering mailer can not protect internal hosts unless incoming SMTP (TCP dst port 25) is restricted at the firewall to official mailers only. Similarly external world deserves protection from possibly infected internal hosts, so outgoing SMTP (TCP dst port 25 again, outgoing this time) needs to be restricted to official mailers. (Use standard tcp port 587 for mail submission from roaming users.)
Similarly, if mail readers can fetch mail from external mailboxes (POP3, IMAP), the SMTP mail gateway can not protect them. One solution is to provide a centralized fetchmail service to users that need access to external mailboxes, and feed such mail to the regular content filtering mailer, while blocking other unofficial access to external POP3 and IMAP servers at a firewall.
Even in e-mail, malware may be carried in encrypted or scrambled form, or simply as a plain text, using social engineering techniques to persuade recipient to fetch or activate malware.
It is not possible to prevent user shooting himself in the foot, or to prevent a dedicated person to transfer malware. There is a tradeoff in keeping e-mail useful, and protecting against threats.
The first line of defense (mail content filtering, firewall) must be complemented by defense mechanisms at the local user's desktop computer. This includes virus scanners run on PCs, keeping software up-to-date, doing backups, and educating users.
Malware does not have to play by the rules. Nothing prevents malware from generating a syntactically incorrect mail, to send it directly to some host ignoring MX and A records, to supply forged SMTP information or forged mail header, to poison DNS, perhaps even to use forged source IP address.
Content filter with virus scanner tries to decide if the mail under consideration will, or can, cause any bad effects on the recipient computer, often without knowing what mail reading software or what computer is used by recipients. This implies that while some mail may be decoded (by adhering to standards) into a harmless text, it might be decoded by some broken MUA or archiver into a virus or exploit, or trigger a MUA bug or vulnerability during decoding, or during displaying a message. External archivers/unpackers called by amavisd-new may be relatively easy to trick into not extracting certain archive members, thus hiding malicious code. See Malformed email project, Bypassing content filtering whitepaper, Declude's list of vulnerabilities, NISCC Vulnerability Advisory 380375/MIME. CAN-2003-1015
Solving this problem would require content filter with virus scanner to emulate all known (and unknown?!) mail readers in the way they respond to malformed mail. While amavisd-new and other content filters try to anticipate some common problems, especially the ones practiced by currently active viruses, there is no guarantee that this approach is always successful.
Even now there are combinations of viruses and virus scanners (e.g. Yaha.K + Sophos) that fail to be detected due to a malformed MIME header, which gets decoded differently (and correctly, considering standards!) by MIME::Parser, yet certain mail readers decode it differently, forming a virus. It often helps to use more than one virus scanner (e.g. clamd along with some commercial virus scanner).
RFC 2046 defines a way to split sending one document into several e-mail messages, which can then be reassembled (automatically or manually) by MUA. The Content-Type value to look for is message/partial (and similarly: message/external-body). Checking mail fragments individually for viruses can not reliably detect viruses, which only get reassembled into a recognizable form by the recipient's mail reader. Most virus scanners at the MTA level (including amavisd-new and all other variants of AMaViS*) check each mail independently from other messages, so the only protection to this threat is to ban these MIME content-types (see $banned_filename_re setting in amavisd.conf), or by disabling auto-reassembly at mail readers, or running a virus checker tightly associated with MUA.
Blocking the MIME content type message/external-body may sound useful, although the mechanism is not much different from letting user freely browse the web or fully interpret HTML mail messages, so if the later is allowed, it probably does not make sense to treat message/external-body differently.
Protection against denial-of-service (DoS) attacks
Because amavisd-new tries to recursively unpack and decode each mail as deeply as possible, this may be abused by malware. The so-called mail bomb, e.g. 42.zip or bzip2 bomb are examples of such malware. Such mail message, when fully decoded, can exceed available disk size several times, or consume a lot of time for decoding. Unless decoding is stopped at an earlier stage, it could cause the message checking to be retried over and over again, each time either hitting the disk full condition, or exceeding the allowed time limit. Note that mail bombs are targeting mail content filters, and are normally not a threat to mail clients (MUA), unless they carry a virus as well.
amavisd-new has several configurable mechanism for limiting the amount of space consumed during decoding - see resource limits in file amavisd.conf. When message decoding exceeds the storage quota, the decoding stops, the virus scanning is not performed to protect the virus scanner, but a header field is inserted, telling MTA it may place the message 'on hold', or reject it, or just pass it - the action depends on MTA configuration. This works well with Postfix, but may not be configurable with some other mailers.
Since amavisd-new-20030616-p8 a string '***UNCHECKED***' is inserted into Subject. Since amavisd-new-2.0 such passed mail is also wrapped into a MIME wrapper (defanged), prepended by a warning message.
See also the AERAsec advisory on decompression bomb vulnerabilities.
Protection against mail loss
When a content filter is positioned in relation to MTA as a mail relay (or proxy), accepting mail from MTA, and passing all checked mail to MTA for final delivery (e.g. Postfix, or dual-sendmail setup), there are only two possible approaches that can prevent mail loss when unpredictable things happen:
* committing messages to secondary storage and keeping evidence, e.g. properly implementing a queueing mechanism, as MTAs do it, or
* confirming mail reception to the input-side MTA only after the forwarded mail reception has been confirmed by the MTA on the output side (or a non-delivery notification sent). This approach is used by transparent SMTP proxy, or by cooperating SMTP server and client.
amavisd-new chooses the second approach. Some alternative mail content filtering solutions based on Perl module Net::Server::SMTP can not guarantee not losing mail under certain circumstances, because they confirm mail reception before being in a position to ensure a mail delivery or bounce.
Besides taking care of not losing mail, it is important the mail contents is not unintentionally changed, as could happen for example when disk is full, or communication or I/O errors occur. amavisd-new is thorough in always checking the status of operations, e.g. all I/O operations, creating/deleting/writing to files, calling external programs, checking all SMTP response codes, etc. When a problem occur, amavisd-new tries to produce an error report in its log file that is as informative as practical. When the situation can not be corrected, a temporary failure (EX_TEMPFAIL, or 450 SMTP response) is generated, telling MTA to try again later, hoping for the postmaster to notice stuck mail if the problem keeps reoccurring.
Do you consider this article interesting? Share it on your network of Twitter contacts, on your Facebook wall or simply press "+1" to suggest this result in searches in Google, Linkedin, Instagram or Pinterest. Spreading content that you find relevant helps this blog to grow. Thank you!
It is written in Perl for maintainability, without paying a significant price for speed. It talks to MTA via (E)SMTP or LMTP, or by using helper programs.
Best with Postfix, fine with dual-sendmail setup and Exim v4, works with sendmail/milter, or with any MTA as a SMTP relay. For Courier and qmail MTA integration there is a patch in the distributed package.
amavisd-new is a high-performance and reliable interface between mailer (MTA) and one or more content checkers: virus scanners, and/or Mail::SpamAssassin Perl module. It is written in Perl, ensuring high reliability, portability and maintainability. It talks to MTA via (E)SMTP or LMTP protocols, or by using helper programs. No timing gaps exist in the design, which could cause a mail loss.
It is normally positioned at or near a central mailer, not necessarily where users' mailboxes and final delivery takes place. If looking for a per-user and low-message-rate solution to be placed at the final stage of mail delivery (e.g. called from procmail or in place of a local delivery agent), there may be other solutions more appropriate.
When calling of Mail::SpamAssassin (SA) is enabled, it calls SA only once per message regardless of the number of recipients, and tries very hard to correctly honour per-recipient preferences, such as pass/reject, check/nocheck, spam levels, and inserting spam-related mail header fields.
amavisd-new benefits from the use of Perl module Net::Server, which offers a fast pre-forked multichild process control. amavisd-new provides rfc2821-compliant SMTP server and client, a rfc2033-compliant LMTP server and client, and generates rfc3462/rfc3464-compliant (ex rfc1892/rfc1894) delivery (and non-delivery) status notifications.
This makes it suitable for mail anti-virus and/or anti-spam checking on a busy mail gateways that care for reliability and standards compliance.
amavisd-new grew out of amavisd(-snapshot) (which in turn is a daemonized version of amavis-perl), but through five years of development turned into a separate product, hardly resembling its origin. The code is several times the size of its predecessor, yet faster in throughput, richer in features, compliant to standards, includes optional support for spam detection, and makes virus scanning optional and easier to adjust/extend. Compatibility with helper programs from amavisd(-snapshot) is retained.
All modifications since the original amavisd done by Mark Martinec, with contribution of ideas, patches and reports from the amavis-user mailing list community and individuals.
-- general
* written in Perl for better maintainability, portability, no risk of buffer overruns or invalid pointers; critical code paths are well optimized;
* a daemonized pre-forking server, saving on startup time; designed with high throughput in mind;
* security conscious, runs under a dedicated user id (not root and not mail), can run in a chroot jail (including SpamAssassin and external programs);
* thorough error checking, informative error reporting, fail-safe failure modes;
* does not lose or corrupt mail when unpredictable things happen, including I/O errors, disk full conditions (to the extent of underlying Perl/file-system/OS ability to detect errors), or a machine crash; the responsibility for mail always stays with MTA;
* does not let mail pass unchecked when unpredictable things happen or when mail is too big (with the exception that a mail size limit for spam checking can be specified); such conditions cause temporary failure to be reported to MTA, mail stays in MTA queue;
* does not load entire mail into memory, so there are no arbitrary size limitations and out-of-memory conditions while transfering, decomposing, virus scanning, quarantining (including SQL quarantining of arbitrary size mail); an exception is mail passed to SpamAssassin, which, due to the way SA works, needs to be in memory, but a size limit can be specified above which SA is not called;
* DKIM and DomainKeys friendly - does not break signed mail, including keeping its integrity in a copy passed to SpamAssassin, so that SA plugins Mail::SpamAssassin::Plugin::DKIM and Mail::SpamAssassin::Plugin::DomainKeys can be reliably used without causing false positives;
* optionally calls one or more anti-virus scanners - the current list includes entries for more than 40 AV scanners and is easily extended, see amavisd.conf file for the list;
* optionally checks MIME types, file names and content types of decoded mail parts (content classifications are provided by a file(1) utility) against a list of banned names and content types;
* optionally checks mail header for invalid characters and some other common violations of rfc2822, and produce informative bounces or notifications;
* optionally calls Perl module Mail::SpamAssassin to check for spam, with optional hard or soft blacklisting/whitelist (regarding spam) of envelope sender address;
* pen pals soft-whitelisting feature (since 2.4.2) reduces spam score on replies to previous correspondence sent from a local user (requires logging to SQL to be enabled);
* may modify mail headers, but it doesn't modify mail body, even if configured to call SpamAssassin (with an exception that the 2.0 defanging feature can wrap original mail into a MIME wrapper). Starting with version of 2.5.0 there is an interface to external utilities altermime or Anomy Sanitizer, which allows for per-recipient sanitation of mail body or adding disclaimers.
* versatile lookup mechanisms (ACL, hash, regexp lists, SQL, LDAP);
* true per-recipient handling of most settings, even for multi-recipient messages;
* modular (package namespaces), yet contained in a single file; only the required modules are compiled;
* adheres tightly to a bunch of RFC specifications - see release notes for details;
* works best with MTAs which interfaces a content filter via SMTP (or LMTP), such as Postfix, Exim v4, or dual (any) MTA setup (all features available, fast), but can also interface with sendmail/milter, Courier, qmail, or other MTAs using helper programs;
* supports client-side LMTP since version 2.5.0, which allows setting up amavisd as a LMTP-to-LMTP filter, feeding a local delivery agent such as Cyrus; note that such a setup does not check outgoing mail and does not allow Pen pals feature to be used, and is only useful for sites with specific needs;
* supports Delivery Status Notifications (DSN) extension to the SMTP protocol (RFC 3461) and notification messages as required by RFC 3462 and RFC 3464 (since 2.4.0);
* quarantining to SQL, to files, or to mailboxes;
* logging to syslog (or a file), and optionally to SQL (where the database is up-to-date and consistent at any time);
* does not duplicate features that a decent MTA or SpamAssassin already provides;
* does not rely on private data structures or private interfaces of a MTA.
* pre-forked reusable children managed by Net::Server Perl module, saving on process creations and startup latency;
* significant speedups (25% with fast AV scanner compared to amavisd-snapshot-20020300 with the same AV scanner) in SMTP relay configuration; directory and temporary file reuse;
* persistent connections to certain AV scanners, e.g. Sophie and Trophie; or directly callable AV scanner via Perl module to access Sophos engine: SAVI-Perl, or via Mail::ClamAV for ClamAV access;
* cache of the recent body message digests (MD5) -- significant speedup for mailing list bursts, or equal-contents bursts of spam or viruses;
* calls spam scanner (and virus scanners) once per message regardless of the number of recipients, gaining 50% speedup on SA calls on the average for free (depending on the average number of recipients per message), while still obeying most per-recipient settings, splitting mail if necessary;
* when configured to call Mail::SpamAssassin (this is optional), it orders SA to pre-load its config files and to precompile the patterns, so performance is at least as good as with spamc/spamd setup. All Perl modules are pre-loaded by a parent process at startup time, so forked children need not re-compile the code, and can hopefully share some memory for compiled code;
* detailed timing breakdown report for each passed message (with log level 2 or higher);
* tunable number of content filtering processes to operate at peak aggregate mail throughput and without wasting more memory than is useful (must be supported by MTA as well, e.g. with Postfix filter setup or dual-sendmail, not with milter or Postfix proxy);
* supports SMTP and LMTP server-side and client-side service extension PIPELINING (client side since 2.5.0);
* provided utility amavisd-nanny and an associated BerkeleyDB database for a quick-overview monitoring in real time of amavisd child processes;
* provided utility amavisd-agent and an associated BerkeleyDB database to provide access to numerous SNMP-like counters and gauges in real time;
Interfaces to MTA
* accepts mail from MTA either via SMTP (fully rfc2821-compliant), or via LMTP (rfc2033), or through a Unix pipe or inet socket from a helper program;
* forwards mail via SMTP or LMTP over a Unix or INET or INET6 socket, or by a pipe to some external command (sendmail mail submission program, or its look-alike), or by telling a helper program the mail is to be accepted or not (e.g. with sendmail milter); an optional forwarding method can produce BSMTP files;
* in a SMTP or a LMTP relay configuration the amavisd-new can be located on a separate host from the MTA host, and several MTAs may share a single amavisd-new service;
* similarly the sendmail/milter allows amavis_milter + amavisd-new to be separated from the MTA host (in sendmail.mc use: INPUT_MAIL_FILTER(`milter-amavis',`S=inet:port@hostname,F=T,T=S:10m;R:10m;E:10m')dnl and start amavis-milter with -p inet:port@0.0.0.0 instead of -p path_to_unix_socket; thanks to Dibo for the tip);
* pass reject reason from the MTA on the output side back to MTA on the input side;
* proper (non-)delivery status notifications (DSN), compliant with rfc3462 and rfc3464 (ex rfc1892/rfc1894);
* informative MTA log entries, especially in the most common SMTP relay setup;
* amavis internal log id (am_id) reported in log entries, passed to MTA in SMTP response, and included in a DSN notification (since 2.5.0), making easier to match MTA and amavisd log entries;
o Postfix supported and thoroughly tested (advanced content filtering model);
o dual-sendmail and other dual-MTA configurations (any MTA type including qmail) with amavisd-new relaying between them (SMTP) is the recommended setup (for speed and flexibility) with other mailers;
o sendmail libmilter interface supported and tested (header rewriting or adding address extensions is only available when using Petr Rehor's amavisd-milter helper program in place of the one provided in the package;
o Exim v4 supported;
o Exim v3 via helper program mostly works, but is not recommended due to deficiencies in the amavis.c helper program or its interfacing with Exim;
o sendmail in the traditional setup with amavis helper program (amavis.c) called as a local delivery agent is still available for compatibility. Please set $forward_method = undef in amavisd.conf if this method is used; the dual-MTA setup with sendmail is preferred;
o since amavisd-new-2.0 a Postfix SMTP extension command XFORWARD is supported on input and output, making available extra information about original SMTP client to amavisd;
o Courier is supported by applying a patch provided in the package;
o qmail interface (qmqpqq) is provided by applying a patch provided in the package;
o Postfix SMTP transparent proxy mode may work for low-traffic sites, but is not a recommended or supported interface method;
User individuality, quarantine
* can specify subgroups of users who want to receive viruses or spam (with alert header fields added, contents optionally pushed to an attachment, notifications can still be generated);
* optionally add address extensions to recipient addresses of mail carrying viruses or spam, facilitating placement of such mail in separate folder at the final delivery stage: e.g. user@domain -> user+spam@domain
* can split a message (by clustering to recipient groups of equal treatment) if not all recipients in a multi-recipient message require the same header insertions/deletions/edits, or the same mail body modifications by external sanitation programs or external programs for adding disclaimers (such external programs are supported since 2.5.0);
* quarantine options: save to individual files, save to a local mailbox file, save to BSMTP files, save to SQL (no arbitrary size limitations or large memory requirements), pass to MTA for delivery to a quarantine mailbox (possibly remote [not advisable with sendmail/milter unless steps are taken to avoid loop]);
* added headers in quarantined messages preserve envelope addresses and quarantine id and facilitate releasing quarantined messages; added headers also identify the reason for quarantining (virus name or spam report);
* provided utility amavisd-release to request (from an amavisd daemon) releasing a message from a quarantine; (or use third-party MailZu for managing a quarantine);
* can suppress quarantining if spam score is above a configured level;
* quarantining can also be specified for passed clean messages, providing an archival
* optionally decodes/unpacks/decompresses the following formats: MIME, uuencode, xxencode, BinHex, compress, gzip, bzip, bzip2, zip, 7-zip, freeze, lzop, tar, cpio, rpm, deb, rar, arc, arj, zoo, lha(lzh), tnef, ole, cab;
* includes support for approx. 40 AV scanners off-the shelf (see file amavisd.conf, variable @av_scanners, for the list);
* easy support for command-line virus scanners (new entries are easily described and added to the list @av_scanners, file amavisd.conf);
* virus scanners can be split in two lists: @av_scanners and @av_scanners_backup, the second list is only consulted if all primary scanners fail;
* supports daemonized virus scanners (e.g. clamd, Sophie, Trophie, DrWebD, FRISK F-Prot, Panda pavcl -tsr) and scanners accessible via Perl modules (SAVI-Perl, Mail::ClamAV);
* supports specialized scanners (like a jpeg-scanner), which can check for just one aspect of mail validity and can report malware, but do not vouch for other aspects of a message, so as not to preclude other virus scanners from running;
* can specify recipient for whom virus checks need not be performed, fully per-user configurable even with multi-recipient mail;
* can specify recipients who want to receive viruses (with alert header field added, contents optionally pushed to an attachment), fully per-user configurable even with multi-recipient mail;
* optionally add address extensions: e.g. user@domain -> user+virus@domain, if virus detected and desired to be passed; fully per-user configurable even with multi-recipient mail
* can specify final_virus_destiny: reject, bounce, discard, pass (defaults to discard, with optional notification);
* bounce suppression: does not accuse innocent users of sending viruses if sender address is believed to be faked;
Anti-spam
* can favour (soft-whitelist) envelope senders (since 2.0) by lowering computed spam score according to recipient's individual wishes;
* pen pals soft-whitelist: can favour envelope senders (since 2.4.2) by lowering computed spam score if incoming message can be associated (through SQL logging database) with a previous outgoing message from a local user to the current sender; starting with 2.5.0 recognizes also a reference to a Message-ID of a previous mail;
* can blacklist (or soft-blacklist) envelope senders or domains whose messages should be considered spam;
* bypass_spam_checks: can specify users or subdomains (envelope recipients) for whom spam checks need not be performed; with proper per-recipient handling of multi-recipient mail;
* spam_lovers: can specify users or subdomains (envelope recipients) who want to receive spam, with alert header line added, contents optionally pushed to an attachment; with proper per-recipient handling of multi-recipient mail;
* optionally add address extensions (known as plus-addressing): e.g. user@domain -> user+spam@domain, if spam is detected and is desired to be passed; with proper per-recipient handling of multi-recipient mail;
* can specify final_spam_destiny: reject, bounce, discard, pass (defaults to bounce, but DSN is suppressed above a cutoff level);
* can send spam notifications to spam admin, which include the full Mail::SpamAssassin report; these notifications can be restricted to reporting only spam from internal hosts, through the use of policy banks;
* spam headers are inserted on a per-user basis according to their tag/tag2 level settings; this means that a multi-recipient message is split into clusters of recipients with same settings if needed (not available with milter interface). This permits per-recipient individual settings, while still being efficient for multi-recipient messages;
* SpamAssassin check is called only once per message regardless of the number of recipients, all header editing and actions taken is then done by amavisd-new for each recipient individually, based on its settings;
* bounce suppression for high-scoring spam, or disabled completely.
Other
* customizable notification messages based on simple macro expander in the spirit of m4 -- see README.customize;
* versatile lookup mechanisms, described in README.lookups, to be used with virus_lovers, spam_lovers, bypass_checks, local_domains, inet_acl, ... The lookup tables supported are: Perl hash, access control list, Perl regular expressions list, SQL lookups, LDAP lookups;
* non-delivery notification are not sent to mailing lists (mail with Precedence: bulk or list, or with a List-Id (rfc2919) header field);
* meticulous error checking and handling;
* well-commented code;
* policy banks -- since 2.0 the whole set of configuration settings may be switched/overlaid, based on incoming TCP port number or original SMTP client IP address (obtained by XFORWARD Postfix SMTP extension command).
Security considerations
Security considerations for the host running amavisd-new
amavisd-new accepts mail from MTA, may call external Perl modules and may fork external programs to decompress and decode message, classify its content, then the checked mail is either passed to MTA for delivery, or rejected and quarantined.
Any component of a program that comes in contact with unpredictable and possibly malicious mail/document content, must be careful not to let the content have any uncontrolled effect on the operation of the program, or its environment.
amavisd-new is written entirely in Perl, with taint mode Perl checking enabled. This in itself is a strong argument that the processing within amavisd-new (and Perl modules it calls) is not likely to be subject to buffer overruns, stack smashing, and other problems that are common source of security problems in programs written in languages like C.
Information coming from external world like SMTP envelope information, mail header and body contents, suggested MIME file names, etc., is only used by amavisd-new for operations that do not influence the program environment. For example, names of created temporary files are internally generated and do not depend on suggested file names from MIME header. Mail addresses or other tainted information is never passed through shell to an external program.
The external Perl modules called by amavisd-new have not been thoroughly screened for possible security implications. They still benefit from the Perl environment, and the Perl taint mode checking is not turned off even when other Perl modules are executing, including SpamAssassin if enabled (which is a relatively complex piece of software). Perl modules that deal with decoding and checking of mail contents may be targets of malicious mail content, especially if they include code written in C, like decoding and uncompressing libraries, e.g. zlib and uulib/uudeview (Convert::UUlib).
External programs that get forked from amavisd-new to perform some decoding/uncompressing or classifying task, are the greatest potential threat to the safe operation of the host running amavisd-new. Some of these programs that are used to decode certain archive formats are quite complex, are old or poorly maintained, and/or written by less security conscious authors. E.g. a vulnerability is present in Unix utility file(1) version 3.41 or older. Generally it is advised that external programs are kept up-to-date and that crashes of such programs are reported immediately to their maintainers (after verifying first the version is recent).
There is a tradeoff in deciding whether to call some external decoder: calling it may open a vulnerability at the host running amavisd-new; not calling it (and not decoding certain types of document) may cause virus checker to miss a malicious mail contents, increasing danger for the mail recipient, while reducing risk for the host running checks.
While it may be true that only a powered-down computer, locked in a basement and disconnected from the network is completely secure computer, this is not practical to get any job done. Besides choosing a content filtering program to be written in Perl and using taint mode checks, there are other things one may do to reduce security threats to the computer running content filter:
* Never run amavisd-new as root or with other elevated privileges. Use a dedicated username (UID) and group (GID) for the purpose (for example: don't use usernames mail or nobody). Start amavisd-new daemon with su(1) (best), or use command line options -u (and -g) to specify username/group when starting amavisd, or at least specify $daemon_user and $daemon_group in amavisd.conf . Later versions of amavisd-new perform certain checks on its environment and fail to start up if some obvious security flaw is found in current UID of the process or in ownership and protections of the most critical files and directories.
* Protect account under which amavisd-new will run, don't let it have a valid password and disallow interactive logins and remote access.
* Check settings for external programs like $arc, $unarj, $unrar, $zoo, $lha, $lzop, $unfreeze, $uncompress, $gzip, $bzip2, $cpio, $rpm2cpio, $cabextract, and disable those not trusted (amavisd.conf, or just remove them). Make sure external programs, their configuration files (e.g. /usr/share/magic) or directories where they reside, are not writable by a non-privileged (non-root) user. Normally such files should be owned by root. This also applies to external virus checking programs and their database, and external programs that may be called by SpamAssassin (if enabled).
* The same applies to configuration files used by amavisd-new and SpamAssassin, e.g. /etc/amavisd.conf, /etc/mail/spamassassin/*, /usr/share/spamassassin/* . Most importantly these files should not be writable by user (UID or GID) under which amavisd-new is running, they should preferably be owned by root and not (world or group)-writable.
* The directories /var/amavis/tmp, /var/virusmail, /var/amavis/db, /var/amavis/.spamassassin, /var/amavis/.razor ($TEMPBASE, $QUARANTINEDIR, $db_home) must be writable for amavisd-new process, and are commonly owned by user and group under which amavisd-new is running. These directories should not be writable by other non-privileged users. It is advised to keep $TEMPBASE in a separate subdirectory (e.g. /var/amavis/tmp) and not letting temporary files be created in the top-level /var/amavis directory.
* chroot mode of operation is a very powerful security concept in Unix. amavisd-new can work in a chroot environment since amavisd-new-20021116. This requires that all external programs called by amavisd-new can operate in a chroot file system subtree. Preparing a chroot environment including all required programs with their shared libraries, is highly system-dependent. Using only sockets to communicate with the external world (e.g. SMTP, daemonized virus scanners) simplifies the setup. Unfortunately setting up chroot environment for amavisd-new is not for inexperienced Unix administrators. Besides documentation in README.chroot, recommended reading for setting up chrooted environment for amavisd-new and other components is also: Fairly-Secure Anti-SPAM Gateway Using OpenBSD, Postfix, Amavisd-new, SpamAssassin, Razor and DCC, by Scott Vintinner.
* Keep all software up-to-date, especially the external decoders and virus scanners.
* Do regular backups, keep file system signatures (e.g. with Tripwire).
* Choosing a platform less widespread and popular may help: Alpha or Sparc CPU instead of Intel, *BSD or Tru64 or Solaris instead of Linux (not to mention Windows) may help. This may be considered security through obscurity, but any additional obstacle can help.
Security considerations for mail clients being protected
Running a virus checking content filter for each mail before it reaches the mail reader is an important line of defense against virus outbreaks and in protecting the (possibly not security conscious) recipients, or their mail reader programs or computer environment.
Not all malware is passed by e-mail. Several viruses or worms use multiple mechanisms to propagate, including WWW, sharing disks or through peer-to-peer 'contents' sharing, social engineering, or even a memory key or a CD brought-in in a pocket or distributed by magazines and software publishing houses may bring in a virus;
Content filtering mailer can not protect internal hosts unless incoming SMTP (TCP dst port 25) is restricted at the firewall to official mailers only. Similarly external world deserves protection from possibly infected internal hosts, so outgoing SMTP (TCP dst port 25 again, outgoing this time) needs to be restricted to official mailers. (Use standard tcp port 587 for mail submission from roaming users.)
Similarly, if mail readers can fetch mail from external mailboxes (POP3, IMAP), the SMTP mail gateway can not protect them. One solution is to provide a centralized fetchmail service to users that need access to external mailboxes, and feed such mail to the regular content filtering mailer, while blocking other unofficial access to external POP3 and IMAP servers at a firewall.
Even in e-mail, malware may be carried in encrypted or scrambled form, or simply as a plain text, using social engineering techniques to persuade recipient to fetch or activate malware.
It is not possible to prevent user shooting himself in the foot, or to prevent a dedicated person to transfer malware. There is a tradeoff in keeping e-mail useful, and protecting against threats.
The first line of defense (mail content filtering, firewall) must be complemented by defense mechanisms at the local user's desktop computer. This includes virus scanners run on PCs, keeping software up-to-date, doing backups, and educating users.
Malware does not have to play by the rules. Nothing prevents malware from generating a syntactically incorrect mail, to send it directly to some host ignoring MX and A records, to supply forged SMTP information or forged mail header, to poison DNS, perhaps even to use forged source IP address.
Content filter with virus scanner tries to decide if the mail under consideration will, or can, cause any bad effects on the recipient computer, often without knowing what mail reading software or what computer is used by recipients. This implies that while some mail may be decoded (by adhering to standards) into a harmless text, it might be decoded by some broken MUA or archiver into a virus or exploit, or trigger a MUA bug or vulnerability during decoding, or during displaying a message. External archivers/unpackers called by amavisd-new may be relatively easy to trick into not extracting certain archive members, thus hiding malicious code. See Malformed email project, Bypassing content filtering whitepaper, Declude's list of vulnerabilities, NISCC Vulnerability Advisory 380375/MIME. CAN-2003-1015
Solving this problem would require content filter with virus scanner to emulate all known (and unknown?!) mail readers in the way they respond to malformed mail. While amavisd-new and other content filters try to anticipate some common problems, especially the ones practiced by currently active viruses, there is no guarantee that this approach is always successful.
Even now there are combinations of viruses and virus scanners (e.g. Yaha.K + Sophos) that fail to be detected due to a malformed MIME header, which gets decoded differently (and correctly, considering standards!) by MIME::Parser, yet certain mail readers decode it differently, forming a virus. It often helps to use more than one virus scanner (e.g. clamd along with some commercial virus scanner).
Blocking the MIME content type message/external-body may sound useful, although the mechanism is not much different from letting user freely browse the web or fully interpret HTML mail messages, so if the later is allowed, it probably does not make sense to treat message/external-body differently.
Protection against denial-of-service (DoS) attacks
Because amavisd-new tries to recursively unpack and decode each mail as deeply as possible, this may be abused by malware. The so-called mail bomb, e.g. 42.zip or bzip2 bomb are examples of such malware. Such mail message, when fully decoded, can exceed available disk size several times, or consume a lot of time for decoding. Unless decoding is stopped at an earlier stage, it could cause the message checking to be retried over and over again, each time either hitting the disk full condition, or exceeding the allowed time limit. Note that mail bombs are targeting mail content filters, and are normally not a threat to mail clients (MUA), unless they carry a virus as well.
amavisd-new has several configurable mechanism for limiting the amount of space consumed during decoding - see resource limits in file amavisd.conf. When message decoding exceeds the storage quota, the decoding stops, the virus scanning is not performed to protect the virus scanner, but a header field is inserted, telling MTA it may place the message 'on hold', or reject it, or just pass it - the action depends on MTA configuration. This works well with Postfix, but may not be configurable with some other mailers.
Since amavisd-new-20030616-p8 a string '***UNCHECKED***' is inserted into Subject. Since amavisd-new-2.0 such passed mail is also wrapped into a MIME wrapper (defanged), prepended by a warning message.
See also the AERAsec advisory on decompression bomb vulnerabilities.
Protection against mail loss
When a content filter is positioned in relation to MTA as a mail relay (or proxy), accepting mail from MTA, and passing all checked mail to MTA for final delivery (e.g. Postfix, or dual-sendmail setup), there are only two possible approaches that can prevent mail loss when unpredictable things happen:
* committing messages to secondary storage and keeping evidence, e.g. properly implementing a queueing mechanism, as MTAs do it, or
* confirming mail reception to the input-side MTA only after the forwarded mail reception has been confirmed by the MTA on the output side (or a non-delivery notification sent). This approach is used by transparent SMTP proxy, or by cooperating SMTP server and client.
amavisd-new chooses the second approach. Some alternative mail content filtering solutions based on Perl module Net::Server::SMTP can not guarantee not losing mail under certain circumstances, because they confirm mail reception before being in a position to ensure a mail delivery or bounce.
0 comments:
Post a Comment