ServWise Home
Howto: Reduce Spam using SpamAssassin Bayesian filters in cPanel
May 14th, 2007Introduction
While this article is intended to demonstrate how to train and use Bayesian filters on cPanel hosted sites, you can use the information in this technique wherever spamassassin and fetchmail are installed.
Requirements
- cPanel based host
- Spamassasin
- Access to the cron scheduler within cPanel
- Access to the file manager within cPanel
- Fetchmail must be installed and usable on your hosted server
Audience
This article assumes a working knowledge of cPanel, file editing, and email protocols. It is useful for anyone wanting to increase the accuracy of SpamAssassin’s spam detection.
SpamAssassin
SpamAssassin inspects emails being delivered to your email account looking for signs that the email is spam. It has a number signatures that it looks for, and each of these can be given a weighting, which results in the email having a spam “scoreâ€. If the score is over a threshold, the email is considered spam. There are a number of actions that can be taken if the email is designated as spam, one of which is to alter the Subject line of the email to reflect it is spam, so that a rule can be defined in your email client to automatically delete the email or move it to a junk email folder.
Bayesian Filters
Bayesian filters are a method that SpamAssassin can use to learn more about the kinds of email you receive and so can enhance its ability identify which emails are spam, and which are not. This also helps reduce the “false-positive†count, where an email is marked as spam when it should not have been.
These filters work by identifying key elements of emails that are common to spam, and those that are common to non-spam email. To achieve this, the filters must be given a sample of both to learn from, and requires at a minimum 200 spam emails and 200 non-spam emails. Unfortunately, for many, obtaining 200 spam emails is a trivial task.
The Setup
This article assumes that you will be using a separate email address for spam training. This can make things easier to manage, especially if you use POP3 as your primary means for downloading email.
The basic steps are as follows:
- Enable and configure SpamAssassin
- Create Spam email account with a non-spam email folder
- Configure .fetchmailrc to allow SpamAssassin to download emails
- Configure cron to automatically train SpamAssassin
- Copy 200 spam emails to Spam inbox, and 200 non-spam emails to non-Spam folder
Enable SpamAssassin
- Navigate to the Mail / SpamAssassin menu in cPanel
- Click Enable SpamAssassin. This will enable SpamAssassin on any incoming email. Note that this will already accurately identify spam, however the remainder of the article shows how to increase SpamAssassin’s accuracy.
- Click Configure SpamAssassin
- Ensure the configuration is as follows:
add_header: all
This adds headers emails that will enable you to see why an email has been classed as spam or non-spam
report_safe: 0
This makes changes to the email “in situâ€. If you would prefer that the email is left unaltered, set this option to “1â€. A new email will be created with the spam details in the content, with the original unaltered email as an attachment
required_score: 5
SpamAssassin uses a series of criteria to determine if an email is spam, and each is given a score. The email will be classed as spam if the determined score is above 5
rewrite_header subject: **SPAM** _HITS_ (_BAYES_)
This will change the subject line of the email. The email subject line will be prefixed with the above line, with _HITS_ replace with the spam score for the email, and _BAYES_ replaced with the Bayesian score
score BAYES_99: 5
This states that any email that has a 99% or above rating as spam due to Bayesian analysis, then add five “spam†points to its score. Note that this will put the score of the email over the 5 point threshold defined above, and so automatically marks the email as spam
- 5) Save your settings
Create Spam Account
This email address will be used to train SpamAssassin; it is not intended that this account be used for sending or receiving normal emails.
- Navigate to Mail / Manage/Add/Remove Accounts
- Click Add Account
-
Create a new account with an appropriate name, such as spam@mydomain.com, and define a password. SpamAssassin will empty this account on a regular basis, so it is not necessary to define a quota.
-
Click Back and you should see a list of email accounts; click Read Webmail for the spam account you just created. Enter the password for the account, and you will be taken to the webmail selection page. Choose your preferred webmail client.
-
Create a new folder called “nonspamâ€.
Configure Fetchmail
SpamAssassin uses fetchmail to read the emails from the spam account to train its Bayesian filters.
-
Navigate to File Manager in cPanel
-
You will be presented with a list of folders, followed by a list of files. At the bottom of the folder list, click Create New File
-
On the right hand side, type .fetchmailrc in the dialog box, and leave the document type as Text Document
-
The .fetchmailrc file should appear in the file list in the left hand panel. Click the filename and then select Edit File from the menu on the right hand side.
-
Complete the file as follows, using the username and password you selected for your spam account above:
poll 127.0.0.1
with protocol imap
username spam+mydomain.com password spampassword is spam
-
Save the file
Note that the username is the email address selected, with the @ sign replaced with a +
Configure cron to Automatically Train Spamassassin
Cron is a task scheduler, and so can carry out tasks at specific times of the day. We will configure cron to run the SpamAssassin training process for spam and non-spam once a day.
-
Navigate to Cron jobs in cPanel
-
Click Advanced
-
Each time cron runs, it can send an email to an administrator to report the results of each task. Enter an appropriate email address in the box provided
-
First we want SpamAssassin to train for non-spam, so put the time that you want this task to run at by entering the minutes of the time in the first box, and the hour in the second box. So to have the non-spam training task run at 1:30am, enter 30 in the minute column, and 1 in the hour column. The command to run the non-spam training task is:
fetchmail -v -u spam+mydomain.com -a -n –folder nonspam 127.0.0.1 -m ’sa-learn –ham –siteconfigpath /usr/share/spamassassin’
This command is saying “Fetch the email from the “nonspam†folder of the specified email account, and pass it to the ’sa-learn’ command, and learn that this email is “ham†(non-spam)
-
The next entry will tell SpamAssassin to train for spam. Enter a later time in the the minute and hour field, such as 35 for minute, and 1 for hour, to have the command run at 1:35am. The command to train for spam is:
fetchmail -v -u spam+mydomain.com -a -n 127.0.0.1 -m ’sa-learn –spam –siteconfigpath /usr/share/spamassassin’
As no folder is specified, the spam emails will be assumed to be in the inbox of the spam account
The cron screen should look something like the following:
Provide Training Emails
As noted before, SpamAssassin requires 200 spam and non-spam emails before it will start using the Bayesian filters in emails.
To provide the emails, you must first access the spam email account in your preferred email client. Follow the instructions for your email client to access an IMAP email account, using the account name and password you selected above. When you have created the account and accessed it, you should see the Inbox and nonspam folders. Copy 200 spam emails into the inbox, and 200 non-spam emails into the nonspam folder. The next time the cron jobs are scheduled to execute, SpamAssassin will create a database from those emails that it will use to determin whether any future emails are spam.
Ongoing Management
Any emails that are identified as spam will be be marked as such with ***SPAM*** in the subject line. You can use a rule or filter in your email client to automatically deal with these emails, perhaps by putting them into a folder the contents of which are automatically deleted after a defined time period.
SpamAssassin gets better with the more data it has to work with, and so from time to time you will get false positives, where an email is classed as spam that should not have been. If this happens, copy the email to the nonspam folder of the spam account and SpamAssassin will update its “nonspam†database. Likewise, some spam will not be identified, and these should be copied to the inbox of the spam account.
Troubleshooting
Most issues that may be encountered with using this method to train SpamAssassin can be diagnosed by reviewing the email received from the cron job. They are usually related to the fetchmail process executed through cron:
-
The SpamAssassin global configuration file is not at /usr/share/spamassassin. Check with your hosting provider as to its location
-
Fetchmail is not installed. Ask your hosting provider if it can be installed.
-
Emails do not have a Bayesian spam score. This is usually because SpamAssassin has not been provided with enough emails to build a profile of what is and isn’t spam. Check that the inbox and nonspam folders are being emptied each day, and that you have provided 200 samples of each type of email
This article also appears in our knowledgebase, along with many other cPanel tips
