Setup Bounce Handling in phpList

In this article, I am going to list down the steps in which you can enable bounce processing in phpList version 3.X.X

What is phpList?

phpList is Open Source software for sending email newsletters, marketing campaigns, announcements and send product related transaction mails as well as track all the type of mails written above.

It comes in two versions:
1. Open-Source version: Checkout here.
2. Paid version: Checkout here.
It is quite similar to how WordPress works in pricing, allowing you to host it yourself or host it on their website. I would continue about how to process bounces on the open source one.

Bounce Process

What is a Bounce?

A bounce is an email message that is returned because it could not be delivered for a number of reasons: The mailbox of the recipient might be full, the email address might no longer be valid or temporarily disabled, the email address might be misspelled, etc. A bounce is not the same as an out-of-office reply. For more info, see this Wikipedia article on bounces.

Why to process it?

The reason why you want to process bounced emails is that if you continually send emails to invalid email addresses you can run the risk of having your server black listed and not being able to send any more emails from there. Processing your bounced emails will keep a record of what emails have bounced so that you do not continue to send to these addresses.
For more info, see this Click here.

Setup Bounce Processing phpList..

Prerequisites:
1. Message envelope :
The email id where all the bounces are returned. Make sure the mail-id is unique as it would serve the purpose of being only a read only mail from where phpList would read and process the bounces and delete the mails processed mails as well.

2. php-imap
phpList requires php-imap extension to be installed to continue processing of bounce mails. This extension is different from the IMAP protocol used in mails. To install it:

• If using the old php5 repository:

sudo apt-get install php5-imap

• If using the new one:

sudo apt-get install phpX.X-imap //where X.X is the php version you are using

Steps:
1. Navigate to the following folder within the phpList directory:

Directory Path: PATH-TO-PHPLIST/public_html/lists/config

2. Set the value of $message_envelope value

$message_envelope = "bounces@yourcustomdomain.extension";

3. Set the protocol to use when handling bounces

• mbox (Wikipedia)
• pop (Wikipedia)

$bounce_protocol = "pop"; // "mbox"

4. If protocol is “mbox”
You need to specify a file in the local machine (where phpList is hosted). The file should be in mbox format

$bounce_mailbox = "/PATH/TO/LOCAL/MBOX/FORMAT/FILE";

5. If the protocol is “pop”
The following are required:

• The mail-box host:
  The host domain where your mail-box can be found/accessed

$bounce_mailbox_host = "your.mail-box.host";

• The mail-box user:
  The username to access your mail-box (In most cases it is the mail id itself).

$bounce_mailbox_user = "your@mailbox.user";

• The mail-box password:
  The password used to access the mail:

$bounce_mailbox_password = "mailbox-password";

• The mail-box port:
  You can set the mail-box port to 110 or if you want a more secure connection, then set it to 995 as follows:

$bounce_mailbox_port = “995/pop3/ssl”;
// or
$bounce_mailbox_port = “110/pop3”;
// or
$bounce_mailbox_port = “995/pop3/ssl/novalidate-cert”;
//Use the configuration that best fits the working model

To setup pop3 with different mail clients:

• Yahoo Mail : Click here
• Gmail : Click here

5. Set the bounce threshold limit:
This limit specifies the number of times that an mail has to bounce of a email id before the email id is blacklisted.

$bounce_unsubscribe_threshold = 3; //Keep the value greater than 0

6. Setup the mail-box purge options
phpList gives two options to purge emails from your mail-box.

• Purge the processed mails
  Set the value to 1, if you want phpList to delete the messages(from you mail-box) that have been processed.

$bounce_mailbox_purge = 1; // or 0

If set to 0, then it would not delete those mails after bounce processing, which would cause the mail to be read every time when bounce emails are processed, causing an email to be wrongly blacklisted. It is recommended to set it to 0 only when testing the functionality.

• Purge the unprocessed mails
  Set the value to 1, if you want phpList to delete the messages(from you mail-box) that have not yet been processed.

$bounce_mailbox_purge_unprocessed = 0; // or 1

Testing:

1. After the above setup is complete, you can test whether the bounce functionality is working. To test the functionality add a bounce email-id to your subscriber list.
2. Start a campaign and send the test mail to the bounce email-id. Furthermore, schedule the campaign to be sent instantly to the bounce email-id.
3. After the campaign is sent, cross-check your mail-box to see if your have received the bounce mail. If yes, then everything is working fine.
4. Process the bounce manually, from the browser:

• In config.php, set the Manually Process Bounce as follows:

define(‘MANUALLY_PROCESS_BOUNCES’, 1);

• Open the browser, and navigate to your phpList hosted and then login.
• After login, Navigate to System > Manage bounces
• Click on the Process Bounces Options.
• The request would continue for some time. At the end, a textbox would be displayed with stats about the processed mails in the mail-box, indicating that bounce processing is working properly.

5. Alternatively, to run it from command-line

• In config.php, set the Manually Process Bounce as follows:

define(‘MANUALLY_PROCESS_BOUNCES’, 0);

• In the command line, paste the following: (The following command is for Ubuntu, it would vary slightly for other systems)

/usr/bin/php PATH-TO-PHPLIST/public_html/lists/admin/index.php -pprocessbounces -c PATH-TO-PHPLIST/public_html/lists/config/config.php

• (Warning) When the processing starts, DO NOT STOP THE PROCESS IN BETWEEN. If stopped, then what to do (Look at the “Bugs and Problems faced during Bounce Processing” section below.).
• At the end, it would display the stats of bounce processing.

Automate Bounce Processing:

To automate bounce processing, you can setup bounce processing as a cron job that would execute at regular intervals as follows:

1. Create a Logs folder inside your phpList folder and name it “log”
2. Inside the “log”, create a file named bounce.log.
3. Open the crontab:

sudo crontab -e

4. Inside crontab type the following command:

0 3 * * * USER=listprocessor; export USER; /usr/bin/php PATH-TO-PHPLIST/public_html/lists/admin/index.php -pprocessbounces -c PATH-TO-PHPLIST/public_html/lists/config/config.php > PATH-TO-PHPLIST/log/bounce.log

The above command process the bounces once a day. For more information on how to schedule cron, Click here. (What Wikipedia says about Cron?)

5. Done, your cron has been scheduled. Bounce Processing would be regularly performed by the system.

Note: If you have access to CPanel, you can schedule your cron from there or alternatively use some other tool to schedule your cron. phplist.com also provides you an options to process your jobs on your hosted domain. For more info, Click here.

Generate Bounce Rules
One bounce processing starts, generate the bounce rules

• Navigate to System > Manage bounces
• Click on generate Bounce Rules.
• A bounce is matched on a regular expression, which is set up in “bounce rules”. The bounce rule determines what needs to be done the bounce.

Note: For more info on Bounce Rules, Click Here

Bugs and Problems faced during Bounce Processing

When processing bounce, if any of the following occurs:

1. If manually using the browser, at the end if empty text-box is displayed
2. If using command line, nothing is displayed and scroll is at the bottom.

It indicates an error has occurred, follow the steps below to fix it:

1. Open you phpList database
2. Open the table `phplist_sendprocess`
3. Unset the alive flag, where page value is `processbounces`
4. Alternatively use the following MySQL query:

UPDATE `phplist_sendprocess` SET `alive` = 0 WHERE `phplist_sendprocess`.`page` = 'processbounces'

The reason this is done, is when phpList starts processing bounces, it sets a flag variable indicating that it has started processing bounces, in order to ensure that no other process if started at the same time also performs the same operation of processing bounces. The second one will simply bail-out. At the completion, it automatically unsets the flag, allowing another process to start.
When stopped in between, it fails to unset the flag, thereby going in a state of Deadlock, in which no process is executing and every new process spawned by phpList thinks that a process is already being performed.

Feedback and Comments are appreciated. Until we meet again.
Found this post useful? Kindly tap the ❤ button below! :)

Cheers,
Manish M. Demblani