XF 1.5 Bypass E-mail Defer Queue

[nrep]

We send out a newsletter using the XF bulk mail system, however since upgrading the XF 1.5 it has become painfully slow. From some helpful advice on the forums, I found out this was because XF 1.5 handles emails by adding them to a defer queue.

Is there a way to bypass this? On XF 1.4 we can send 5k e-mails in less than a minute. On XF 1.5 it takes closer to an hour.

 

[Floyd R Turbo]

But something has to be communicating back to XF to tell it to slow down, right?

There has to be a reason why XF would slow down the number of emails it tries to send in a batch - there has to be a trigger mechanism for this type of action, doesn't there?

 

[Chris D]

I think you're analysing it wrong. XF will send as many emails from the queue as it can in 8 seconds. There's no batch size, it will attempt to send as much from the queue as possible until the queue is empty or until 8 seconds has passed.

If one email is taking ~8 seconds to send (which is out of XF's control because we just pass the email off to the configured mail transport) then it will only send one at a time.

 

[Floyd R Turbo]

Thanks - so then my question becomes, what on the server side of things typically controls how many emails can be sent per unit of time?

 

[Chris D]

Well, nothing, typically. If there's some sort of throttling it would have to be happening upstream in the mail server itself. With it being a server issue, it's really impossible for us to be specific.

 

[Sim]

Thanks - so then my question becomes, what on the server side of things typically controls how many emails can be sent per unit of time?

Which ESP are you using? Are you sending via SMTP?

By way of comparison - my weekly email goes out via the XF mail queue, but I have a custom addon which sends via HTTP API on SparkPost and sends between 3K and 4K emails per hour according to the SparkPost stats.

I haven't benchmarked it using SMTP though.



I should also note that I'm not using the built-in "deferred" message processing - I disabled that and instead run a unix cron job to trigger deferred processing once per minute. Not sure if that would have an impact on sending throughput.

 

[Floyd R Turbo]

Which ESP are you using?

I'm using the default with the -f parameter

 

[snoopy5]

Hi,

Xenforo clean install version 1.5.17. I tried to add in my config.php file:

$config['rebuildMaxExecution'] - default: 8

I wanted to test this code to reduce the seconds to 1 or 2 so that less emails are sent out per reload when doing the mass mailing in ACP/users/email users.

But then I only get a blank screen, if I put that code into library/config.php. Even with 8 seconds.

Does this code is still valid in XF1.5.17?

 

[Paul B]

Yes, it's still valid.

8 seconds is the default so no need to add the code.
The correct entry however would be:

$config['rebuildMaxExecution'] = 8;

 

[snoopy5]

Yes, it's still valid.

8 seconds is the default so no need to add the code.
The correct entry however would be:

$config['rebuildMaxExecution'] = 8;

Would it be a bad idea if I would add this code:

$config['rebuildMaxExecution'] = 1;

each time into the config.php file, if I want to send en email to all users, so that it does not send more than 1000 emails per 10 minutes?

 

[Paul B]

That likely won't work for various reasons.

You will need a more definitive solution if your host is imposing a limit such as that.

 

[snoopy5]

easier said then done...

I do not want to start a payed service like Mandrill, Sendi...., Mailchimp etc. Once you start it and your userlist is growing, you end up with very high bills monthly.

There are reason why people use shared hosting. Price is the main reason. So why saving on hosting and then paying large bills for newsletters?

This is a huge market Xenforo is totally ignoring with the current way how massemailing users is offered within Xenforo