Not a bug Potentially poor performance with lots of [GALLERY] BB code embeds

[420]

If you want to do this for a test period only for an hour or two, that's a different story altogether and not a problem at all.

The only thing we unfortunately are unable to agree to, is keeping them like that. Your brand new, modern technology surely has at minimum, the capabilities of that prehistoric Photopost written in alien code.

The only thing I am concerned with is whether or not this will take yet another self-made script to perform the change, taking 7 days to complete or if it's a simple setting to switch, while you trouble shoot.

Please let me know how I can help or direct Jake to work his magic from his end.

Thank you.

 

[Chris D]

I am thinking of more permanent solutions for you, as it happens, but quick question.

The thread you sent me the website test results for by PM (page 124). What do you think to the loading speed of that vs page 1, for example?

In Photopost, did you have something which automatically reduced the image size down to a width of 1024? Most older gallery images seem to be constrained to that width, but most new gallery images seem to be huge - often over 4-5 times the size. Is that intentional?

 

[420]

The thread you sent me the website test results for by PM (page 124). What do you think to the loading speed of that vs page 1, for example?

Are you referring to the tests or the actual pages? I did not see a page linked to, just the test.

In Photopost, did you have something which automatically reduced the image size down to a width of 1024? Most older gallery images seem to be constrained to that width, but most new gallery images seem to be huge - often over 4-5 times the size. Is that intentional?

Photos in the gallery remained at 1280x1024 I believe, but were resized to fit the width of each post they were embedded in. So I would imagine they were resized to 1024x768 for display purposes. Let me know if I can offer any more info. Thank you.

 

[Chris D]

Are you referring to the tests or the actual pages? I did not see a page linked to, just the test.

One of the tests was of page 124 of a specific thread. In that same thread, earlier pages (pre XF) load fine which sort of goes some way to confirming my thoughts.

But if you check that first page, it seems as though the actual image sizes posted in there are around 1024 wide.

XF has a feature to automatically resize images, but that doesn’t appear to be enabled as most of the new images I’ve seen in the gallery are fairly huge.

 

[420]

One of the tests was of page 124 of a specific thread. In that same thread, earlier pages (pre XF) load fine which sort of goes some way to confirming my thoughts.

But if you check that first page, it seems as though the actual image sizes posted in there are around 1024 wide.

The only connection to that I can see would be the fact that we upped the amount of photos to 50 per page, whereas before it was 40. I am having trouble finding that setting now. Edit: found it and changed it back to 40.

Where are you seeing the link to the journal with page 124, I could not find a link, only to her profile page.

They were not tested at different times to rule out the fact that they were just testing during times when the issue was occurring or not occurring. True test would be to test those pages at several different times of the day, spread apart. Anyhow, as we found the issue is causing the entire site to slow to a crawl when it happens, which supports other theories. From what I can tell thus far, it appears the gallery is causing the problem, locking up the entire site. Once we find the culprit, everything should resolve itself.

XF has a feature to automatically resize images, but that doesn’t appear to be enabled as most of the new images I’ve seen in the gallery are fairly huge.

Can you please be more specific, how/what exactly does it resize and where is this setting?

We need to be certain we don't resize all images below 1024x768 or even 1280x1024, unable to revert back once done. These gallery photos are not only our bread and butter, but essential to teaching our community with class, therefore the quality they preserve are priceless to us and must be retained at all costs.

Thanks Chris.

 

[420]

One of the tests was of page 124 of a specific thread. In that same thread, earlier pages (pre XF) load fine which sort of goes some way to confirming my thoughts.

But if you check that first page, it seems as though the actual image sizes posted in there are around 1024 wide.

XF has a feature to automatically resize images, but that doesn’t appear to be enabled as most of the new images I’ve seen in the gallery are fairly huge.

Found the page url finally at top of the test page, now I am on the same page, literally.

The page loads slower, because there are a lot more photos on it, but also because the images are huge, like you say. This is obviously an issue, because the page sizes are much bigger, which explains slow page loads for photo pages and people using more bandwidth. Where are the settings to change the sizes once uploaded?

We allow up to 20mb to upload, does that mean 20mb photos are being displayed as well on pages, once you post the image url into your browser?

 

[Chris D]

The page loads slower, because there are a lot more photos on it, but also because the images are huge, like you say. This is obviously an issue, because the page sizes are much bigger, which explains slow page loads for photo pages and people using more bandwidth. Where are the settings to change the sizes once uploaded?

There is another aspect too which is the potential overhead of XenForo fetching the images in the first place. That's one fairly major difference between PhotoPost and XF - in PhotoPost the URL to the images in posts just point directly at the image file on the file system. In XF, the URLs are actually fetched by XF and this enables us to do things like permission checks and that kind of thing, though loading the XF framework for each image and then checking permissions etc. when there's so many images on the screen is quite heavy so it is possible that is also having an impact.

I've created a new version of XFMG for you to try out a different approach. It will not change the way anything looks on your forum, but it is a much performant approach at fetching the images from the server. I will send that to you in a moment with instructions.

The settings for the maximum width / height of an uploaded image are actually in user group permissions. So theoretically you can have some users who can upload larger images than others. But for simplicity for now, I imagine you'd just want to apply this globally to all users.

You can do that under Admin CP > Groups & permissions > User group permissions > Registered



Find "Maximum image width" and "Maximum image height" (near the bottom of the page) and set those to a reasonable value (1024, 1280 whatever you prefer) and then click save.

This won't affect images that have already been uploaded, though we can address that at some point.

Primarily, I'm keen for you to try the new version, first, and let me know if there's a significant improvement or not. I'll send that to you in a second.

 

[420]

Hey Chris,

I changed the max width to 1280 on both, that was a good call.

We are also upgrading to PHP 7 in the morning, which should help as well.

I'd like to ask some questions first about this new gallery, before jumping in.

The day has gotten the best of me though and I need to recharge, so I'll resume this in the morning.

Thank you very much for your support, we are truly grateful.

 

[Chris D]

I changed the max width to 1280 on both, that was a good call.

If you look at the newest images being uploaded, they're still exceeding those dimensions so I'm not certain the change has been applied.

You might need to use Analyse permissions (and the Global permissions tab on that page) to figure out where it's being overridden.

 

[420]

Thanks Chris, looks like that worked finally, with your help

Our systems admin just upgraded to PHP 7, which cut the server load in half as well.

How do we get the photos uploaded since the migration a few weeks ago, to resize to the 1280 max height/width?

 

[Chris D]

Thanks Chris, looks like that worked finally, with your help

Our systems admin just upgraded to PHP 7, which cut the server load in half as well.

Yeah I've been nosing around and it looks like things are a lot better - even on the pages which have huge images. It may mean, in fact, we don't need to make any changes.

How do we get the photos uploaded since the migration a few weeks ago, to resize to the 1280 max height/width?

There's nothing built in to do that. If it's a significant amount of images and therefore worth doing then a script would need to be written to identify the larger images and resize them accordingly.

 

[420]

Yeah I've been nosing around and it looks like things are a lot better - even on the pages which have huge images. It may mean, in fact, we don't need to make any changes.

That would be epic, as I'd rather stay true to the standard structure of the software, than to start experimenting with our enormous database that could get effected in some way, during these experiments. We'd much rather you guys test things on your own stuff, instead of ours.

There's nothing built in to do that. If it's a significant amount of images and therefore worth doing then a script would need to be written to identify the larger images and resize them accordingly.

Ok, I'll get with Jake to see if he can handle this. Hopefully he's reading this already.

Thanks guys, we are making progress!

 

[Chris D]

That would be epic, as I'd rather stay true to the standard structure of the software, than to start experimenting with our enormous database that could get effected in some way, during these experiments. We'd much rather you guys test things on your own stuff, instead of ours.

The changes I sent to you yesterday were tested, obviously. We wouldn't go releasing code to customers without testing it first. Nor would the changes have made any changes to your database or any significant structure changes. It was merely a different way of fetching the images from the file system.

Thanks guys, we are making progress!

 

[420]

The changes I sent to you yesterday were tested, obviously. We wouldn't go releasing code to customers without testing it first. Nor would the changes have made any changes to your database or any significant structure changes. It was merely a different way of fetching the images from the file system.

I figured as much, but needed to go through a list of questions first to be sure, before moving forward. Hopefully, we won't even need to do that now, let's see what happens over the next day or so. Thanks again

 

[Jake B.]

@Chris D I've taken a brief look but can't really track down what method is being used to resize the images, do you recall offhand where this is being done so I can run it specifically for these images?

 

[Chris D]

It happens in the Upload object, usually, based on the attachment constraints.

You'd probably just have to write something custom, loop through the attachment data records, resize them if they exceed the currently defined max dimensions and, most importantly, ensure the file_size field is updated as well. I don't think it will be necessary to modify the file_hash. If you did that, you'd need to update the attachment file hashes too (for the default XF attachment thumbs).

 

[Jake B.]

I do recall running into an issue with our image optimizer add-on where I had forgotten to update the file hash and chrome blocked it because the hash was in one of the response headers, or maybe that was file size will have a look into it, thanks Chris!

 

[Chris D]

It would be file size. We send that out in the Content-Length header. Chrome throws a fit if the size is different to what's actually being loaded.

 

[Jake B.]

ah yep, that was it then