eZ Community » Forums » General » Template-block cache - overall control
expandshrink

Template-block cache - overall control

Template-block cache - overall control

Wednesday 04 February 2015 3:41:02 pm - 10 replies

Hi there, Here is the thing: in less than a month the site has generated more than 20 GB of template-block cache files (Ending up in a server crash for having satured the available disk space). Either than running a cronjob for purging the cache every while, is there a setting to set the max no. Of cache file or something to remove the old cache files to prevent anything like that from happening again? Thanks blunk.gif Emoticon

Modified on Tuesday 10 February 2015 10:18:00 pm by Lo' F.

Wednesday 04 February 2015 11:33:39 pm

Hello Lo' F,

I would strongly consider creating a crontab entry to run the eZ Publish cache clearing script at a regular interval (as needed) to avoid using all the available disk (ie: avoid the 20GB cache problem).

php -d memory_limit=-1 ./bin/php/ezcache.php --clear-all --purge;

Here is a ruff crontab example doing the above. The following example clears all caches in the morning at 4:20am on the 14th and 28th of every month.

# This must be set to the directory where eZ Publish is installed.
EZPUBLISHROOT=/mnt/abc/sites/ezecosystem/ezpublish_legacy
 
# Location of the PHP Command Line Interface binary.
PHP=/usr/bin/php
 
# Instruct cron to run the "clear all cache" command
# in the morning at 4:20am on the 14th and 28th of every month
20 4 14,28 * * cd $EZPUBLISHROOT && $PHP -d memory_limit=-1 ./bin/php/ezcache.php --clear-all --purge 2>&1

Naturally you can change the above example to clear the caches more frequently as needed to ensure you do not run out of disk space.

Alternatively you can manually clear all caches doing something like this incomplete example:

rm -vrf var/cache/* var/ezdemo_site/cache/* var/ezwebin_site/cache/* var/yoursiteacess_var_dir_name/cache/*;

Here is just one example describing the cronjob format: http://www.adminschoice.com/crontab-quick-reference

From: https://www.google.com/#q=crontab format

I hope this helps!

Cheers,
Heath

Modified on Wednesday 04 February 2015 11:35:11 pm by // Heath

Thursday 05 February 2015 2:17:33 pm

It may also be helpful to check the cache-block keys to see if you can reduce the number of blocks.

http://share.ez.no/learn/ez-publish/ez-publish-cache-in-details-template-block-cache/%28page%29/3

Saturday 07 February 2015 10:03:09 pm

Thank u, guys!

Set the crontab in the ezpublish.cron and installed it to run through the command

crontab /var/www/vhosts/domain.it/httpdocs/ezpublish_legacy/ezpublish.cron

One question, if you allow me happy.gif Emoticon is there any function to allow me to regenerate the cache of a particular page or cache-block right after the cache has been cleared? I am asking this because the cache-block, responsible for growing so high, it takes a while to be displayed once the cache is clean, because of its massive fetch of objects, so that would be of use to save some time... you know what I mean... blunk.gif Emoticon

And, by the way, that also helped, Betsy, since I just realized the expiration of that cache-block had no control (and was left to the default two hours), well not anymore blunk.gif Emoticon

{cache-block keys=array($view_parameters, $node.url_alias) ignore_content_expiry  expiry=604800}

Sometime I should be more focused when I read the documentation...by myself ;P

But now, I was just wondering one thing. I made a function of a class that reads from an xml and use it to update the objects (through eZContentFunctions::updateAndPublishObject($object, $params)blunk.gif Emoticon. If I use

subtree_expiry='parent-node/'

instead of the other parameters, when the objects are updated the cache should expire, right? There I would like to call a function, if any available, to recreate that specific template-block in background. Can it be possible? 

So far, thanks to you, a big achievement, anyway!

Modified on Saturday 07 February 2015 10:07:14 pm by Lo' F.

Sunday 08 February 2015 6:49:31 am

Hello Lo' F,

To my knowledge there is no 'hook' which allows for you to regenerate all caches after clearing them. This area of eZ Publish is a bit of a black box to most people.

You can however re-compile your template cache using the './bin/php/eztc.php --help' php cli script. You should do this within the same crontab entry as your clear all caches command just after you clear all caches. Here is some older documentation on this command.

https://web.archive.org/web/20100921021524/http://ez.no/ezpublish/documentation/development/scripting/supplied_scripts/template_compiler

Side note, this article on the topic of caches is an informing read, http://share.ez.no/layout/set/print/learn/ez-publish/ez-publish-cache-in-details-template-compilation-and-ini-cache

I also wanted to mention (though this may not be of interest to you, not certain). You can combine the use of keys and subtree_expiry parameters within a cache-block usage. We did this for a cache block in ezecosystem recently, https://github.com/brookinsconsul...n/eze/templates/parts/sources.tpl#L1

Here is a link to the cache-block documentation which is very informative, https://doc.ez.no/eZ-Publish/Technical-manual/4.x/Reference/Template-functions/Miscellaneous/cache-block/(language)/eng-GB

EDIT: You -could- use httrack to create / prime all caches (including content, ini, etc) by spidering the site in question. I've done this in the past for site's that really needed full caches for optimum performance in production.

EDIT2: I've also done some work in the past on an admin module view to help cache all urls from within eZ Publish but it's honestly been a long time since I've last used it so there may be bugs, if you like it but find problems let me know and I will do my very best to help. https://github.com/brookinsconsulting/bcupdatecache 

Please let us know how this issue evolves and what solutions you end up finding more helpful happy.gif Emoticon

I hope this helps!

Cheers,
Heath

Modified on Sunday 08 February 2015 6:54:27 am by // Heath

Tuesday 10 February 2015 4:54:56 pm

Diving into this cache things, taking one thing at a time:

Let's start with the setting part. I added a viewcache.ini.append.php ini inside eng siteaccess with the following setting

[ViewCacheSettings]
SmartCacheClear=enabled
 
[project]
DependentClassIdentifier[]
DependentClassIdentifier[]=page
ClearCacheMethod[]
ClearCacheMethod[]=object
ClearCacheMethod[]=parent
MaxParents=2

in order to empty the cache of an object of the class project and its 2 nodes above parents, when it's modified/published, in a content tree like this:

Projects (page class)

|__Projects DB (folder class)

|_____single project (project class)

which seems to work just fine. But I also need to regenerate the cache of that Projects page in background before it's first accessed by a user (to save him from the first loading time).

And, "ok" I thought "not a big deal" if I use the PreViewCache

[ContentSettings]
...
PreViewCache=enabled
PreCacheSiteaccessArray[]
PreCacheSiteaccessArray[]=eng

as I was hoping that on object creation/modification, it would have generated the cache of the object and its parent nodes. But it's not happening.

In other words it's the "grand-parent" node cache I need to regenarate.

Is this possible?

Modified on Tuesday 10 February 2015 4:55:34 pm by Lo' F.

Friday 13 February 2015 2:29:18 pm

Hello Lo' F,

Just remember that PreViewCache is not the same as all caches blunk.gif Emoticon

What you want is not supported internally by default in eZ Publish.

Question: Why did you not respond to any of the suggestions made in my last post?

EDIT: I've actually done what your talking about (generating a cached copy of modified content tree hierarchies) using the static cache feature + customized view cache rules on eZecosystem.org but I don't think you want to use static cache just to gain this end result since it's a side affect feature (static cache is generated via rendering the whole url alias much like my httrack suggestion above) not an actual solution to the real issue your trying to solve (again which is not supported by default).

I hope this helps!

Cheers,
Heath

Modified on Friday 13 February 2015 2:29:56 pm by // Heath

Friday 13 February 2015 11:11:41 pm

Hi Heath! Sorry I am a little late but it was worth to go through all your suggested links to get to be more acknowledged cache wise. Just a little overwhelmed these days...
Well, I think I've come to an end, after all, and all it's handled by cron

EZPUBLISHROOT=/var/www/vhosts/domain.it/httpdocs/ezpublish_legacy
 
PHP=/usr/bin/php
 
# Instruct cron to run the "clear all cache" command
# at 2:30am every morning
30 2 * * * cd $EZPUBLISHROOT && $PHP bin/php/ezcache.php --clear-all --purge 2>&1
 
...
 
# Instruct cron to re-generate the cache
# at 3:30am every morning
#MAILTO=info@xxxxxxx.com
30 3 * * * cd $EZPUBLISHROOT && $PHP bin/php/eztc.php -s eng 2>&1

This way I regenerate the whole siteaccess cache. I've tried to force the regeneration of just a specific template, the one I actually needed

 cd $EZPUBLISHROOT && $PHP bin/php/eztc.php -s eng --force  extension/extname/design/designname/override/templates/full/page.tpl

but, even though the output was "Compiled template file: extension/extname/design/designname/override/templates/full/page.tpl", the page takes a while to load. So I cut the story short and finished it off this other way.

Some may wonder, "every day, clearing cache's too much", but one of the set crontabs has to be launched daily 'cause it has to update objects from an external source and so the clear/recompile cache command completes the whole circle.

I cannot thank u enough for all the precious tips I am gathering here blunk.gif Emoticon

p.s. and yeah, previewcache setting not so handy here.

Saturday 14 February 2015 1:11:38 am

Hello Lo F,

Thanks for the update. Again I'm very happy to help you as much as possible happy.gif Emoticon We're the eZ Community ... it's what we do happy.gif Emoticon

I don't have any problem with frequent cache clearing, it's often vital to an integration, aka, it's normal; The trick is all in what caches you clear, why, how and when. The real devil is always in the (integration) details.

But, it seems your using cron to clear all cache (again that's fine) but .. in your content update from an external source code you should be using eZ Publish to clear content object cache instead dynamically (during the update from an external source).

This is supported and I use this so frequently it's an every day thing blunk.gif Emoticon

Remember for the disk cache over full of 20gb+ you can of course use the crontab solution we shared above but ..

Remember there is a difference between template cache (does not need to be cleared as often as this changes much less frequently) and content cache (this often changes depending on your eZ Publish API usage). I think if you review your cache disk size problem in greater detail you would find that template cache size is not the problem rather content cache (among other kinds of cache) could be ...

With specific regard to your updating of content objects from external source. You don't need (per say) to clear all caches via cron every day as that is wasteful and most likely highly unnecessary. I still recommend trying my original schedule of all cache clearing to avoid the disk space problem as that should be enough to prevent it from overloading your limited disk space. 

When your updating content object content from an external source you should use this eZ PHP API method to clear your content object cache. Here is an example:

eZContentCacheManager::clearContentCacheIfNeeded( $objectID );

Here is a real world example of it being used in our own content import handlers for remote content, https://github.com/brookinsconsulting/ezecosystem/blob/master/ezpublish_legacy/extension/ezecosystem/classes/sqliimporthandlers/sqligithubatomimporthandler.php#L154

Alternately you may find this helpful as well (tho ezecosystem does not use/require this):

<span>eZContentCacheManager::clearObjectViewCacheIfNeeded( $objectID );</span>

Please let us know your thoughts!

PS. You have my imagination burning, what do you mean by "So I cut the story short and finished it off this other way." How did you really solve the cache regeneration problem? If you found a solution we would love it if you would share happy.gif Emoticon

I hope this helps!

Cheers,
Heath

Saturday 14 February 2015 6:48:31 pm

Quote from // Heath :

what do you mean by "So I cut the story short and finished it off this other way." How did you really solve the cache regeneration problem?

..nothing more than what said above: the first cron to purge

 bin/php/ezcache.php --clear-all --purge 2>&1

and the last to regenarate the whole cache (unable to make the one to force just one .tpl work..)

 bin/php/eztc.php -s eng 2>&1

but again that was "reaaallly" meant to cut the story short and finish it off ;P

The source of the pain is/was this page which template cache-block contains a huge amount of data fetched from content objects updated on a daily basis on a massive way. By passing the parameter subtree_expiry='parent-node/' to its cache-block, the physical template-block directory grows on size very quick (again in two weeks time reached the size of 8GB - that's the urge of emptying the cache more often). On the other side if the cache is not generated, the page takes a while to load when accessed for the first time. 

So by doing this crontabs circle (clear-cache/update-objects/generate-cache) seems to "solve" the thing.

In order to launch the update scripts I am just calling the page that does the job

 /usr/bin/GET -s http://www.xxx.it/run.php

...

About the cache method I added it, following the update method, as a new ingredient to the recipe. Again good to know it exists!

... 
$object = eZContentObject::fetch($objectid);
        $attrs = $object_attrs;
        $params = array(
            'remote_id'  => md5(microtime(true)),
            'language'   => 'eng-EN',
            'attributes' => $attrs
        );
        eZContentFunctions::updateAndPublishObject($object, $params);
        eZContentCacheManager::clearContentCacheIfNeeded($objectid);
...

P.S. I know u where expecting some better solution with me saying "this way" like something secret.. but indeed it was just all about the above speech :P Sorry for the disappointment :/

Modified on Monday 16 February 2015 12:48:16 am by Lo' F.

Sunday 15 February 2015 5:14:53 am

Hello Lo F,

Thank you for your clarification!

Also thank you for marking this thread as solved!

Cheers,
Heath 

expandshrink

You must be logged in to post messages in this topic!

36 542 Users on board!

Forums menu

Proudly Developed with from