eZ Community » Forums » Developer » Reduce memory usage on users import...
expandshrink

Reduce memory usage on users import (120k +)

Reduce memory usage on users import (120k +)

Tuesday 01 March 2011 9:56:43 am - 8 replies

Hi,

I've made a script, used as a cronjob in ssh.
This script open a txt file, and for each line, import data as new user in eZPublish.

All is ok, but the eZOperationHandler::execute content publish memory usage increase at each iteration.

I make an eZContentObject::clearCache(), and the memory usage is steady, but the memory peak increase ( I associate it to the content publish action).

Is there any solution to reduce actions on publishing ?
I've already added the [SearchSettings]DelayedIndexing=enabled in site.ini.append of my siteaccess.

Many thanks.

Modified on Tuesday 01 March 2011 9:59:07 am by Remi B

Wednesday 02 March 2011 12:02:23 am

I believe you should use:

$object->resetDataMap();
eZContentObject::clearCache($object->attribute('id'));

after import of every user. Maybe also using some unset() for used variables.

If you want some more details, I had a question about this here:

http://share.ez.no/forums/develop...amap-and-ezcontentobject-clearcache/

Modified on Wednesday 02 March 2011 12:05:08 am by Mavko Žmak - Žmale

Wednesday 02 March 2011 12:06:02 am

Also, for faster import, you could disable some module operations, look at this discussion for more info:

Wednesday 02 March 2011 9:30:17 am

Thanks for your second post with the disableModuleOperation, the memory peak does not increase at each iteration.

I was already do the reset stuff, memory usage is about 30Mo between each publication.
But the publication take 500Mo happy.gif Emoticon

Btw, i've just relaunched my import with your trick, i'll post here to debrief about the memory usage.

Wednesday 02 March 2011 11:56:26 am

From your case I can conclude that executing module operations operations upon publishing increases the memory usage.

That would suggest that some of the module operation has a memory leakage...

Anyone from the eZ crew wishes to comment on this one?

Modified on Wednesday 02 March 2011 11:56:46 am by Mavko Žmak - Žmale

Wednesday 02 March 2011 1:59:40 pm

ATM, the script memory usage peak is 547 Mo
Since this morning, 2980 objects have been imported

I've got 31375 objects in the destination node.

Script is still runing...

Wednesday 02 March 2011 2:20:05 pm

Hi Remi B,

I have used those lines inside an import loop, with significant results :

eZContentObject::clearCache();
unset( $GLOBALS['eZContentObjectContentObjectCache'] );
unset( $GLOBALS['eZContentObjectDataMapCache'] );
unset( $GLOBALS['eZContentObjectVersionCache'] );
unset( $GLOBALS['eZContentClassAttributeCache'] );
//unset( $GLOBALS["eZUserObject_".$node->ContentObjectID] );

Give it a try !

Jey

Wednesday 02 March 2011 3:09:37 pm

Hi Jey,

Thanks for your post, but, as i said, I already make this stuff happy.gif Emoticon

The memory peak coming during content publish action.

Wednesday 02 March 2011 3:39:53 pm

One thing I can say for sure is that using the Asynchronous Publishing that comes with eZ Publish 4.5 will significantly reduce the memory usage that comes from the publishing process (make it close from 0), and will fasten the process a LOT.

expandshrink

You must be logged in to post messages in this topic!

36 542 Users on board!

Forums menu

Proudly Developed with from