This site has been archived. To learn more about our current products Ibexa Content, Ibexa Experience, Ibexa Commerce head over to the Ibexa Developer Portal

eZ Community » Forums » Extensions » eZ Find » Errors when reindexing site in eZ...
expandshrink

Errors when reindexing site in eZ Find LS

Errors when reindexing site in eZ Find LS

Monday 07 July 2014 4:34:24 pm - 13 replies

Hi

I just upgraded to 5.3 and so using eZ Find LS.

My local environment is a VM with same kernel, package versions as production.

Now, when I run the command:

 php extension/ezfind/bin/php/updatesearchindexsolr.php -s ezflow_site_admin

On my local installation everything is fine, but on staging and production environments, only some of the content is indexed and I see these errors in var/log/error.log:

After searching some old threads, I found a temporary fix is to hack updatesearchindexsolr.php so $useFork is set to false.

I don't know if this is a bug with ezfind or there is something in my server environment that needs changing.

Monday 18 August 2014 8:55:19 am

I had exactly the same issue on my production server.

To avoid hacking the file, you can just use the conc option. If set to 1, no fork will be used.

But the problem exists. I tried with conc=2 but i still had errors.

Monday 18 August 2014 3:55:05 pm

Hi Jérémy

Using conc=1 worked for me too.

Also, when I changed mysqli.reconnect = On in php.ini then indexing with forking works.

This is new when using ezfind LS 5.3. I don't think using concurrent should cause mysql connection to be lost?

Tuesday 19 August 2014 10:27:24 am

One mysql parameter which might be worth investigating is the maximum connection timeout - if a php cli script spends large amounts of time without sending commands to the db, the db might drop the connection.

Another thing to investigate is the usage of the ezdfs cluster mode: iirc when using forking, the main php process reopens a new db connection for each forked child. Maybe it does not do the same for clustere db connections. Are you btw using different db connections for cluster and content database?

Tuesday 19 August 2014 3:50:12 pm

Hi Gaetano

I'm using the same database for the cluster tables and content tables.

Changing mysql.connect_timeout from 60 to -1 in php.ini does not effect anything.

When I reindex with mysqli.reconnect = On there are up to 5 connections at one time.

Tuesday 19 August 2014 6:34:48 pm

The best-practice (or shall I say only official way) for ezdfs is to have 2 separate connections for content and cluster.

This is because there might be transactions mixing up if you share the connection.

But I wonders if side-effects might spill in the forking setup used by that script (we already had pains troubleshooting oracle connections from that script. forking and open file pointers are quite a pain)

If the db is the same, an easy way to make sure you have php using separate connections is to create 2 separate db user accounts, one for cluster and one for content.

Could you test using a separate-connections config, and report back?

Modified on Tuesday 19 August 2014 6:35:57 pm by Gaetano Giunta

Wednesday 20 August 2014 12:18:27 pm

Hi Gaetano

I created a new mysql user so have seperate user in site.ini.append and file.ini.append

So when I ran updatesearchindexsolr.php with mysql.recconect = Off, I see only 1 connection from mysqluser_content at the start of indexing. After this, all the rest are from mysqluser_content and errors reported as above.

With mysql.reconnect = On, I see connections from both users.

Thanks

Mark

Tuesday 09 September 2014 1:53:38 pm

Note that a new DB connection is always required. The reason is quite simple: a resource can't be used by multiple processes. Since a forked process will share all resources that existed at this moment, unless the connections are explicitly killed/recreated, it will fail with a mysql server has gone away error.

Friday 12 September 2014 11:07:03 am

@bertrand but I assume that the ezfind script does this automatically for both the main-db connection and the cluster-db connection, so that the developer does not need to do anything on his own, right?

Friday 12 September 2014 11:07:51 am

@mark q: did this problem only start when upgrading to eZ 5.3 ?

Friday 12 September 2014 11:58:08 am

Hi guys

Yes, this is only happens with 5.3.

Previously I was able to leave mysql.reconnect = Off without problems

Friday 12 September 2014 12:23:37 pm

I wonder if this can be related somehow to the fact that the new kernel now uses doctrine instead of ezcomponents to connect to the db...

Monday 15 September 2014 2:19:29 pm

Hi

You're using Symfony stack, right ? Then you should run the script using legacy wrapper :

php ezpublish/console ezpublish:legacy:script --siteaccess=ezflow_site_admin extension/ezfind/bin/php/updatesearchindexsolr.php

Furthermore, there is an error while updating ezsession table ??? You shouldn't have this while going through Symfony...

Monday 15 September 2014 3:06:40 pm

For what it's worth, we're also seeing this on 5.2 during a re-index.  Although in our situation, we use RabbitMQ to handle the indexing of the actual objects.  I.e. the indexation script adds all the objects to a RabbitMQ queue, and consumers on different servers do the actual heavy lifting.

From what I've been able to determine, the RabbitMQ consumer keeps a never-ending hold on the database connection.  So once it goes bad, it never tries to reconnect.  The only way to recover was to restart the consumer itself (which makes sense if mysqli.reconnect is Off).  This makes sense since these scripts never terminate (they need to always be listening for messsages from the controller).

I'm currently trying to determine if it's best to change this in php.ini, or leave it as part of the script itself.

expandshrink

You must be logged in to post messages in this topic!

36 542 Users on board!

Forums menu

Proudly Developed with from