eZ Community » Forums » General » Is EZ Publish righr for me?
expandshrink

Is EZ Publish righr for me?

Is EZ Publish righr for me?

Tuesday 10 April 2012 1:31:48 pm - 9 replies

Hi all,

Short background.

I have a project that needs to load listing data (Job, car, classifieds etc) into a database of some description and then have it search with Apache SOLR because their faceted support is excellent.

I am not a coder but have built hundreds of sites using Joomla and Wordpress.

I have SOLR installed on the server.

I came across EZ P CE and thought I might test it out. Im hoping for some advice from the community.

I thought of using D7 but loading content into the site is proving to be impossible. Each new content field becomes its own table (2 actually) which means a CSV file containing 27 000 records with up to 50 fields each, takes over 5 hrs on a very powerful server. Clearly not acceptable for a site wanting to store 500 000 documents and possibly update/add 50 000 a day.

Joomla simply doesnt have the ability to do this either, you can use Solr to handle simple documents but not a listing type of document where you have specific information like price, type, model, colour etc etc.

 

So my question is can EZ P CE handle this?

I notice it has EZ find ready to go out of the box which is awesome.

Can you construct different content types (Eg a job listing will have Title, salary, description, contact info, industry etc etc)?

How hard is it to import large CSV files - eg 50000 row files each day.

 

I see no pricing information for the commercial editions, would love to have a round figure in my head if I need to go that route, so if anyone has a heads up on pricing I appreciate it. Coming from a Joomla background, I like community editions better.

 

Thanks for any advice.

Tuesday 10 April 2012 2:11:18 pm

I have made one script before, the only problem is that sometimes it requires some tweaks because of memory usage, so maybe you'll need to create a shell script to call this script several times specifying offset and limit, and of course you'll need to adapt this to your needs, check http://share.ez.no/blogs/thiago-campos-viana/tip-google-login-sample-code before, in my script it avoids duplicating content by checking the username, but you can adapt to check for another field, the content object created is a content user with some custom attributes. Your database will remain with the same amount of tables if you create a custom content class, I'm sorry I'm kind of busy, so I can't give any further advice,  maybe somebody can help you a bit more.

 

<?php 
 
class Cliente{ 
 
    public function __construct($data) { 
        $this->Codigo=$data[0]; 
        $this->Situacao=$data[1]; 
        $this->CodigoPai=$data[2]; 
        $this->RazaoSocialNF=$data[3]; 
        $this->RazaoSocialLD=$data[4]; 
        $this->NomeFantasia=$data[5]; 
        $this->Setor=$data[6]; 
        $this->CNPJ=$data[7]; 
        $this->InscricaoEstadual=$data[8]; 
        $this->KmNF=$data[9]; 
        $this->KmLD=$data[10]; 
        $this->EnderecoNF=$data[11]; 
        $this->BairroNF=$data[12]; 
        $this->CepNF=$data[13]; 
        $this->CaixaPostalNF=$data[14]; 
        $this->CidadeNF=$data[15]; 
        $this->UfNF=$data[16]; 
        $this->ResponsavelNF=$data[17]; 
        $this->SetorRespNF=$data[18]; 
        $this->FoneNF=$data[19]; 
        $this->FaxNF=$data[20]; 
        $this->EmailNF=$data[21]; 
        $this->EnderecoComercial=$data[22]; 
        $this->BairroComercial=$data[23]; 
        $this->CepComercial=$data[24]; 
        $this->CidadeComercial=$data[25]; 
        $this->UFComercial=$data[26]; 
        $this->ResponsavelComercial=$data[27]; 
        $this->SetorRespComercial=$data[28]; 
        $this->TelefoneComercial=$data[29]; 
        $this->FaxComercial=$data[30]; 
        $this->EmailComercial=$data[31]; 
        $this->EnderecoLaudo=$data[32]; 
        $this->BairroLaudo=$data[33]; 
        $this->CepLaudo=$data[34]; 
        $this->CidadeLaudo=$data[35]; 
        $this->UFLaudo=$data[36]; 
        $this->ResponsavelLaudo=$data[37]; 
        $this->SetorRespLaudo=$data[38]; 
        $this->FoneLaudo=$data[39]; 
        $this->FaxLaudo=$data[40]; 
        $this->EmailLaudo=$data[41];     
         
     
     
    } 
 
    public function getEmail(){ 
        if(strlen($this->EmailLaudo)>3) return $this->EmailLaudo; 
        else if(strlen($this->EmailNF)>3) return $this->EmailNF; 
          else if(strlen($this->EmailComercial)>3) return $this->EmailComercial; 
          else {  
            return null; 
        } 
 
    } 
 
} 
 
function genRandomString() { 
    $length = 10; 
    $characters = '0123456789abcdefghijklmnopqrstuvwxyz!@#$'; 
    $string = ''; 
 
    for ($p = 0; $p < $length; $p++) { 
        $string .= $characters[mt_rand(0, strlen($characters))]; 
    } 
 
    return $string; 
} 
 
function deleteAll(){ 
 
  $objects =& eZContentObjectTreeNode::subTreeByNodeID(  array( 
            'Limit' => 500 
            ), 
            3179 
        ); 
         
    foreach ( $objects  as $node ) 
    { 
     
     $object = $node->object(); 
     //echo 'dasdas '; 
     $object->purge();       
      
    } 
     
    exit; 
 
 
} 
 
set_time_limit(0); 
require 'autoload.php'; 
 
 
 
 
$cli = eZCLI::instance(); 
$script = eZScript::instance(array('description' => ( "eZPersistentObject tutorial.\n\n"), 
            'use-modules' => true, 
            'use-extensions' => true)); 
 
$script->startup(); 
$script->initialize(); 
 
$options = $script->getOptions( "[offset:][limit:][delete:]"); 
 
 
 
$user = eZUser::fetchByName('admin'); 
$userID = $user->attribute( 'contentobject_id' ); 
eZUser::setCurrentlyLoggedInUser( $user, $userID ); 
 
 
if($options['delete']==1) deleteAll(); 
 
 
$file = fopen("extension/csvimport/clientes.csv","r"); 
fgets($file); 
 
$offset=$options['offset']; 
$limit=$options['limit']; 
$current=-1; 
while(! feof($file)) 
{ 
     
  $c=new Cliente(explode(';',str_replace('"','',fgets($file)))); 
   
    $current++; 
    if($current < $offset || $current >= $offset + $limit){         
        continue; 
    } 
   
  $email=$c->getEmail(); 
 
  //$user = eZUser::fetchByEmail($email); 
  //$userID=($user instanceof eZUser)?$user->attribute('contentobject_id'):false; 
  $objects =& eZUser::fetchByName( $c->Codigo ); 
  if($email!=null && !($objects instanceof eZUser)){ // nova importação 
  //if($email){ // atualizar 
    echo "$email\n"; 
      //$user = new ezpObject('user', 3179, 14, 2,false,$userID); 
    $user = new ezpObject('user', 3179, 14, 2); 
        $first_name = strlen($c->NomeFantasia)>3?$c->NomeFantasia:(strlen($c->RazaoSocialNF)>3?$c->RazaoSocialNF:$c->RazaoSocialLD); 
 
        $last_name = strlen($c->RazaoSocialNF)>3?$c->RazaoSocialNF:$c->RazaoSocialLD; 
 
        $username = $c->Codigo; 
        $password = genRandomString(); 
 
        $user->__set("first_name", $first_name); 
        $user->__set("last_name", $last_name); 
        //if(!$userID) 
            $user->__set("user_account", "$username|$email|" . md5("$username\n$password") . "|2"); 
        $user->__set("code", $c->Codigo); 
    $user->__set("situacao", $c->Situacao); 
    $user->__set("codigopai", $c->CodigoPai); 
    //$user->__set("razaosocialnf", $c->RazaoSocialNF); 
    $user->__set("razaosocialld", $c->RazaoSocialLD); 
        //$user->__set("nomefantasia", $c->NomeFantasia); 
        $user->__set("setor", $c->Setor); 
        $user->__set("cnpj", $c->CNPJ); 
        $user->__set("inscricaoestadual", $c->InscricaoEstadual); 
        $user->__set("endereconf", $c->EnderecoNF); 
        $user->__set("bairronf", $c->BairroNF); 
        $user->__set("cepnf", $c->CepNF); 
        $user->__set("caixapostalnf", $c->CaixaPostalNF); 
        $user->__set("cidadenf", $c->CidadeNF); 
        $user->__set("ufnf", $c->UfNF); 
        $user->__set("responsavelnf", $c->ResponsavelNF); 
        $user->__set("fonenf", $c->FoneNF); 
        $user->__set("faxnf", $c->FaxNF); 
        $user->__set("emailnf", $c->EmailNF); 
 
    $user->publish(); 
 
  } 
   
} 
 
fclose($file); 
 
 
$script->shutdown(); 
?>

Modified on Tuesday 10 April 2012 2:29:22 pm by Thiago Campos Viana

Tuesday 10 April 2012 3:32:30 pm

Hello Dan, and welcome to the eZ Community.

Different content types is the basis of eZ Publish, through Content classes & attributes, each attribute being of a particular datatype, datatype being fully customizable and extensible to fit your needs: http://doc.ez.no/eZ-Publish/Technical-manual/4.6/Concepts-and-basics/Content-management.

About the enterprise edition, it is honestly much easier if you drop your local representative an email, imho.

Tuesday 10 April 2012 8:54:54 pm

Hello Dan,

As for importing external data, also have a look at SQLIImport; https://github.com/lolautruche/SQLIImport. Developed by one of the eZ Engineering team members. Very powerful, and extendable also.

Regards, Robin

Wednesday 11 April 2012 12:40:08 am

Thanks everyone for the fast response, im currently uploading it for install and will post back my results.

Wednesday 11 April 2012 2:59:03 am

I have run into issues with Ezfind activation.

Trying to run the ezpgenerateautoloads.php file I get errors like this.

Warning: require(Base/src/base.php) [function.require]: failed to open stream: No such file or directory in /home/****/public_html/ezpub/ezpub/bin/php/ezpgenerateautoloads.php on line 33
Fatal error: require() [function.require]: Failed opening required 'Base/src/base.php' (include_path='.:/usr/lib/php:/usr/local/lib/php') in /home/****/public_html/ezpub/ezpub/bin/php/ezpgenerateautoloads.php on line33

 

When i search the forums and google it tells me i need ezcomponents installed as its a dependency, however the last version I can find of that is 2010 which doesnt sound right to me.

http://ezcomponents.org/download
What am I doing wrong?

Wednesday 11 April 2012 9:25:02 am

Did you customize your config.php to point to the pear version of the components instead of the lib version?

Is the lib/ezc directory intact?

 

Wednesday 11 April 2012 9:26:55 am

Hi Dan,

 

To help you it will be good to know what version of eZ Publish and eZ Find  you've installed  so that I can try to replicate and see why isn't working.

If you installed a community edition then eZComponents should be already in your lib/ezc/ folder and they are seen without other steps. 

One thing to check, if you created a config.php, is to uncomment the line:

//define( 'EZP_USE_BUNDLED_COMPONENTS', true );

As said in the documentation of that file it's not required but is good for speed.

eZ Components can also be installed system wide using PEAR:

# pear channel-discover components.ez.no
# pear install -a ezc/eZComponents

 

Anyhow, don't give up, eZ Publish is powerful and flexible and I think could serve your case well.

 

All the best,

Virgil

Wednesday 11 April 2012 3:37:26 pm

Quote from Dan T :

I have run into issues with Ezfind activation.

Trying to run the ezpgenerateautoloads.php file I get errors like this.

Warning: require(Base/src/base.php) [function.require]: failed to open stream: No such file or directory in /home/****/public_html/ezpub/ezpub/bin/php/ezpgenerateautoloads.php on line 33
Fatal error: require() [function.require]: Failed opening required 'Base/src/base.php' (include_path='.:/usr/lib/php:/usr/local/lib/php') in /home/****/public_html/ezpub/ezpub/bin/php/ezpgenerateautoloads.php on line33

 

When i search the forums and google it tells me i need ezcomponents installed as its a dependency, however the last version I can find of that is 2010 which doesnt sound right to me.

http://ezcomponents.org/download
What am I doing wrong?

Just download community version with ezc, then copy lib/ezc to your installation lib folder.

Friday 13 April 2012 2:15:14 am

Thanks for the help ill work through the suggestions and report back

expandshrink

You must be logged in to post messages in this topic!

36 542 Users on board!

Forums menu

Proudly Developed with from