eZ Community » Forums » Developer » Import XML Data Topic
expandshrink

Import XML Data Topic

Import XML Data Topic

Wednesday 07 December 2005 10:09:59 pm - 54 replies

I created this thread to discuss further evolutions (and issues) of the ImportXMLData contrib. This should help leaving the contrib comment area to message directly related to the use of the contrib and not implementation detail.

Best regards

Olivier

Sunday 19 November 2006 11:39:43 pm

Hi all,

I need to import xml data for two different classes with relations (storing datatype ezobject_relations). Is it possibile in some way? Any idea to do this?

Any suggestion will be very appreciated. Thanks in adavance.

Maurizio Betti

Monday 20 November 2006 2:02:07 am

Hi Maurizio

my xmlimport extension should be able to do this. it isn't published in the contributions yet; i'm waiting for a response from olivier pierret. it's tested with the 'objectrelationbrowse' datatype, but 'objectrelation' and 'objectrelationlist' should work, too.

take a close look at these screensnaps:
http://pike.kw.nl/browse/files/pr...2006/ezpublish/XMLImport/screensnaps
and notice the relations in the xmlfile.

as you see, relations are nested. inside the tag of the relationattribute, there are entries of the right datatype with a set of fields that should together form a unique identifier of the related resource. the importscript will perform a search on those. if it finds multiple applicable objects, it will return a warning and use the first one; if it doesn't find any applicable object, it will only issue a warning. see the output screensnaps for examples. ofcourse, the related object should have been imported before the relating object, it should already exist.

if you think you need this, contact me (pike-ez [AT] kw.nl)

*-pike

Modified on Monday 20 November 2006 2:17:42 am by *- pike

Monday 20 November 2006 6:01:42 pm

I have a question about import data using ImportXMLData contrib.

In my job, I used this contrib to import data, with sucess. But, after install this in my client (using the same version of EZ, 3.7 with brazilian portuguese as default language), I have problems with some caracters. For example, in the word "não" the "ã" is replaced with a square.

Using a MYSQL Front End, I look the tables where the data is imported (I dont remember what) and the text is stored right. The word "não" for example, is stored "não". In the source code of the browser (IE - View - Source Code), the words are OK, but, the navigator show this page with squares replaced the accents.

In first time, i suposed was problem with the encoding of the page (in IE, View - Encoding), but changes in this nothing solve the problem. I try changed the encoding of the XML file, without sucess.

Searching in the forum, I foung that one solution was edit the PHP class and add some changes.($parser = xml_parser_create('UTF-8')blunk.gif Emoticon In my job, works perfectly, in the client results in "page not found".

To understand better the problem, the client send me your database, to compare with the original database, I found that in my database, a word like "não" is stored exactly. No matter if the data is inserted with EZ or imported, the text is stored exactly as inserted, and I have no problem to view in browser.

In the client database, with exception of data imported with XML, the data inserted using EZ replaced the accents with some codification. To my client, view of this data are OK, but view data imported no.

What is the solution for my problem? Is possible to import data using other file format, like Excel or Access?

Tuesday 21 November 2006 11:01:49 pm

Hi Maurizio

a complicated story, as usual with encodings happy.gif Emoticon

in your office, mysql4, ezp3.7,
- stuff in the browser looks ok
- the mysql source seems (?) ok

at your clients office, mysql unknown, ezp3.7,
- there are strange characters in the browser
- the mysql source seems (?) ok

then you notice something else:

in your clients database,
- strange chars inserted via the admin get some encoding
- strange chars inserted via the import are stored literally

in your own database,
- strange chars inserted via the admin are stored literally
- strange chars inserted via the import are stored literally

is that all correct ?
so at both offices, working with the admin is no problem,
but the databasecontent is different, one 'literally', one
'with some encoding'.

i have no clue happy.gif Emoticon) but consider where you want to go.
do you want to

- keep working at your clients office with that encoding, iow
- leave their setup (and admin) as it is
- find out what goes wrong with the import, or

- get everything working at the clients office like in your own office, iow
- compare their setup with yours, make it the same
- leave the importscript as it is, probably working by then

I would go for the 2nd option, compare the mysql version, ini files,
ezp settings, line everything up. there must be some difference.

*-pike

ps. did you add the <?xml header to your import file ?

Tuesday 28 November 2006 3:38:30 am

Thanks pike. You are right!

The problem was the encoding of XML file. In my office, with or without the encoding line in the XML file, the import are OK. In the other server, nothing. The XML file was created in Windows Notepad.

But, the client edit this XML file in a Linux machine, insert the encoding line, save the file and import again. And this time, the import data are OK. The same process in Windows notepad do not works.

Tuesday 20 February 2007 7:44:50 pm

Hello,

I try to integrate ImportXMLData contrib in a multi-langague site and I want to know if somebody has an idea on positivity to add the option to select site language in the imported xml data.

Sorry for my English

Modified on Tuesday 20 February 2007 9:10:16 pm by Badii Bouteraa

Thursday 22 March 2007 2:28:46 pm

Hi all;

I have installed the extantion and it works ok.
But it seem that I have memory problems... When trying to import files of ca. 60 KB (Not very big files) I receive this message:

<i>
Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 2 bytes) in /var/www/no.hia.ezweb/ezpublish/base/var/au/cache/trans/g-lowercase-635518252-utf-8.ctt.php on line 274
Fatal error: eZ publish did not finish its request
</i>

If I repeat the same action with the same file, this message appears:
<i>
Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 18000 bytes) in /var/www/no.hia.ezweb/ezpublish/base/lib/ezi18n/classes/ezchartransform.php on line 296
Fatal error: eZ publish did not finish its request
</i>

I added: <i> $attribute->setAttribute("data_text",utf8_encode($xml_content_string)) </i>
To the importXMLDatafunctioncollection.php file to import the norwegian letters into the database.

Any solutions?
Commandline? or other fixes.

Thanx, appreciate the help!

Saturday 31 March 2007 11:33:58 am

Hi Mauricio

Yes, there's been more post about the memory issue. There are 2 big factors in ImportXMLData: parsing the xml file and creating new objects.

I've loaded files of 900k with a 256Mb memory limit. It was slow, but it worked.
But when creating new objects, every object stays in memory (sortof). This can be improved in the script, but not taken away completely; something happens deeper in the kernel that eats memory for every created object.

>I added: $attribute->setAttribute("data_text",utf8_encode($xml_content_string))
>To the importXMLDatafunctioncollection.php file to import the norwegian letters into the >database.

good tip, thanks.

>Any solutions?
>Commandline? or other fixes.

I dont think commandline will help much (but havent tried). See
http://ez.no/community/forum/developer/memory_leak_on_content_publish
for other possible solutions. No final solution though ...

$2c,
*pike

Sunday 22 April 2007 12:11:43 pm

Hi,

I would like to use the multi-language aspect of an object in adding a language tag in my xml file.
How does eZPublish manage several languages of an object?Shoud I create a new version?Have you and idea?

Thx

Sorry for my english I'm a french guy too :p

Tuesday 24 April 2007 2:57:02 pm

----moderated: spam---

Modified on Tuesday 24 April 2007 3:16:29 pm by luci lucia

Thursday 16 August 2007 5:23:03 pm

I uploaded my XMLImport contribution to the contributions. It was originally based on this code, however, reworked back and front quite unrecognizable and in no way compatible anymore happy.gif Emoticon

This thread is used to talk about ImportXMLData, so let's keep it like that. If you have questions or comments about XMLImport, add comments to the contribution's page.

http://ez.no/community/contribs/import_export/xmlimport

thanks!
*-pike

Wednesday 26 December 2007 5:15:51 am

Merry Christmas.

Does this extension work on eZ 4.0?
I tried to import some data, but got these errors:

Error: eZModuleFunctionInfo::execute  	Dec 26 2007 01:35:49

No class 'XMLImportFunctionCollection' available for function 'XMLImport' in module 'XMLImport'

Error: eZTemplate  	12�� 26 2007 10:20:02

Cannot retrieve attribute of a NULL

How can it work on PHP 5?

Please...

Monday 06 September 2010 12:59:12 pm

This extension can be made to work with eZ Publish 4.0, but not eZ Publish 4.1 and above. This is because the image import functions use the ezimage class which is deprecated and removed from eZ Publish 4.1. Can anyone suggest how to use the ezimagefile class to replace the functionality of the ezimage class? It seems a shame for this extension to become extinct!

Monday 06 September 2010 1:51:02 pm

Hi Andy

You may consider SQLIImport : http://projects.ez.no/sqliimport happy.gif Emoticon

expandshrink

You must be logged in to post messages in this topic!

36 542 Users on board!

Forums menu

Proudly Developed with from