Home Forums Plugins Product Importer Deluxe Support eCommerce Importer memory problems

Welcome to our community support forums! We're here to help - but if you have an urgent request for a Pro Plugin, you will get a prioritised response through our Premium Support page.

Viewing 15 posts - 1 through 15 (of 19 total)
  • Author
    Posts
  • #8672
    jlarysz
    Participant

    Well, I finally have the latest spin of the eCommerce Importer working correctly – on an unrestricted development machine. On the production hosting server, where I need it, it crashes after around 25 imports with the message

    Error 500 – Internal server error
    An internal server error has occured!
    Please try again later.

    I have already re-hosted 8 web sites to get enough memory to do this. My service now gives me an 80 Mbyte slice – pretty near the top of the offerings for this country – the good ol’ USA, I’ve been looking.

    Has anyone any suggestions for reducing the memory footprint of the importer? Something is accumulating on every cycle.

    #8739
    davidjeanneret
    Participant

    Ah – so other people are suffering the same!

    I’ve increased the memory limit for php.ini to 256MB and the timeouts to 20 minutes

    Still no joy (although mine gets up to around 70 products before 500 Internal Server Error

    My logs were showing some timeout issues, but it’s all like this now:

    [Mon Nov 14 23:23:12 2011] [error] [client 82.32.52.201] Premature end of script headers: admin.php, referer: http://www.flamingbeads.co.uk/wp-admin/admin.php?import=wpsc_pd
    [Mon Nov 14 23:36:32 2011] [error] [client 82.32.52.201] Premature end of script headers: admin.php, referer: http://www.flamingbeads.co.uk/wp-admin/admin.php?import=wpsc_pd
    [Mon Nov 14 23:48:03 2011] [error] [client 82.32.52.201] Premature end of script headers: admin.php, referer: http://www.flamingbeads.co.uk/wp-admin/admin.php?import=wpsc_pd
    [Mon Nov 14 23:56:12 2011] [error] [client 82.32.52.201] Premature end of script headers: admin.php, referer: http://www.flamingbeads.co.uk/wp-admin/admin.php?import=wpsc_pd

    Michael – how do we get it working ASAP please?

    #8740
    davidjeanneret
    Participant

    I’m trying to import 272 products.
    The thing works completely succesfully if I don’t include an image column.
    I’ve uploaded the images to the server (in the current wp-content/uploads/year/month folder)

    If we include an image column in the csv that has the filenames (and I’ve gone so far as duplicating identical images to give them unique filenames per product in case that was an issue) – it fails having imported a random ammount of products and always less images than products!

    I’ve wasted in excess of 20 hours trying to get this to work and I’m getting quite cross!

    Michael – please help
    (cos 20 hours is a lot of time to spend trying to get something to work that I’ve already paid £60 for)

    #8741
    davidjeanneret
    Participant

    Also, why does it translate a quantity of 0 to be N/A in the stock column – which means it is still available for sale.

    Surely your plugin needs to put 0 in the stock column.

    #8742
    davidjeanneret
    Participant

    Well – solving the stock quantity issue was easy, I ran the following SQL:
    UPDATE `wp_postmeta`
    SET meta_value='0'
    WHERE `meta_key` LIKE '_wpsc_stock' AND `meta_value` <1

    I tried telling the importer that I would choose to upload images in the next step. Spent 25 minutes choosing the 272 images for each product. Hit the Upload and Import button. (went away and made a sandwhich) came back and the screen had refreshed to the start of the import products screen – no products or images imported.

    How can this be?

    #8743
    jlarysz
    Participant

    Well, I finally concluded that I wasn’t having memory problems on my hosting service any more – I was having execution time limits being hit and that the message was just a tad unfriendly. There was no way around this – I had already re hosted to beat the memory constraint. I’m ashamed to admit that I hacked the code and set up another import option – the code runs exactly as if images were being imported and thumbnails generated but instead of actually generating thumbnails, the details of what is needed are written to a file and I generate thumbnails separately on a development machine that I own and upload them to the internet host. I think my execution time limit is around 60 seconds; I can import at least 950 products this way at one pass. I’ll try a larger import when I have one ready.

    #8744
    neddynosh
    Participant

    Have you tried to update any of the products previously imported? i’m having trouble with the update option, it sets the measurements to zero for products already imported.

    #8745
    jlarysz
    Participant

    Not yet. What measurements do you mean? I haven’t dealt this them before.

    #8746
    neddynosh
    Participant

    sorry, the weight and its units, the length and units, width & units, height & units. its possible that you don’t use them so it won’t matter to you. The first import worked great for me with everything populated, its when you update the products its set back to default.

    #8747
    jlarysz
    Participant

    No, I don’t use these at all. I will give it a try though, next time I import.

    #8775
    Michael Visser
    Keymaster

    Hi guys, just catching up on topics, been focused on e-mails of late. Here’s a summary.

    davidjeanneret, I too have been looking at reproducing and putting a workaround in the import process to avoid the 500 error, I hadn’t looked at execution time limits though, these can be overridden by some hosts. Since dependance on server sessions has been removed it’s possible to add support for partial imports when a fatal error like the above occurs. I’ll be tinkering with this in the coming Plugin update.

    neddynosh, the measurement vales dropping off when using merge has been fixed in the most recent Plugin updates.

    davidjeanneret, good point. I’ll make the change so that if a value of ‘0’ is provided then it is considered out of stock. I’ve also added an option in the next Plugin update so that you can turn on/off the notify store owner of low stock levels within WP e-Commerce 3.8.

    #8806
    Michael Visser
    Keymaster

    The ‘0’ value issue has been fixed in the 3.0.2 Plugin update going out this morning, I’m working on a fix for the 500 Internal Server Error now. Stay tuned. 🙂

    #8953
    PaulCutcliffe
    Participant

    Michael

    Is there any news on the 500 Internal Server Error? We’re now wanting to go live with a brand new site, but can’t until we can do another import. I had hoped that rebooting the server would reset something so it might work for a while, or a few imports, again, but alas all I did was highlight an issue iwth the server that prevented it from booting properly! One stressed hour later & the server was back up & running, but sadly the import process still fails almost immediately with the 500 Internal Server error.

    Having convinced my colleagues & the client that your plugin was the only way to go when you have a few hundred products, I’m kind of out on a limb now. I have a feeling it might work if I delete all the products & then don’t select the update option during import, but I daren’t risk it in case we end up with no products at all!

    Any feedback today would be greatly appreciated – especially your thoughts on whether it’s more likely to work if not updating, if it’s worth importing wtih fewer columns – I could take out the category column, then put it back in & take out the image column – is that likely to help? A few of my categories go over 255 characters (which I know because it caused all sorts of truncation issues when exporting from Access!) – could that be an issue? And most of all of course, any idea when it will be properly fixed?

    Don’t me wrong, for now I’m happy to implement whatever workarounds you suggest, as I just need this site to do one or two more update imports so I can go live with it.

    hoping to hear from you today Michael,

    Paul Cutcliffe

    #8954
    jlarysz
    Participant

    Hi Paul:

    Are you sure it’s a memory problem? I had that, rehosted with a more generous service, and hit the next issue, which looked pretty similar. This time the error came from the php script execution time limit, which was way too small to handle the generation of all the thumbnails that ecommerce wants. I blew past this by hacking the code – never a good idea but like you, I was stuck. I set up an option that did everything the standard importer did when importing a product with images, with the images referenced in CSV columns, but without actually generating the thumbnails. Instead, it generates a file with a list of the thumbnails needed. I then use that file with a little php code and ImageMagik to generate the thumbnails on my own computers and upload them.

    If you can import a small number of products – say 20 or so, then it’s execution time and not memory that’s blocking you.

    It’s a little messy and not my proudest moment, but like you I was (and still am) stuck. I can readily import batches of over 1000 products in the 60-second php execution time I’m allowed; I’m nervous but happy.

    If you have any coding experience at all I’ll send you my code, but IT’S A HACK – sooner or later Michael has to take this one up properly. If you are not technical you might find this worse that keeping beating up on Michael.

    #8956
    PaulCutcliffe
    Participant

    The urgency for today has gone now, especially since I checked and found that we’re working in the UK, whilst it’s 4am Sat in New Zealand, so we’re unlikely to hear from Michael during what we call ‘today’, so we’ve manually added all the products to the additional categories, which is what we wanted our final import/update to do before going live. However, we do have some new product images, so we do still need it to work really soon, just not necessarily today.

    Thanks.

Viewing 15 posts - 1 through 15 (of 19 total)
  • The forum ‘Support’ is closed to new topics and replies.