Most Powerful Open Source ERP

How To Import CSV Data

How To demonstrating how to import CSV files into ERP5.
  • Last Update:2016-02-11
  • Version:001
  • Language:en

How to demonstrating how to ingest CSV data into ERP5

Table of Contents

  • System Preparation
  • CSV File Preparation
  • Problems when importing
  • Related Articles

    System Preparation

    First install ERP5 and some modules, then install erp5_csv_style Business Template. Now create Script (Python) object with id Base_importCsvLine in portal_skins/custom folder with content:

    if object_property_dict.has_key('uid'):
      object = context.portal_catalog.getObject(object_property_dict['uid'])
    else:
      object = context.newContent()
     
    # activity doesn't support security rights yet...
    for key in ['uid','id']:
      if object_property_dict.has_key(key):
        object_property_dict.pop(key)
    
    object.edit(**object_property_dict)
    

    (For now original version of Base_importCsvLine has little problem).

    CSV File Preparation

    Create CSV file, which is like in this example:

    "title" "corporate_name"  "vat_code"  "default_address_region"  "default_address_city"  "role"
    "title" "corporate_name"  "vat_code"  "default_address_region"  "default_address_city"  "role"
    "Ventis"  "Ventis s.c."  "1234567889"          "1/1"               "Tarnowskie Góry" "producer" 
    

    Note:

    • First two lines containing name of fields, which you would like to import, eg in case of Organisation (TODO: second will be ignored, isn't it?):
    • Third and next lines are with data, like
    • Every line shall be separated with enter. To have special characters in your data
      • replace " with ""
      • replace newline-character with TODO (I suggest to replace newline with some special combination of charaters, to made it post-import praseable.)

    Problems when importing

    The best way to analyze problems is to turn off activities. To do so replace:

    method(priority=4, activity="SQLQueue", passive_commit=1).Base_importCsvLine(attribute_value_dict)
    

    with:

    context.Base_importCsvLine(attribute_value_dict)
    

    in Base_importCsvFile. (Of course copy it from erp_core skin folder to custom before customization).

    Note: importing take some time. Long time sometimes - in our case it took longer than writing this wiki-page and drinking much of wine (working on Duron 600 with 768MB of RAM).

    Special thanks to Tomasz Brzezina for support and testing data.

    Related Articles