Import/Export Entity Data¶

Import/Export Configuration¶

Import is a basic operation for any entity. The import operation is one step. See the following example configuration:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

# Oro/Bundle/ImportExportBundle/Resources/config/batch_jobs.yml
connector:
    name: oro_importexport
    jobs:
        entity_import_from_csv:
            title: "Entity Import from CSV"
            type: import
            steps:
                import:
                    title:     import
                    class:     Oro\Bundle\BatchBundle\Step\ItemStep
                    services:
                        reader:    oro_importexport.reader.csv
                        processor: oro_importexport.processor.import_delegate
                        writer:    oro_importexport.writer.entity
                    parameters: ~

The import algorithm being performed is (in pseudocode):

Process job:
  - Process step 1:
    - loop
      - read item from source
      - if source is empty exit from loop
      - process item
      - save processed item to array of entities
    - end loop
    - save array of prepared entities to DB

The OroBatchBundle provides the ItemStep class that executes each step of a job. In its doExecute() method, it creates a StepExecutor instance, passes a reader, a processor and a writer to it and executes it in the StepExecutor through the execute() method. After this step is done, all imported items are written to the destination.

The Import Process in Detail¶

For example, here is what happens in detail when you import contact data from a CSV file:

1. The CsvFileReader reads one row from the CSV file in its read() method and transforms it to an array representing the columns of that row.
2. The data being read is then passed to the process() method of the ImportProcessor class which converts the item to a complex array using the convertToImportFormat() method of the ConfigurableTableDataConverter data converter class.
3. The processor deserializes the item from the converted array using the Serializer class.
4. Optionally, the deserialized object can then be modified by the ConfigurableAddOrReplaceStrategy class.
5. Finally, the processed entity is returned by the processor and then passed to the EntityWriter class. This writer stores the data when its write() method is executed.

The Export Process in Detail¶

The export process is essentially the import process in reverse, except that it doesn’t use a strategy:

1. First, the EntityReader class reads an object.
2. Then, the ExportProcessor class serializes and converts the object into an associative array with property names as keys and the property values as values of the array.
3. The Serializer class normalizes each field and converts objects to complex arrays.
4. A data converter converts the associative array into a dimensional array.
5. Finally, all array entries are written to a CSV file by the CsvFileWriter class.

The export algorithm being performed is (in pseudocode):

Process job:
  - Process step 1:
    - loop
      - read entity from DB
      - if source is empty exit from loop
      - process entity
      - save plain array to array of items for save
    - end loop
    - save array of prepared items to DB

Serializer & Normalizer¶

One very important concept to know is how we normalize/denormalize relations between entities and other complex data.

The Serializer class extends the standard serializer of the Symfony Serializer component and has its own normalizers and denormalizers. Each entity that you want to export/import should be supported by the serializer. This means that you should add normalizers and denormalizers that will take care of converting your entity to the array/scalar representation (normalization during serialization) and vice versa, converting arrays to the entity object representation (denormalization during deserialization).

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

if ($data[$fieldName] !== null
    && ($this->fieldHelper->isRelation($field) || $this->fieldHelper->isDateTimeField($field))
) {
    if ($this->fieldHelper->isMultipleRelation($field)) {
        $entityClass = sprintf('ArrayCollection<%s>', $field['related_entity_name']);
    } elseif ($this->fieldHelper->isSingleRelation($field)) {
        $entityClass = $field['related_entity_name'];
    } else {
        $entityClass = 'DateTime';
    }
    $context = array_merge($context, ['fieldName' => $fieldName]);
    $value = $this->serializer->denormalize($value, $entityClass, $format, $context);
}

1
2
3
4
5
6
7
8
9

 /**
  * @ConfigField(
  *      defaultValues={
  *          "importexport"={
  *              "order"=200,
  *              "full"=true
  *          }
  *      }
  */

"Addresses 1 First name"

"Owner Username"

Extension of Import/Export Contacts¶

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

parameters:
    oro_importexport.reader.csv.class: Acme\DemoBundle\ImportExport\Reader\ExcelFileReader
    oro_importexport.writer.csv.class: Acme\DemoBundle\ImportExport\Writer\ExcelFileWriter

services:
    oro_importexport.reader.csv:
        class: "%oro_importexport.reader.csv.class%"

    oro_importexport.writer.csv:
        class: "%oro_importexport.writer.csv.class%"

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

# src/Oro/Bundle/ContactBundle/Resources/config/importexport.yml
parameters:
    orocrm_contact.importexport.strategy.contact.class: Oro\Bundle\ContactBundle\ImportExport\Strategy\ContactAddOrUpadteOrDeleteStrategy

services:

    orocrm_contact.importexport.strategy.contact.add_or_replace:
        class: "%orocrm_contact.importexport.strategy.contact.class%"
        parent: oro_importexport.strategy.configurable_add_or_replace
        calls:
            - [SetRegistry, ["@doctrine"]]

Learn more¶

Read the ImportExportBundle documentation for more information.