Enable CSV Import in Silverstripe GridFields

What are we Building?

Silverstripe offers the CsvBulkLoader class for importing DataObjects from a CSV-file in ModelAdmin views. There is however no simple solution to import DataObjects from a GridField displaying a one-to-many or many-to-many relationship. This snippet describes, how one can implement a similar user experience as the ModelAdmin version for importing DataObjects from a CSV-file for a one-to-many (or many-to-many) relationship.

The example stems from a relatively simple use case of importing a set of codes (Code) and appending them to a group (CodeSet). Each Code object has a string, the actual code, stored in the database, which should be unique inside its group (The uniqueness is also enforced by creating a unique key from the CodeSetID and the Code field).

How to Build it

To import DataObjects from a GridField, we need to do the following:

Create a custom subclass of the CsvBulkLoader to import the new objects
Create a custom controller to handle the import
Add a GridFieldImportButton to the GridField we want to start the import from

Note: while the process of getting an import button displayed and working is easily achieved by the mentioned steps, one should make sure that the imported file has the desired format, as the CsvBulkLoader class by itself does not validate the input.

The CsvBulkLoader Subclass

To import anything, we use the base CsvBulkLoader implementation and extend it, so it is able to handle a relation. To do this, we create a child class which additionally adds the relation to the imported object.

We do this by adding a second argument to the constructor (the same can be done using a setGroupID method, but the constructor way ensures that there must be a group ID present at instantiation). We then extend the processRecord function to inject the group ID into each record.

We then add some decoration fitting our DataObject to tell the bulkloader which fields it should consider during import and how to identify duplicates and end up with the following:


use CodeSet;
use DiscountCode;
use SilverStripe\Dev\CsvBulkLoader;
use SilverStripe\Dev\BulkLoader_Result;

class DiscountCodeCsvBulkLoader extends CsvBulkLoader
{

    private $group_id = null;

    public $columnMap = [
        'Code' => 'Code',
        'CodeSet' => 'CodeSet.ID'
    ];

    public $relationCallbacks = [
      'CodeSet.ID' => [
         'relationname' => 'CodeSet',
         'callback' => 'getCodeSetByID'
      ]
    ];

    public $duplicateChecks = [
      'Code' => 'Code',
      'CodeSet' => 'CodeSetID'
    ];

    /**
     * __construct - constructor, extended to take a target relation ID
     *
     * @param  string $objectClass the class to generate objects
     * @param  int    $groupID     the relation ID
     * @return void
     */
    public function __construct($objectClass, $groupID)
    {
        $this->group_id = $groupID;
        parent::__construct($objectClass);
    }

    /**
     * processRecord - modified record processing function, injecting the target
     * relation ID
     *
     * @param  array             $record    the record from the CSV
     * @param  array             $columnMap the column map to use
     * @param  BulkLoader_Result $results   results
     * @param  boolean           $preview   = false wether preview is active
     * @return int
     */
    protected function processRecord($record, $columnMap, &$results, $preview = false)
    {
        $record['CodeSet'] = $this->group_id;
        return parent::processRecord($record, $columnMap, $results, $preview);
    }

    /**
     * getCodeSetByID - return the relation by the value in the column
     *
     * @param  DiscountCode $obj    the newly created object
     * @param  string|int   $val    the value in the column that triggered the callback
     * @param  array        $record the entire row
     * @return CodeSet|null
     */
    public function getCodeSetByID(&$obj, $val, $record)
    {
        return CodeSet::get()->byID($val);
    }
}

DiscountCodeCsvBulkLoader.php

The Import Controller and Import Form

To handle an uploaded file and information, we have to create a custom controller. In the ModelAdmin version, this is taken care of for us. In the case we want to build our own import button, we must create and handle the form by ourselves.

This is easily done by creating a new controller. There are however some to make this approach work:

First, as we are using the controller with the singleton pattern to render our form, we need to tell the controller its url.
Second, we need to tell the controller that it should not render anything when opened directly.
Third, there should be some security check present, so our form is safe from abuse. This can be done by implementing a check in the action itself, but in our case, there will most likely be a ModelAdmin which allows us to use the generated permission. Note that the bulkloader does not check for canCreate permissions, so you are responsible to check for this in case your model is more intricate in terms of permissions.
Last, we must tell the form (and the subsequent action) into which group we want it to import the new codes. As we will pass the form to the button in the gridfield, we can directly specify the group on creation and pass it to the form action via a hidden field.

We finally end up with the controller:

use SilverStripe\Forms\Form;
use SilverStripe\Forms\FieldList;
use SilverStripe\Forms\HiddenField;
use SilverStripe\Forms\LiteralField;
use SilverStripe\Forms\FileField;
use SilverStripe\Forms\FormAction;
use SilverStripe\Forms\RequiredFields;
use SilverStripe\Control\Controller;
use SilverStripe\Control\HTTPResponse;
use DiscountCodeCsvBulkLoader;


class DiscountCodeImportController extends Controller
{
    private static $url_segment = 'importdiscountcodes';

    private static $allowed_actions = [
        'index' => 'CMS_ACCESS_DiscountCodeAdmin',
        'DiscountCodesUploadForm' => 'CMS_ACCESS_DiscountCodeAdmin'
    ];

    protected $template = "BlankPage";

    /**
     * DiscountCodesUploadForm - create the form for the upload
     *
     * @param  int $target = null this is the target ID of the relation. Given when the form is defined for the button
     * @return Form
     */
    public function DiscountCodesUploadForm($target = null)
    {
        $form = new Form(
            $this,
            'DiscountCodesUploadForm',
            new FieldList(
                FileField::create('CsvFile', false),
                HiddenField::create(
                    'target',
                    'target'
                )->setValue($target)
            ),
            new FieldList(
                FormAction::create('doUpload', 'Upload')->addExtraClass('btn btn-outline-secondary font-icon-upload')
            ),
            new RequiredFields()
        );
        return $form;
    }

    /**
     * doUpload - resolve the upload using the custom bulkloader
     *
     * @param  array $data the data from the form (contains the target relation)
     * @param  Form  $form the original form
     * @return HTTPResponse
     */
    public function doUpload($data, $form)
    {
        $loader = new DiscountCodeCsvBulkLoader(\DiscountCode::class, $data['target']);
        $results = $loader->load($_FILES['CsvFile']['tmp_name']);

        $messages = [];
        if ($results->CreatedCount()) {
            $messages[] = sprintf('Imported %d items', $results->CreatedCount());
        }
        if ($results->UpdatedCount()) {
            $messages[] = sprintf('Updated %d items', $results->UpdatedCount());
        }
        if ($results->DeletedCount()) {
            $messages[] = sprintf('Deleted %d items', $results->DeletedCount());
        }
        if (!$messages) {
            $messages[] = 'No changes';
        }
        $form->sessionMessage(implode(', ', $messages), 'good');
        // $form->sessionMessage('Sent this Form for '.$data['target'] , 'bad');
        return $this->redirectBack();
    }
}

DiscountCodeImportController.php

Which we then promptly connect to the routing:

---
Name: csvimport_routes
After: framework/routes#coreroutes
---
SilverStripe\Control\Director:
  rules:
    'importdiscountcodes': 'DiscountCodeImportController'

app/_config/routes.yml

Adding the Import Button to the GridField

The last step is to connect our import form to the GridField we want to import the file from. For this, we use the provided GridFieldImportButton and add it to our Gridfield config. As mentioned, we use the singleton pattern to create an instance of our import form and hand it to the button:

use SilverStripe\ORM\DataObject;
use SilverStripe\Forms\GridField\GridField;
use SilverStripe\Forms\GridField\GridFieldConfig_Base;
use SilverStripe\Forms\GridField\GridFieldImportButton;

class CodeSet extends DataObject
{
  //
  // ...
  //
  public function getCMSFields()
  {
    //
    // ...
    //
    $gridFieldConfig = GridFieldConfig_Base::create();
    $gridFieldConfig->addComponent(
      $importButton = new GridFieldImportButton('buttons-before-left'),
    );
    $importButton->setModalTitle('Import Discount Codes');
    $importButton->setImportForm(
      DiscountCodeImportController::singleton()->DiscountCodesUploadForm($this->ID)
    );
    $fields->addFieldToTab(
      'Root.Codes',
      GridField::create(
        'Codes',
        'Codes',
        $this->DiscountCodes(),
        $gridFieldConfig
      )
    );
    //
    // ...
    //
  }
  //
  // ...
  //
}

Conclusion

Following the steps above lets you enable a CSV-based import for a one-to-many relation. Many-to-many relations are also possible, but you will have to adapt the custom bulkloader class to handle these relations correctly. You can also handle multiple relations by adding more relation fields (and IDs) to the form.

The most important thing to consider when applying this to your own site or application is to ensure everything is secure. You absolutely need to make sure your form is protected from unauthorized use, as this could lead to unwanted imports to your database.

Enable CSV Import in Silverstripe GridFields

Development,

Silverstripe,