CRM import

Import is a process that you can define in Synerise in order to import your data continuously or at a scheduled pace. You should use this feature if you want to import, visualize and manipulate your historical data and/or the data that will be collected in the future. You can import different types of data: clients, transactions, promotions, vouchers and product feed. Here we will focus on the clients data.

Synerise allows a backend synchronization of clients through API. This enables you to import and update clients based on the unique identifier which can be an email or a custom ID (f.e. a unique number in your database). More about user synchronization can be found in API documentation.


Before you start…

API key

As we deal with API synchronization, please make sure you have an API key in your business profile that enables you to update clients data before you start creating an import.

Screenshot of the settings menu

  1. Click the Settings icon on the left side panel,
  2. Choose Api keys A list of your Api keys will be displayed.

Screenshot of the API key list

  1. Create a new API key.

Screenshot of the new API key details

Give the new API key a name and choose Business profile as Audience.

Screenshot of the new API key details

In the Scope section, check at least Client and Client account management options and click Save. 2. Alternatively, you can click the name of an existing API key.

  • In Audience section, make sure the Business profile is selected.

Screenshot of the API key details

  • Scroll to the bottom of the page to the Scope section, then make sure the options Client and Client account management are checked.
    • Check those options if necessary and click the button Update

If you intend to import other data like Transactions, just check all the options.

Now your API key allows you to update clients’ data. Let’s return to the CRM section.

Screenshot of the CRM page


Import button

Screenshot from CRM module presenting list of customers

  1. You can import your customers’ data by clicking the import button, in the top-right corner of the screen.

This will take you to the wizard for configuring the import of your customers’ data. Once the wizard is completed, you will be able to import files automatically from a defined source and manually - by file upload.

Tip: If you are somewhat experienced with the import tool, you can also use a Custom import feature and combine several imports (such as clients, transactions or promotions) in the same wizard flow.

Create an import

You should have one of the files supported by the application.

  1. Comma-separated values file (CSV or txt).
  2. Json.
  3. XML (used only for product feed, do not use it for client import for now).

CSV or txt file

  • The CSV/txt file should have column headers and should be UTF-8 encoded.
  • Typical separators are accepted - comma, tab, vertical bar or semi-colon.
  • the text in the headers row should not contain whitespaces before or after the name.
  • there should be only one value in each column.

Screenshot of a CSV file example

The data in your file should approximately resemble the example given above.


JSON

The json file should be UTF-8 encoded.

Screenshot of a JSON file example

The data in your file should approximately resemble the example given above.


Import methods

There are three ways to import your data.

Screenshot from clients import

  1. Push – if you intend to send data continuously via webhook, a URL will be created and will receive your POST requests.
  2. Pull – if your data are stored on SFTP or HTTP and you want to import updates periodically.
  3. File upload - if you want just upload a file with data (up to 200 Mb).
  4. The steps of each import are listed below the buttons.

For every type of import, you will be asked to provide a Sample file. The file should be a representative sample of the data you intend to import (10 lines in CSV or a representative structure in Json). This sample will be used for mapping (the process of matching your data structure with the Synerise data system). The data provided in the sample will not be imported.

Once your import is created, you will be able to upload a file (not larger than 200Mb). Read more about the upload at the end of this article (section Import data from a file).


Push import and File upload

For those two methods the first step looks the same:

Screenshot from import - push import

  1. Here you can change the method you have chosen in the previous step.
  2. You should provide a sample file. This file is needed for data mapping and should be only a representative sample of the data you intend to import. The data provided in the sample will not be imported.

Pull import

Screenshot from clients import - pull import - SFTP configuration

A pull import will require a connection to your SFTP or HTTP storage, i.e. you should provide the access information to your storage. Make sure you have it when you start creating an import. You should also set the schedule, so the data will be imported at a defined pace.

Note: The imported files should not exceed 100Mb.

Screenshot from clients import - pull import - SFTP configuration

  1. Here you can change the method you have chosen in the previous step.
  2. You should provide the access information.
  3. You will be asked to schedule your import.
  4. You should provide a sample file. This file is needed for data mapping and should be only a representative sample of the data you intend to import. The data provided in the sample will not be imported.

SFTP Access

Screenshot from clients import - pull import - SFTP configuration

  1. Choose the type of your storage.
  2. Provide the SFTP address of your storage (e.g. sftp://example.com/) and Port (usually, this will be 22).
  3. Choose the type of your connection. Depending on your choice, you will have to provide additional information.

    • Guest requires no further information.
    • For User name this will be the Login and Password to your SFTP storage. (Your browser may try to autocomplete those two fields with your Synerise account credentials, which is wrong. You should provide here the credentials to your SFTP storage.)

    Screenshot from clients import - SFTP user name

    • Use a Public Key. Public key authentication allows you to log into an SFTP account with a key instead of a password. It always come with a private key. The private key will allow Synerise to prove the rights to connect to your SFTP. To set this connection, follow this steps:

    Screenshot from clients import - SFTP Key

    • Download and save the public key.
    • In a text editor, open the authorized_hosts file stored in .ssh which should be stored in your directory (for user1 it should be /home/user1/.ssh/authorized_hosts).
    • Still in the text editor, open the public key previously saved and copy the whole content.
    • Return to the authorized_hosts and paste the content of your public key at the end of the file.
    • Save the authorized_hosts file.
  4. If your data is stored in a directory, provide a path to it (e.g. /home/some-directory/).

  5. If you intend to import just some of the files stored in this directory, you can define a pattern for the file names to be considered. You can use the asterisk () sign to substitute any number of characters and the question mark (?) to substitute one character. Example: Let’s say, your files are named as follows: sales201901.csv, sales201902.csv, sales201903.csv, and so on. You can then create your pattern like this: ‘sales.csv’ or if you want to be precise with the number of characters after “sales”, you can type your pattern like this: ‘sales??????.csv’. If all the files in a directory should be considered, provide just one asterisk.

  6. Your files may be protected with an access key. If this is the case, you should check this box.

  7. You should then upload the access key.

  8. Once your Access information is provided, you can click Apply

API Access

Screenshot from clients import - pull import - SFTP configuration

  1. Choose the API configuration.
  2. Choose one of the available methods (GET, POST or PUT) and provide a URL endpoint,
  3. There are 2 types of connection: Guest and Authorization header. The former requires no information.
  4. For the latter, you will be asked to provide the Authorization header.
  5. Click Apply.

Schedule

Screenshot from clients import - pull import - schedule

  1. Select the interval between imports,
  2. Set the start time,
  3. You can also set the end date if you know when this import should not be repeated,
  4. Click Apply.

Sample file

This file is needed for data mapping and should be only a representative sample of the data you intend to import. The data provided in the sample will not be imported.

Screenshot from clients import - pull import - SFTP configuration

  1. Select the format of your file,
  2. Upload your file, or drag and drop one into this area,
  3. Click Apply.

Mapping

Data from the file you provided is displayed in a table (CSV or txt) or as a structure (JSON).

Note: Sometimes, a JSON file may be presented in a table as well. This happens when JSON file is flat (does not contain any embedded data), so the table view is more appropriate.

Screenshot from clients import - mapping - required and optional fields

On the left side of the screen, you will find a list of all the required fields for this mapping. This means that Synerise needs at least that data to identify your customers. This step is obligatory.

  1. You can map any number of the optional fields (listed below the required ones).
  2. On the right of the Required list header, there is a counter of correctly mapped required fields.

For a Clients import, the counter displays 0/1 at the beginning. This is because you only need to map one of the listed fields (Custom ID, Email or UUID). Mapping one of these fields will enable you to proceed to the next step.

Custom ID can be your custom identifier (e.g. from an external system) but it has to be unique. Email should be a valid email format (no uppercase, no spaces before, inside or after the address). UUID is an identifier composed of 36 characters, formatted according the UUID scheme (8-4-4-4-12).


Map your file

Screenshot from clients import - mapping - connecting data with fields

Mapping involves dragging and dropping actions, starting from the blue points on the right of each mappable field name to the column (CSV or txt) or input (JSON) displayed in the middle of the screen.

While dragging a connection, a line appears between the blue point and the cursor. When you hover a cursor over a column or an input that can be selected, it is highlighted with a light-blue background. When you drop the point on a desired input, a connection is established.

Screenshot from clients import - mapping - connected data with the field

  1. This connection is signaled by the name of the field displayed below the column header (CSV) or displayed on the right side of the input (JSON).
  2. The field turns green – this means that this field is connected. You can still drag another connection from the connected point, but the previous connection will be replaced by the new one.

Each field can be connected to one input only, but an input can be connected to many fields.


Mapping optional fields

Usually, you will want to import more information about a client than the identifier. You can find many Client information fields on the Optional list. This list contains all the fields other than the required ones available in your profile and in this type of import.

You can add your own custom fields if none of the default ones match your data. A new field will be added to the Optional list for this import and future imports.

Screenshot from clients import - mapping - add custom field

Mapping agreements (consents)

When updating clients’ information, one of the options you will probably use more often than others is Agreements.

This kind of data informs if a given client gave their consent to receive your marketing communication through a given channel.

Agreements concern 6 channels of communication:

  • Email,
  • SMS,
  • Push,
  • Bluetooth,
  • Rfid,
  • Wifi.

When preparing a marketing campaign, you should respect the will of your clients of not receiving any communication through a given channel from you. To import or update agreements you need to provide boolean data (true/false or 0/1). No other pair of values will be accepted. The pair of true/false can be written with uppercase i.e. pairs TRUE/FALSE and True/False will be accepted.

If you do not provide any information about the agreements, they will assume a default value of false.

Mapping attributes

Attributes are custom, user-defined items which allow to add any information about clients, other than that listed in the Synerise default fields. You should not create attributes with the following names:

  • email,
  • clientId,
  • phone,
  • customId,
  • uuid,
  • firstName,
  • lastName,
  • displayName,
  • company,
  • address,
  • city,
  • province,
  • zipCode,
  • countryCode,
  • birthDate,
  • sex,
  • avatarUrl,
  • anonymous,
  • agreements,
  • tags,
  • businessProfileId,
  • time,
  • ip,
  • source

For that data, you should use the default fields instead.

In order to create a new attribute, you should add a custom field, but you should type attributes. (with a dot) before the name of your attribute. Hence in this example:

Screenshot from clients import - mapping - add attributes

We will create an attribute new_attr. Let’s add it by clicking the plus icon.

Screenshot from clients import - mapping - add attributes

This new attribute appears as attributes.new_attr and as the last item on the list of optional fields (not in the Attributes list). This is because it is a new attribute which was not saved yet, but, once saved, it will be present on the attributes list the next time you create an import.


Transformations

Sometimes your data might not match the required field or format. Sometimes it may be different from what you intend to import. In these cases, you can use transformations.

Transformations perform a defined operation on your data in the context of one connection, so you can transform the same input for several fields differently or transform it only for selected fields.

Note: You can use only one type of transformation on each connection.

To make a transformation, just click its name in the column header (CSV) or on the right of the input (JSON).

Screenshot from clients import - mapping - transformation

Choose one of the available transformations. You can see a preview of your transformed data on the right side.

Math transformation

Screenshot from clients import - mapping - Math transformation

You can do simple mathematical operations (add, subtract, multiply, or divide).

  1. Type a number next to the mathematical operation you want to use
  2. Click Apply
  3. The operation appears on the top. You can undo it by clicking the x icon.
  4. The preview shows the data transformed with this operation.

The operations will be applied in the order you create them, regardless of the usual operator precedence.

Text

Screenshot from clients import - mapping - Text transformation

You can add some text before or after the values of a given column.

  1. Type the text you want to add.
  2. Decide whether you want to place it before or after the values.

Time

Screenshot from clients import - mapping - Time transformation

You should use this transformation every time when you are not certain if your time format (date and hour) is in accordance with the Synerise time format. You should choose the format of your data, either from the list of the most common formats or you can use the strftime reference to provide the pattern of your format. Synerise will transform it automatically to the correct format.

Screenshot from clients import - mapping - Time custom transformation

  1. Click the link. You will be directed to the external page where you can learn how to create a pattern for your specific date/time.
  2. Paste your pattern here and test it. If it matches the format of your date/time, then your transformation is correct.

Catalog map

To make this transformation, you need two things:

  • A column with data that you need to transform,
  • A catalog stored in the Synerise Assets with at least two columns – one serving for reference and the other with values you want to actually use.

You can think of this catalog as a dictionary which will translate one data into another. For example, using this transformation you can obtain country codes (UK, FR, PL, US, etc.) out of the full country names (United Kingdom, France, Poland, United States, etc.) if you have both listed in your catalog.

Screenshot from clients import - mapping - Catalog map transformation

  1. You should select a catalog stored in Assets.
  2. Here you should type the exact name of the column in the catalog which will serve as the reference.
  3. In the case when there is no data in the cell of your input file, you can define a value to be used instead of it being blank.
  4. Here you have to type the exact name of the column from which you want to obtain the new (transformed) values.

Edition of a transformation

Screenshot from clients import - mapping - Add a column

Once a transformation is created, it is signaled by an icon of checkerboard squares in a circle placed on the left of the field name. You can click on this icon or the field name to edit (or remove) a transformation. By removing or changing a connection to a field, you will also remove the transformation.


Add column / input

You can add a column or input to your sample. The added columns will appear at the end of the table (CSV or txt) or at the top of the structure (Json).

Screenshot from clients import - mapping - Add a column

You have to define the column name and the data that will fill its rows.

By default, you can fill the rows with Plain text (the same text will be placed in every row).
You can choose other types of inputs by clicking the list icon on the right of the input field. The other options are:

  1. Currency - Synerise requires information about the currency every time a monetary amount occurs in your import. If you do not have it in your example, you can choose some of the most common currencies (the same currency will be placed in every row).

  2. Source - there are 4 options (Web, Desktop, mobile and POS). That is the only data accepted in Synerise as Source (the same source will be placed in every row).

  3. Tags - you can add tags (existing or new) to the data you are about to import. Once the column is created, you should connect it with the output Tags (the same tags will be placed in every row).

    Note: Tags can only be sent as a list (even if there is only one). If you try to send tags as a simple string or anything else, an error will occur and your import will not work properly. The correct syntax of a list is: ["item1","item2","item3"]

  4. Template - (available only in the table view) You can use Jinja expressions to combine several columns and/or mix them with strings, mathematical operations and so on. Depending on your expression complexity, to obtain a desired result, you may need some programing skills. To make a reference to a column in your file, you should use the pattern: {{root.ColumnName}} where ColumnName is the name of the column you want to target. The column name cannot contain a space or a special character (only basic letters, digits, hyphen and underscore) and you should type the name exactly as it appears in the table (or you can copy-paste it). The first character of this name must be a letter. The result should appear in the table rows so you can verify if your template works as intended.

Tip: If you make a mistake in your template, you will not be able to correct it later on. So, if you are not sure whether your template will work or not, copy it before saving and reuse if needed.

Summary

Verification of all the data you provided and a list of mapped fields.

Screenshot from clients import - import data from file

Take time to read all of the summary before proceeding to the next step as you will not be able to make any changes afterwards. If you need to change or complete something in your import, just go back to the appropriate step. No data will be lost when you return to the previous steps.


Saved import mapping

Screenshot from clients import - import data from file

Once your import is saved, you can see it on the imports dashboard. Go to Settings → Import from the icons panel on the left. This is the place where you will find all your imports and control them. You can switch an import off or on, check its type and upload a file that comply to the mapping of a given import.

Screenshot from clients import - import data from file

By clicking a chosen import, you can check its status, i.e. the progress of the ongoing process and information about the past processes.


Import details

Screenshot from clients import - import details

Here you will find a summary of your import, i.e. the configuration you have made and the mapping of your sample file. In a push import, you will see the webhook URL to which your data should be sent. In a pull import, you will see the schedule settings and the button which allows to run an import immediately, in addition to the scheduled ones.


Progress

Screenshot from clients import - import progress

Here you will find information about the recent activity of your import. You can check at which stage it is and if there are any issues.


Logs

Screenshot from clients import - import logs

In the Logs tab, you will find more precise, yet mostly raw information about what happened along the way. This is especially useful for developers when issues occur during the import.

Possible issues

  • Wrong type of data. You may encounter errors when you try to provide a certain type of data to the field accepting another type of data, e.g. you will get an error if you try to import a string to a field expecting a number. Another common error is importing random data to a boolean value (which accepts only true/false or 0/1).
  • List. An error will also occur if you try to import a string or a number to a field expecting a list of elements, even though you intend to import just one value, it should be provided as a one-item list.
  • Wrong format Your data may contain errors like wrong format of email, phone or date/time. This may cause an error during the import.

In case of doubt, you can check all the information about the data format in the API documentation.


Import data from a file

Regardless of your import method, you can always import data from a file aside of the regular schedule or continuous import. The file should not exceed 10Mb.

Click Import data from CSV (you will see here the file format used in the selected import).
Screenshot from clients import - import data from file You will be asked to upload a file and check if it matches the import mapping. Screenshot from clients import - import data from file


Note: Now you know how to create a client import and reuse it to add data from a file. You can read more about the CRM itself in the CRM list section.
😕

We are sorry to hear that

Thank you for helping improve out documentation. If you need help or have any questions, please consider contacting support.

😉

Awesome!

Thank you for helping improve out documentation. If you need help or have any questions, please consider contacting support.