HOW TO SYNCHRONIZE SHAREPOINT LISTS AND LIBRARIES

This "how-to-connect" documentation will explain the initial configuration of an example connection between two SharePoint Online Document Libraries in Office 365.

This guide presupposes that you have installed the Layer2 Cloud Connector and that you are familiar with its basic functionality. The Layer2 Cloud Connector User Documentation will provide you with all necessary information.

Contents

1. Setting up the SharePoint Libraries

2. Configuring the Layer2 Cloud Connector

2.1 Creating a new connection

2.2 Configuring the Data Entity 1

2.3 Configuring the Data Entity 2

2.4 Running your connection

3. Hints and known issues

3.1 Connection direction

3.2 Bandwidth

3.3 Limit of files

3.4 SharePoint view threshold

1. Setting up the SharePoint Libraries

On your Sharepoint tenant, please search for the documents library that you will be getting the data from and the documents library where the data should be written to.

You must copy and save the url to this libraries, in my case I copied the urls:

• https://layer2.sharepoint.com/sites/cloudconnector/Layer2DocumentsOrigin/Forms/AllItems.aspx
• https://layer2.sharepoint.com/sites/cloudconnector/DestinationFiles/Forms/AllItems.aspx

Later on step 2.2 we will use these urls to build our Layer2 Cloud Connector connection string.

2. Configuring the Layer2 Cloud Connector

2.1. Creating a new connection

Create a new connection by using the Create New Connection option in the Actions pane (right-hand side). The new connection will appear at the bottom of the Connection Manager List (left-hand side). Click on your newly created connection to open the connection configuration settings.

Choose a meaningful name for your connection and replace the current "New Connection" Connection Title with it.

The Connection can be bi-directional. An initial connection should always be uni-directional to assure that both data entities are identical before switching to bi-directional. Therefore, choose Left to Right as Direction. You can change this setting after your initial synchronization finished successfully.

2.2. Configuring the Data Entity 1

We will now set up our Data Entities. Go to the data entity “Data Entity 1” to open the configuration settings.

Choose a Data Entity Title. It is recommended to give your entities meaningful names to maintain an overview when you decide to set up multiple connections.

Select the Data Provider for SharePoint from the data provider list. You can search for Sharepoint(Layer2) by typing into the selection box.

For more information about the SharePoint provider please visit Layer2 SharePoint provider Specification. 

For the Connection String, we need the in step 1 mentioned information. You can copy the below connection string and adjust it to match your information. Use the Verify Connection String option to evaluate if the provided connection string is valid.

Url=https://layer2.sharepoint.com/sites/cloudconnector/Layer2DocumentsOrigin/Forms/AllItems.aspx;
Authentication=Microsoft_Modern;

• "URL=https://your_sharepoint_library_url/Forms/AllItems.aspx;" this property defines the URL of your SharePoint library that will be addressed.
• "Authentication=Microsoft_Modern;" this property will determine the authentication method used. The Microsoft_Modern authentication method connects to your SharePoint online tenant using the Microsoft ADAL auth method. This method will not work in case of federation, e.g. ADFS. For further authentication method you can consult the provider specifications linked above.

Save your changes by using the right-hand pane option Save Changes.

To check if all necessary columns are received, you can use the Preview Data option on the right-hand pane which will provide you with a pop-up window showing the data from your Document Library.

2.3. Configuring the Data Entity 2

Now we are going to send the data to the destination Document Library on your SharePoint Online. It's required that you set up this library prior to the next steps. Your library should contain matching columns according to your source entity.

Use the left-hand pane to switch to the data entity "Data Entity 2". We will be using the Layer2 SharePoint Provider as well. 

You can copy the below Connection String which contains the minimum of required properties to connect to your custom SharePoint Online Document Library.

Url=https://layer2.sharepoint.com/sites/cloudconnector/DestinationFiles/Forms/AllItems.aspx;
Authentication=Microsoft_Modern;

• "URL=https://your_sharepoint_library_url/Forms/AllItems.aspx;" this property defines the URL of your SharePoint library that will be addressed.
• "Authentication=Microsoft_Modern;" this property will determine the authentication method used. The Microsoft_Modern authentication method connects to your SharePoint online tenant using the Microsoft ADAL auth method. This method will not work in case of federation, e.g. ADFS. For further authentication method you can consult the provider specifications linked above.

Save your changes by using the right-hand pane option Save Changes.

In the next step, we will configure our mapping settings. Click on the Mappings option on the left-hand pane. If your fields from SharePoint are named identical to the fields from your source system, the Enable Auto Mapping option will match those columns. Disabling this option allows you to match your columns as needed. We enabled auto-mapping in our setup. Save your changes by using the right-hand pane option Save Changes.

2.4. Running your connection

To run your connection switch back to the main connection configuration node and use the Run Now Button located on the bottom of the setup page. The Run Synchronization Toolbox will also display the synchronization process. 

Below is a data preview of the information we have accessed in our source entity:

This will be the result in our SharePoint Online library after our initial successful synchronization:

If you want to use a bi-directional syncronization, you can now switch your connection direction after our initial synchronization finished successfully. After adjusting the direction, you should check your Mappings settings again because some systems might include read-only columns that cannot be mapped directly.

We also recommend to choose a Conflict Resolution that matches your enviroments needs. You can find out more about the different conflict resolutions in our Layer2 Cloud Connector User Documentation.

3. Hints and known issues

• Data queries are not supported. Please make use of views to select your data.

• In case of SharePoint libraries, files are synchronized as well as metadata. List are for metadata only, attachments are not supported at this time.

• Microsoft SharePoint Server 2007 and earlier is not supported.

• Synchronization to or from a SharePoint Asset Library is not supported.

• SharePoint list attachments are not managed at this time and are not supported.
• SharePoint Online/ Office 365 supports files up to 2GB (via CSOM). Larger files will be rejected. See Software Boundaries and Limits for SharePoint 2013 and Blocked File Types.
• Path length limitations on the SharePoint side could cause issues while synchronising files that contains large path length names.
  Microsoft is about to increase the length of the url path in SharePoint Online and OneDrive for Business to 400 characters, see Invalid file names and file types in OneDrive and SharePoint. That will fix some possible synchronization issues on the SharePoint side, as this limit was lower than the maximum Windows file path length before.

• Editor, Author, Created, Modified, ContentType, and HtmlFileType field can only be written to by a site administrator. If you get errors that these fields are read only and the account being used is a site administrator, it may be due to a known SharePoint issue where it denies acess if "AddAndCustomizePages" is set to Deny for the user. To fix this you must explicitly give "AddAndCustomizePage" permissions to the account. For more information, see this article.

3.1. Connection direction

This connection supports uni-directional as well as bi-directional synchronizations.

3.2 Bandwidth

Take care about bandwidth limitations and Office 365 throttling, especially for the first sync. Estimations are displayed if using the toolbox for first sync. Estimations can change during the sync process, stay patient. You can stop and restart the sync whenever you want.

3.3 Limit of files

While there is no hard limit for the amount of files to sync, keep the number of documents per library (connection) as low as possible. To keep 100.000 documents in sync should not be problem, a powerful configuration can do much more. But consider the SharePoint / OneDrive limitations, and your Office 365 plan.

3.4 SharePoint list view threshold

Please be aware of the SharePoint List View Threshold. The List View Threshold is triggerd when over 5.000 records are returned in a view that contains any kind of sorts and/or filters. If a result returns over 5.000 records it could come to issues. You will need to either remove all sorts/filters on the used view or make sure less than 5.000 records are returned by it. Please note that targeting a specific folder is considered to be a filter option.

↑Top