Layer2 Data Provider for SharePoint (CSOM): Specifications for LAYER2 Cloud Connector

The Layer2 Cloud Connector can be used to connect to almost any data source, even external SharePoint data - on-premise, internally or externally hosted, or in the Microsoft SharePoint Online / Office 365 cloud. This FAQ gives the full specifications of the CSOM-based Layer2 Data Provider for SharePoint that comes with the Layer2 Cloud Connector.

​The Layer2 Data Provider for SharePoint (CSOM) connects to lists and libraries* of:

• Microsoft SharePoint Server Subscription Edition
• Microsoft SharePoint Server 2019
• Microsoft SharePoint Server 2016
• Microsoft SharePoint Server 2013 including free Foundation
• Microsoft SharePoint Server 2010 including free Foundation
• Microsoft SharePoint Online / Office 365
• Microsoft OneDrive for Business

To retrieve data from SharePoint lists (e.g. contacts, tasks, events, or custom), calendars and document libraries* and optionally write-back changes.

Fig. Example connection string for the Layer2 Data Provider for SharePoint (CSOM) to connect to a Microsoft Office 365 list.

Connection String

The following parameters can be used in the connection string:

• Url: This is the URL of the SharePoint site where the list or document library is located or the full URL to the connected list, view or folder as pasted from browser. When full URL is used then List, View or Folder information is automatically parsed from URL. In all other cases the List parameter must be given. The View parameter optionally can be provided separately in the connection string. The URL information is mandatory.  It is recommended to use a view, e.g. the AllItems view, to reduce the amount of fields transferred. Using a view is relevant for performance. To sync more that 5.000 items, please make sure that your view is still allowed, best to use the unmodified allitems view in this case.
• Authentication: This setting specifies how the connector authenticates against the SharePoint server. It can be one of the following:
• IntegratedWindows: This is the default authentication method. With this method, the connector uses the current user account to log in on the SharePoint server with given URL. This server must be configured to allow Windows Integrated Authentication. If the provider is used in the context of an automatic background synchronization, the user that will be used to perform Windows Integrated Authentication is the service account of the Layer2 Cloud Connector Windows Service (a local service account by default). Please note that the account must have the required permissions on the Sharepoint server given with URL, means read for read-only operations, create and delete (normally contribute) for write.
• Windows: This authentication method is using a Windows Active Directory account to authenticate against a SharePoint server. If this method is used, it is necessary to define both parameter, User and Password, whereas the user is specified in the format DOMAIN\username.
• Anonymous: If the SharePoint server given by URL is configured to support anonymous authentication, this method can be used to connect to this SharePoint server without any authentication.
• Office365: This authentication method is depricated. If this is configured as the authentication method, User and Password must be provided. Please use the same values as in browser-based dialog, e.g. for Windows Live. This method will not work in case of federation, e.g. ADFS.
• Microsoft_Modern: This is the default authentication method to access Microsoft Office 365 instances and should work in most cases, even if the SharePoint site is connected to an ADFS. This authentication does not need any further connection-string settings, other than the URL of the connected system.
• IECookie: Use the Cookie Manager to set an authentication cookie before connect. Please note that this method has its limitations, e.g. limited cookie lifetime, no support for background update. [Deprecated]
• ADFS: This is the authentication method for accessing Office 365 using federation via ADFS. For this authentication type OnlineUser, User and Password must be provided (see below). How to know that ADFS must be used? If you normaly login to your Office 365 in browser with a specific company login page (instead of the plain Office 365 page) and with an user email address that contains your specific domain, this a clear indication to try ADFS authentication method.
• ADFSWindowsIntegrated: This is the authentication method for accessing Office 365 using ADFS with current user credentials. For this authentication type OnlineUser parameter is mandatory.
• SharePoint_FBA: The SharePoint Form Based Authentication (FBA) with user and password parameter required.
• RMUnify: "Authentication=RMUnify" allows an authentication to Office 365 which is federated via RM Education. RM Education Ltd. is one of the leading suppliers of ICT to UK education.
• Please note: Authentication setting is optional. If not provided, IntegratedWindows is used by default.
• User: This part of the connection string specifies the username for the account which is used to authenticate against the external SharePoint. This information must be provided if one of the authentication-methods Windows, Office365 or SharePoint_FBA is used.
• Password: This parameter defines the password for the account which is used to authenticate. It needs to be specified if Windows, Office365 or SharePoint_FBA is used as the authentication method.
• OnlineUser: This Parameter is used for ADFS authentication types. Please use the mapped user id used as online id for Office 365 authentication. This parameter is mandatory for ADFS related authentications.
• List: The list parameter specifies the SharePoint list, calendar or document library to be used as the data source. This could be the internal name, the display name or the ID/GUID of the list or document library. This setting parameter is mandatory if full Url to the list or view is not used in connection string.
• View: The view parameter is optional and can be used to define a specific data subset of a list or document library to be synchronized. The view can be created and configured on the SharePoint portal as usual. The view parameter will accept the name of the view as well as the URL of the view aspx site or just the name of the view aspx site. It is recommended to use view, e.g. the AllItems view, to reduce the amount of fields transferred and created in BDLC. Using a view is relevant for performance.
• Folder: The folder parameter is used in case you want to sync to any specific folder in a list or library. Please note that all content above the folder is completely out of scope. Using the folder attribute is especially helpful if files are synced in a specific folder of an already existing library. The "folder" parameter in connection string is not supported above the list view threshold (as it is implemented as a filter by SharePoint).
• SecureTokenService: This setting is optional and should not be specified in most cases. It is of relevance only in case Office365 authentication is being used. It defines the URL of the secure token service which is used for authentication. In most cases this should be https://login.microsoftonline.com/extSTS.srf which is the default.
• SignInUrl: This setting is optional and should not be specified in most cases. It is of relevance only in case Office365 authentication is being used. It is the site collection relative URL which is used to sign in after the authentication token has been retrieved from the secure token service. If omitted, it will be by default: /_forms/default.aspx?wa=wsignin1.0.
• Realm: This setting is of relevance only in some cases if the Office365 authentication is being used. It needs to be specified, if the URL which is used to access the SharePoint Online instance is not the default URL. SharePoint Online default URLs have the format https://myCompany.sharepoint.com. In case you have your own domain there, please use the realm parameter as follows:
  
  This URL is used in two different contexts: First it is used to identify the SharePoint instance to the secure token server (STS). In this context, the URL is called a Realm. Second, it is used to locate and access the SharePoint instance, for example in a browser, as a normal URL . If a different URL than the SharePoint Online default URL has been established to access the SharePoint Online instance, the URL will be for example https://mySharepoint.myCompany.com, but the realm will still be https://myCompany.sharepoint.com. In this case the connector will no longer be able to infer the realm from the URL.
  
  In this case the realm needs to be defined explicitly through this setting.
• Office365UserRealm: This parameter is used to query online user id information like it is a valid adfs user, adfs server url. Parameter is optional. Default value is 
  https://login.microsoftonline.com/pp910/GetUserRealm.srf .
• WsTrustVersion: This parameter is used to set WS-Trust version which defines message format of the Secure Token Server authentication token. Possible values are :"WSTrustFeb2005" or "WSTrust13". Default value is "WSTrustFeb2005". Parameter is optional and specific to the ADFS authentication.
• ADFSEndpointUrl: This parameter is used to define the local adfs server endpoint url for issuing adfs token. By default it is queried from Office365UserRealm by online user id. It is something like https://<your-sts-address>/adfs/services/trust/2005/usernamemixed. Parameter is optional and specific to the ADFS authentication.
• BatchReadItems: This setting is optional and specifies how many SharePoint items will be read at once from the server in one request. By default SharePoint allows to read up to 5000 items during one request. The default value for this parameter is 5000. This setting is relevant to performance and can be adapted to configuration.
• BatchWriteItems: This setting is optional and specifies how many SharePoint items will be committed together to the server in one update / insert / delete request. By default SharePoint allows committing up to 50 items together during one request. The default value for this parameter is 50. This setting is relevant to performance and can be adapted to configuration.

• FilterPath: This setting is the optional and specifies, how items in a list or library with folders should be read. Value “Recursive” tells that all items will read including subfolders, and folders are not listed in results. Value “RecursiveAll” gets all items plus folders. Default value is “Recursive”.

• FilenameFilter: Optional search pattern for filtering files by name or extension. Parameter accept standard windows file search pattern like *, ?. Aliases are FilenameFilter, FileNameFilter, filenamefilter.  

Select Statement / Query 

The CSOM provider does not support any query / select statement. Querying can be done by setting up an appropriate SharePoint list view externally and use it connection string.

Layer2 Data Provider for SharePoint (CSOM): Sample Connection Strings

• To connect to a list on SharePoint with current user access rights please use:
  Url=http://mydomain/sites/mysite/lists/myList/AllItems.aspx
• To connect to a Microsoft Office 365 list please use the following connection string:
  Url=https://mydomain.sharepoint.com/sites/mysite/lists/myList/AllItems.aspx; Authentication=Microsoft_Modern;

Mapping: Writing Created and Modified timestamps

Usually SharePoint maintains the fields Created and Modified. When a list-item is created in a SharePoint-list or file is added to a document-library, SharePoint automatically assigns the current date and time for the Created field. Likewise, whenever an entry changes SharePoint will update the Modified field.

However, if the fields are mapped, the dates of the source are written/synchronized instead.

Due to its intended internal usage, there are some things to consider when writing to those fields:

• The user, which is used to synchronize, needs to have Site-Collection administrator permissions; otherwise, Created and Modified are read-only.
• If checkout/checkin is enforced on a document-library the Modified field will always be overwritten with the current date on check-in.

Layer2 Data Provider for SharePoint (CSOM): Use Cases

• ​Any-to-SharePoint connections: Can be used to sync almost any data source with native SharePoint lists or libraries*.
• SharePoint-to-SharePoint connections: Retrieve data from a SharePoint list or library, e.g. same or other site collection, other SharePoint installation (including externally hosted).
• On-premise-to-cloud connections: Sync local local calendars, tasks, contacts etc. with Office 365, e.g. for mobile access outside company.

SharePoint Integration - Known Issues and Workarounds

• Synchronization to or from a SharePoint Asset Library is not supported.
• SharePoint list attachments are not managed at this time and are not supported.
• Versioning is not fully supported. If versioning is enabled on the document library, the provider might create multiple versions for a single update.
• SharePoint Online/O365 supports files up to 2GB. Larger files will be rejected.
• For other SharePoint restrictions, see Software Boundaries and Limits for SharePoint 2013 and Blocked File Types.​​
• Editor, Author, Created, Modified, ContentType, and HtmlFileType field can only be written to by a site administrator. If you get errors that these fields are read-only and the account being used is a site administrator, it may be due to a known SharePoint issue where it denies acess if "AddAndCustomizePages" is set to Deny for the user. To fix this you must explicitly give "AddAndCustomizePage" permissions to the account. For more information, see this article.

Next Steps

​​Register, download and evaluate the free shareware edition of the Layer2 Cloud Connector today here.  

*Libraries can not be connected in the Layer2 Cloud Connector List and Table Edition. 

Ready to go next steps?