I have my Dynamics ECMA retrieving 7000 contacts from Dynamics. I noticed that I only get 5000 contacts in the Connector Space (CS) after I do a full import. I checked my ECMA code and see this
private int m_importDefaultPageSize = 1000;
private int m_importMaxPageSize = 5000;
Of course I say, I have to increase the limit. So I increase it to 11,000, then I find out that ECMA has a limit for import of 9,999. See this Technet article
ECMA 2.0: ECMA 2.0 does not allow a page/batch size of greater than 9999
When configuring the batch size for a ECMA 2.0 based MA, you may not configure a batch size larger than 9999 objects. The UI will not allow a larger number to be configured, and there is no way to exceed this size in your Connector’s configuration.
Let me explain, this is a limit on how many objects you can return to the sync engine after an import operation. So let’s say I have 15, 000 users, you can save 15000 but at the end of you operation when you return the result to the Sync engine, the count cannot be greater than 9999, you have to do it in batches. It is not a limit on how many records you can read at once from the remote directory. You can request all records from the other system, the system returns 15000, but the ECMA limit occurs when you try to close out your import operation. So I increased it to 9000. My Import still only pulls 5000 contacts. I look at if there are any Dynamics limitations, yes there is
Dynamics has a search result limit of 5000 by default. See this Microsoft article. There are actually two steps/ways to removing this limit in Dynamics CRM.
Add a registry setting
- To turn off the paging feature and then ignore the MaxRowsPerPage parameter of 5,000, add the TurnOffFetchThrottling DWORD Value to the registry. To do this, follow these steps.
- Click Start, click Run, type regedit in the Open box, and then click OK.
- Locate and then select the following registry subkey:
- HKEY_LOCAL_MACHINE\Software\Microsoft\MSCRM
- On the Edit menu, point to New, and then click DWORD Value.
- Type TurnOffFetchThrottling, and then press ENTER.
- Right-click TurnOffFetchThrottling, and then click Modify.
- Type 1 in the Value data box, and then click OK.
- On the File menu, click Exit.
- Do IISREST
Update SQL via SQL Studio
Run this query
USE MSCRM_CONFIG
GO
Update DeploymentProperties
Set IntColumn=-1
Where ColumnName=’TotalRecordCountLimit’
NOTE:
-1 (or any negative number), will disable the limit but you can also change it to another limit. Note that you cannot lower the count below the number of rows displayed, which makes sense.
Once updated then DO an IISREST which will apply the changes. This is mandatory.
To increase the limit from 5000 to 10000 then change the query accordingly as below.
USE MSCRM_CONFIG
GO
Update DeploymentProperties
Set IntColumn=10000
Where ColumnName=’TotalRecordCountLimit’
Paging in Dynamics and in your ECMA
You have removed the pull limits in CRM, so the import now works well. But you can do paging in the way you retrieve records from CRM [See blogpost] and you can also do paging in ECMA (see blogpost) to save in batches. I will be taking about these two steps in another blog post
tlktechidentitythoughts 