Today's blog post was written by Mike Dearing, Principal Developer at Sonoma Partners.
When attempting to import a CRM 2016 SP1 solution the other day, I was greeted with an error that I hadn’t seen in a while: “cannot insert duplicate key.” Typically, this means that someone created a new field in the target environment and the source environment with the same name but different casing such as “new_test” in the source and “new_Test” in the target. SQL ignores casing, but CRM detects the difference and assumes these are 2 separate fields, so it attempts to create the new column, causing the duplicate error message. When checking a trace though, the following was logged:
The dependent component Entity (Id=XXXXX) does not exist. Failure trying to associate it with CustomControlDefaultConfig (Id=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx) as a dependency. Missing dependency lookup type = EntityObjectTypeCodeLookup.
So, I began my Google journey. It turns out I was not alone here, and that many others had already documented their struggles and their progress. Everything pointed back to the customcontroldefaultconfig entity, as the trace suggested. There appear to be a few possible cases of bad data that can occur:
- There can be multiple instances of one customcontroldefaultconfigid occurring in this table.
- There can be multiple instances of a primaryentitytypecode occurring in this table.
- There can be instances of primaryentitytypecode in this table that don’t exist in this environment.
#1 and #2 may be closely related, but I at least confirmed that #2 was an issue for me. I also noticed #3 in my environment. For #3, it looks like Microsoft is bringing over the entity type code of an entity from a source environment and using it in a target environment, which is not a reliable practice for custom entities since you can’t ensure the type code will be the same between environments. For #1 & #2, I’m not certain how these come to exist, but it may be through the usage of segmented/partial solutions.
Each thread I found online suggested direct SQL detection and deletion. I didn’t immediately see a reason why, since this entity is queryable through the API, so I wrote a LINQpad script and checked it out. I queried the target environment’s metadata for all custom entity type codes to try to fix #3 from above. This returned one record, and the primaryentitytypecode in the result set actually said "none" since it was trying to convert a type code into an entity name but failing to find the entity in this organization. I went ahead and tried to delete the record, and no errors occurred. Looking in the database though, the record was still there. While frustrating to hit this dead end, at least now I understand why every thread suggested direct SQL cleanup, since API deletions appear to be ignored for this entity type. Here is the code for reference:
var connection = CrmConnection.Parse(connectionString);
var orgService = new OrganizationService(connection);
var entityFilter = new MetadataFilterExpression();
entityFilter.Conditions.Add(new MetadataConditionExpression("objecttypecode", MetadataConditionOperator.GreaterThan, 9999));
var attributeExpression = new AttributeQueryExpression();
attributeExpression.Properties = new MetadataPropertiesExpression("logicalname");
attributeExpression.Criteria.Conditions.Add(new MetadataConditionExpression("logicalname", MetadataConditionOperator.Equals, "donotreturn"));
var relationshipExpression = new RelationshipQueryExpression();
relationshipExpression.Properties = new MetadataPropertiesExpression("schemaname");
relationshipExpression.Criteria.Conditions.Add(new MetadataConditionExpression("schemaname", MetadataConditionOperator.Equals, "donotreturn"));
var entityQueryExpression = new EntityQueryExpression()
Criteria = entityFilter,
AttributeQuery = attributeExpression,
RelationshipQuery = relationshipExpression
var retrieveMetadataChangesRequest = new RetrieveMetadataChangesRequest()
Query = entityQueryExpression
var metadataResponse = (RetrieveMetadataChangesResponse)orgService.Execute(retrieveMetadataChangesRequest);
var entityTypeCodes = metadataResponse.EntityMetadata.Select(x=>x.ObjectTypeCode).ToList();
var ccdcsToDelete = orgServiceBCG.RetrieveMultiple(new FetchExpression(String.Format(@"
<condition attribute='primaryentitytypecode' operator='not-in'>
<condition attribute='primaryentitytypecode' operator='ge' value='10000'/>
</fetch>", String.Join("</value><value>", entityTypeCodes))));
foreach (var ccdcToDelete in ccdcsToDelete.Entities)
When discussing with Microsoft, they suggested manually removing the "CustomControlDefaultConfig" xml elements from each entity node in the solution import every time, which does appear to prevent the error from occurring since it isn’t attempting to insert the duplicate row any longer. This is pretty time consuming though, so if you are on CRM Online, see if you can get your support representative to clean up your customcontroldefaultconfigbase table for you. For On-Prem, you can go the direct SQL route and delete duplicate rows or rows that have no matching entity in the environment. Be extremely careful and take a database backup before doing any direct SQL deletion since this is unsupported and may have unexpected results. The following SQL should allow you to detect the rows for the three cases discussed above:
-- #1 Detect duplicate CustomControlDefaultConfigid
select count(CustomControlDefaultConfigid), CustomControlDefaultConfigid
group by CustomControlDefaultConfigid
having count(CustomControlDefaultConfigid) > 1
-- #2 Detect duplicate PrimaryEntityTypeCode
select count(PrimaryEntityTypeCode), PrimaryEntityTypeCode
group by PrimaryEntityTypeCode
having count(PrimaryEntityTypeCode) > 1
-- #3 Detect PrimaryEntityTypeCode that doesn't exist in this environment
select CustomControlDefaultConfigId, primaryentitytypecode
where PrimaryEntityTypeCode >= 10000 AND
PrimaryEntityTypeCode not in
where objecttypecode >= 10000
Microsoft said that this is in their product backlog and being worked on, but had no ETA for a fix. They also unfortunately didn’t have any details on what the cause is for these issues. If you’re encountering these as well, please consider opening a support ticket to help escalate a fix. Here’s to hoping for a new year with one less solution import error on each of our plates!