Sonoma Partners Microsoft CRM and Salesforce Blog

Dynamics 365 – Email Engagement

Continuing with our posts regarding the release of Dynamics 365, next up is the new Email Engagement functionality Microsoft has added to the core product.  Additionally, head over to the CRM Roadmap site, or the CRM What’s New site to see more of the features that have recently gone live. 

Email Engagement is the ability to see tracking statistics about emails that are created in, and sent out of Dynamics 365.   Or in other words, Email Engagement enables monitoring of activity taken by the recipient of the email.  Specifically you can monitor when the recipient:

  • Opened the message
  • Clicked on a link
  • Opened an attachment
  • Replied to the email

You also have the ability to schedule when the email should be sent based on when you think the most effective delivery time is for your message.  You can also set alerts to remind you when you should follow up on the email.  You can view additional information on Email Engagement from Microsoft via this link.

We’ll go into more detail in using Email Engagement below, but first lets discuss how to get it enabled.

Enable Email Engagement

In order to enable Email Engagement for Dynamics 365, you need to accept the Preview Terms and Conditions.  To do this, navigate to Settings –> Administration –> System Settings.  Then click on the Previews tab, and make sure you’ve checked off that you agree to the license terms, and enable Email Engagement.


On top of enabling the preview feature, you’ll also need to enable Server Side Sync for SharePoint and Exchange in Dynamics 365, and enable OneDrive for Business in Dynamics 365.

You’ll also have to enable document management for the Email entity, which can by done by going to Settings –> Document Management –> Document Management Settings.  Check the Email checkbox and you should be good to go here.

Finally,  you need to accept the Email Engagement terms and conditions in Dynamics 365 and set some other options (cards etc.).  This can be completed by going to Settings –> Relationship Insights.  With the initial screen, you’ll have to click on the “I have read and accept these terms and conditions” checkbox and click on Continue.  Then click on the Email Engagement tab, and click on the checkbox by “By enabling this feature, you consent to share your data…” and click on “Begin Setup.”


The setup process will then begin and could take some time, but once it completes, you’ll have to click on another checkbox next to “Turn on email engagement for your organization” and click on Save.  You should now be good to go to use Email Engagement!


Using Email Engagement

Note that Email Engagement only works for emails that are sent out of Dynamics 365 (versus emails sent from Outlook / Exchange that are tracked in CRM).

The first thing you’ll need to do is simply create a CRM email as you would normally.  However, now when you do so, you’ll see additional content on the email message in an Email Engagement section.


When new emails are created, they’re automatically followed.  You can click on the DON’T FOLLOW link to not follow this particular email.  However, there is a “Follow Email” field on the Contact that you can update to “Do Not Allow” and when doing so, the next time you create an email in Dynamics 365, by default the email will be set to not be followed.




You can click on View Preferences to see which recipients on the email have their setting set to Do Not Allow, and you can also attempt to retry following the email should the recipients’ preferences be updated.

Note:  That there seems to be a bug with this functionality currently at the time of this blog post.  When the email is initially created, it still shows that the “Recipient Activity will be followed” when the email form opens, regardless of the “Follow Email” preference on the contact record.  If you click on DON’T FOLLOW and then click on FOLLOW, that’s when you’ll see the indication above that one or more of the recipients prefer to not have their email activity followed.  A Microsoft Connect feedback item has been logged here.

From this area, you can also specify a time to actually send the email if you don’t want it to be sent immediately (based on research that you may have done that indicates best time to send specific communication).  Clicking the SEND LATER button will pop up a dialog that will allow you to pick a date and time for the email to be sent.


If you click on SET REMINDER you’re able to set one of the following reminders:

  • If a reply isn’t sent
  • If the email isn’t opened
  • Remind you regardless of lack of recipient activity


Both reminders and email scheduling must be done in the future, and the system will prevent you from setting them up for the past.  Once they’re setup, you’ll see the Email Engagement section change slightly which will allow you to modify the settings you setup, as well as remove them should you desire.


When you’re in CRM and looking at the relationship assistant, you’ll see a card for the reminders you setup that executed.  In this example, since the email wasn’t opened by the date specified, a card shows up for me indicating as such.  I can open the recipient, open the email, snooze the reminder card for 12 hours, or dismiss the reminder card.   Out of the box, these appear to only display on the Dashboards, and not on the actual Contact record.




You have the ability to not only follow activity on a particular email (opens, clicks, replies), but you can also follow activity on attachment views.  Each time an attachment is opened is counted as an attachment view.

In order to follow attachment activity, you must first save your email (don’t send it).  Then click on the + sign in the Attachments sub grid.

Once you Browse to the file and click on Attach, you’ll see a new Follow button.  After you click on the Follow button (which will upload the attachment to OneDrive for Business which you configured earlier), you’ll have the option to stop following, or simply close out of the attachments dialog.  You can always get back to the dialog to follow, stop following, and remove the attachment by clicking the attachment hyperlink in the attachment sub grid on the email form should you decide to do so later.





Recipient Activity

When you’re on an email that’s being followed, you’ll notice a new area Recipient Activity that shows a summary view of the number of opens, attachment views, link clicks, and replies.  Directly below the summary you can see each individual detailed activity which is rolling up to that summary.  Note:  I had to track the reply from the recipient for it to be counted as a reply (Dynamics didn’t automatically notice that there was a reply in Exchange and instead needed the reply in Dynamics to count).


You’ll notice that the location of where the email was opened is also tracked.  In this case I just opened it from my laptop, but it would also show if I opened it from my phone as well.

Recipient activity is also visible on dashboards in the new Relationship Assistant area as a separate card.  This is also displayed on the new Dynamics 365 tablet and phone apps, you can see the same card displayed in the new Landing Page


image image

The Email Engagement functionality extends a little bit beyond the actual email that was sent.  If you used an Email Template, Dynamics 365 will track the opens and replies that were made to emails that used that template.  Dynamics 365 will also let you know if a particular template is recommended due to past performance.  Below you can see that Dynamics 365 is recommending the Contact Reconnect email template due to past recipient activity. 


When a user goes to create a new email in the future, the recommended templates will be easily visible to select from via the green star indicator.


Topics: Microsoft Dynamics 365 Microsoft Dynamics CRM Microsoft Dynamics CRM Online

I’ve Got 99 Problems, And CustomControlDefaultConfig Is One

Today's blog post was written by Mike Dearing, Principal Developer at Sonoma Partners.

When attempting to import a CRM 2016 SP1 solution the other day, I was greeted with an error that I hadn’t seen in a while: “cannot insert duplicate key.” Typically, this means that someone created a new field in the target environment and the source environment with the same name but different casing such as “new_test” in the source and “new_Test” in the target. SQL ignores casing, but CRM detects the difference and assumes these are 2 separate fields, so it attempts to create the new column, causing the duplicate error message. When checking a trace though, the following was logged:

The dependent component Entity (Id=XXXXX) does not exist.  Failure trying to associate it with CustomControlDefaultConfig (Id=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx) as a dependency. Missing dependency lookup type = EntityObjectTypeCodeLookup.

So, I began my Google journey. It turns out I was not alone here, and that many others had already documented their struggles and their progress. Everything pointed back to the customcontroldefaultconfig entity, as the trace suggested. There appear to be a few possible cases of bad data that can occur:

  1. There can be multiple instances of one customcontroldefaultconfigid occurring in this table.
  2. There can be multiple instances of a primaryentitytypecode occurring in this table.
  3. There can be instances of primaryentitytypecode in this table that don’t exist in this environment.

#1 and #2 may be closely related, but I at least confirmed that #2 was an issue for me. I also noticed #3 in my environment. For #3, it looks like Microsoft is bringing over the entity type code of an entity from a source environment and using it in a target environment, which is not a reliable practice for custom entities since you can’t ensure the type code will be the same between environments. For #1 & #2, I’m not certain how these come to exist, but it may be through the usage of segmented/partial solutions.

Each thread I found online suggested direct SQL detection and deletion. I didn’t immediately see a reason why, since this entity is queryable through the API, so I wrote a LINQpad script and checked it out. I queried the target environment’s metadata for all custom entity type codes to try to fix #3 from above. This returned one record, and the primaryentitytypecode in the result set actually said "none" since it was trying to convert a type code into an entity name but failing to find the entity in this organization. I went ahead and tried to delete the record, and no errors occurred. Looking in the database though, the record was still there. While frustrating to hit this dead end, at least now I understand why every thread suggested direct SQL cleanup, since API deletions appear to be ignored for this entity type. Here is the code for reference:

var connection = CrmConnection.Parse(connectionString);
var orgService = new OrganizationService(connection);

var entityFilter = new MetadataFilterExpression();
entityFilter.Conditions.Add(new MetadataConditionExpression("objecttypecode", MetadataConditionOperator.GreaterThan, 9999));

var attributeExpression = new AttributeQueryExpression();
attributeExpression.Properties = new MetadataPropertiesExpression("logicalname");
attributeExpression.Criteria.Conditions.Add(new MetadataConditionExpression("logicalname", MetadataConditionOperator.Equals, "donotreturn"));

var relationshipExpression = new RelationshipQueryExpression();
relationshipExpression.Properties = new MetadataPropertiesExpression("schemaname");
relationshipExpression.Criteria.Conditions.Add(new MetadataConditionExpression("schemaname", MetadataConditionOperator.Equals, "donotreturn"));

var entityQueryExpression = new EntityQueryExpression()


                Criteria = entityFilter,
                AttributeQuery = attributeExpression,
                RelationshipQuery = relationshipExpression


var retrieveMetadataChangesRequest = new RetrieveMetadataChangesRequest()


                Query = entityQueryExpression


var metadataResponse = (RetrieveMetadataChangesResponse)orgService.Execute(retrieveMetadataChangesRequest);

var entityTypeCodes = metadataResponse.EntityMetadata.Select(x=>x.ObjectTypeCode).ToList();

var ccdcsToDelete = orgServiceBCG.RetrieveMultiple(new FetchExpression(String.Format(@"
                                <entity name='customcontroldefaultconfig'>
                                                <attribute name='primaryentitytypecode'/>
                                                                <condition attribute='primaryentitytypecode' operator='not-in'>
                                                                <condition attribute='primaryentitytypecode' operator='ge' value='10000'/>
                </fetch>", String.Join("</value><value>", entityTypeCodes))));

foreach (var ccdcToDelete in ccdcsToDelete.Entities)


                orgService.Delete("customcontroldefaultconfig", ccdcToDelete.Id);


When discussing with Microsoft, they suggested manually removing the "CustomControlDefaultConfig" xml elements from each entity node in the solution import every time, which does appear to prevent the error from occurring since it isn’t attempting to insert the duplicate row any longer. This is pretty time consuming though, so if you are on CRM Online, see if you can get your support representative to clean up your customcontroldefaultconfigbase table for you. For On-Prem, you can go the direct SQL route and delete duplicate rows or rows that have no matching entity in the environment. Be extremely careful and take a database backup before doing any direct SQL deletion since this is unsupported and may have unexpected results. The following SQL should allow you to detect the rows for the three cases discussed above:

-- #1 Detect duplicate CustomControlDefaultConfigid
select count(CustomControlDefaultConfigid), CustomControlDefaultConfigid
from CustomControlDefaultConfigBase
group by CustomControlDefaultConfigid
having count(CustomControlDefaultConfigid) > 1

-- #2 Detect duplicate PrimaryEntityTypeCode
select count(PrimaryEntityTypeCode), PrimaryEntityTypeCode
from CustomControlDefaultConfigBase
group by PrimaryEntityTypeCode
having count(PrimaryEntityTypeCode) > 1

-- #3 Detect PrimaryEntityTypeCode that doesn't exist in this environment
select CustomControlDefaultConfigId, primaryentitytypecode
from CustomControlDefaultConfigBase
where PrimaryEntityTypeCode >= 10000 AND
PrimaryEntityTypeCode not in
                select objecttypecode
                from entity
                where objecttypecode >= 10000

Microsoft said that this is in their product backlog and being worked on, but had no ETA for a fix. They also unfortunately didn’t have any details on what the cause is for these issues. If you’re encountering these as well, please consider opening a support ticket to help escalate a fix. Here’s to hoping for a new year with one less solution import error on each of our plates!

Topics: Microsoft Dynamics CRM 2016

Troubleshooting Currency Rollup Fields

Today's blog post was written by Ian Moore, Senior Developer at Sonoma Partners.

Rollup fields are an excellent feature in Microsoft Dynamics that allow you to compute aggregate data for an entity across child records. They work great with currency fields - but if you run into trouble, you must be aware of some native fields that help control currency values.

We recently encountered an issue where currency rollup fields were not calculating for a particular account – other accounts in the system were fine, but one record would give warnings when it was opened.

Ian moore 1

The warning seems to point to the exact issue; one that can be common if you are loading data from external sources. Any record that has currency fields will automatically have a Currency (transactioncurrencyid) field added to it in order to have the correct value.

Trying to refresh the rollup field through the UI caused the following error pop-up with the same message:

Ian moore 2

The log file from this error also pointed to missing a missing currency somewhere in the rollup calculation:

Ian moore 3

The logical next step is to use Advanced Find to locate any records in this rollup that do not contain data for the Currency field, as well as verifying that the account itself had a Currency set.

In our case, everything looked in order. Both the account and all child entities in the rollup field contained “US Dollar” as their currency.

So, why the error?

It turns out there is a second native field involved for any currency data in Dynamics CRM: Exchange Rate

Adding Exchange Rate to our account form, we could see it was blank:

Ian moore 4

How do we set this field? Any effort to populate it through the API were fruitless – the value was simply ignored. What worked? Update one of the currency fields on your record to any value, save, and the system will automatically set the Exchange Rate for you.

Ian moore 5

With this value in place, our roll up fields calculated as expected!

Ian moore 6

If you’re having trouble with currency rollup fields, just remember to check two pieces of data:

  1. Currency field on the parent record and all child records
  2. Exchange Rate field on the parent record and all child records

Thanks for reading! If you run into further problems with this or otherwise, feel free to reach out.

Topics: Microsoft Dynamics 365

Quick and Easy Trello to CRM Integration

Today's blog post was written by Angel Shishkov, Principal Developer at Sonoma Partners.

At Sonoma Partners, we use CRM to track our internal work tasks, among many other things. Some of our clients and projects use their own systems for work and defect tracking, like Jira, Trello, or Excel, so we often have to consolidate data between the systems.

Today, I am going to talk about a quick and easy integration we built to bring Trello cards into CRM tasks.


First of all, I will assume you know what Trello is and how it is used. In this particular case, we used the cards as the equivalent of defect reports, along with comments, image attachments, and moving the card between different lists to indicate a status change. The manual process was to have our QA open the card in Trello, review the comments, create a record in CRM, and manually copy the card contents into it. The goal was to automate this process and eliminate the manual copying of the card contents into CRM.


I’d like to qualify what I mean by “quick” and “easy,” since both of those are relative terms. As far as integrations go, this one is quick because you could set it up in less than a couple of hours, and it is easy because it does not require you to learn the intricacies of either system being integrated (Trello and CRM in this case). There is some code involved, and your mileage will vary based on your development experience.

We need a way to create a CRM Task from a Trello Card, so we are going to need a way for the user to trigger this action while they have the Trello card open. For this example, we are going to use a Chrome Extension to put a button in the Chrome toolbar for the user to click. The actual logic we will write to read the Trello card and create the CRM Task, however, is not specific to Chrome Extensions.

The way we are going to connect the two systems is where the “easy” part comes in – we will not use either the Trello, CRM API, or SDK to do this, so you do not need to learn either. We are going to pull the Trello card’s contents through the Trello JSON feed, which is a feature that allows you to get a card’s contents in JSON format simply by adding “.json” to the end of the card’s URL. Since the user already has the card open in their browser, this should be easy. As for creating a Task in CRM, we are going to cheat a bit and not actually create the Task, but rather open a new Task form and pre-populate the subject and description fields with data from the Trello card. The user can then review the Task, make any changes, and hit Save to create it in CRM.


We need to set up our Chrome Extension first, however, I will not go in-depth about Chrome Extensions since that isn’t the focus of this article. I will lay out the basic steps, and know that there is much more detail online for those interested.

  1. If you have Visual Studio or you have done this before, you can use the Chrome Extension project template available here.
  2. If you are setting up the extension folder manually, create the following folder structure:
    1. app
    2. app\img
    3. app\lib
  3. In the img folder, put a 19x19 pixel image file called, "icon.png." This will be the icon of the extension in Chrome.
  4. In the lib folder, put a copy of the minified jQuery library. You can download it here. In this example, we will use jquery-3.1.0.min.js
  5. In the app folder, create the following three files. We will add content to them later.
    1. json
    2. html
    3. js

Now you should have your three folders set up with some files in them. Let’s add some content.

Open manifest.json in a text editor.

  1. manifest.json is the file that tells the Chrome Extension what components are included, what permissions are required, etc. Paste the following and save the file. If you did not use the file names above, make sure to change them below as well.

Open popup.html in a text editor.

  1. popup.html is the little web page that will open when you click on your extension icon in the Chrome toolbar. We don’t strictly need a web page for a simple extension like this, but we will add one for future enhancements anyway. Our web page has a title and a button for the user to click. Paste the following and save the file.

Open popup.js in a text editor.

  1. popup.js is the JavaScript “code behind” for the HTML page we set up above. It will handle the button click, read the Trello card, and open the CRM Task form. We are going to paste the whole code in there, and I will explain the functions individually. There is one part that you will need to edit in this code. The part marked with a //TODO comment has the URL to the CRM org. Replace yourorgurl with your CRM org URL. Paste the following and save the file.
  2. We will begin at the bottom first. The part that start with “document.addEventListener” just tells the page to call our init() function when it has loaded (that is, whenever the user clicks the extension icon in Chrome, which causes the page to display).
  3. Back up at the top, “TrelloToCRMController” is just a nice way to encapsulate all our logic into one “namespace.” All our methods are inside this namespace.
  4. Taking the methods from the bottom again – the init() function is what is called when our Chrome Extension webpage opens. It finds the Create Task button on the page and attaches a click handler function to it. This function will get called when the user clicks the Create Task button and it is called btnCreateTask_Clicked.
  5. The btnCreateTask_Clicked() function is called when the user clicks the Create Task button. It uses the Chrome Extension API to get the currently active tab in the browser and pass it to the createTaskFromTab function.
  6. The createTaskFromTab(tab) function takes a Chrome tab as a parameter and is the main function that reads the Trello card and opens the CRM task. It goes through several steps:
    1. Append “.json” to the end of the URL of the current tab (which we assume to be an opened Trello card). You can try this in your browser – open a card in Trello, then edit the URL in your browser to add “.json” to the end. This will take you to a page that has a JSON representation of the card. JSON is a format that is easily read by JavaScript into an object we can use in code. There is a comment in the code that shows the basic JSON format for a Trello card, or you can use a JSON visualizer online to check it out yourself.
    2. Call getJSON to retrieve the JSON contents of the Trello card. We are using jQuery, which was included in the extension during setup, so we have access to the $.getJSON method. It is an asynchronous method, so the rest of the code is inside the callback function.
    3. We are going to set two fields on the CRM Task – the title and description. The title will be the card name, but we want the description to be a list of all the actions on the card – comments, attachments and moves from one list to another. These actions are stored in the card.actions collection, so we loop through all of those to construct a description string.
    4. There are different types of Trello card actions, such as “commentCard”, “updateCard”, etc. They have slightly different properties, so we have a case statement to process each. For “createCard” we just log who created it. For “commentCard” we record the comment text. “updateCard” is a move from one list to another, so we record the previous list and new list in the string. And for “addAttachmentToCard” we just record the URL of the attachment.
    5. Once the description string is prepared, the last piece left is to open the Task form in CRM. Forms in CRM can pre-populate their fields with data based on parameters passed in the URL. Specifically, there is a parameter called “extraqs” that can contain a list of name/value pairs of field names and their values. We are going to use this feature to open the Task form with a pre-populated subject and description. Finally, the newTaskUrl variable is the URL to a new Task form in CRM and we use the Chrome API to open a new tab to that URL.


This is the last step (besides all the bug fixing and enhancement requests that will come) in the example. Since we used a Chrome Extension to host our solution, we will need to install it in Chrome to test it out.

  1. Open Chrome, click the ellipsis on the top-right for the menu and open the Settings page. Select Extensions on the left.
  2. Check the Developer Mode checkbox on the top. This will display a couple of extra buttons. We are going to use Developer mode to quickly deploy our extension, so we don’t have to bother with deploying it to the Chrome app store.
  3. Click the Load Unpacked Extension button on the top. If you don’t see it, make sure Developer Mode is checked.
  4. Browse to your “app” folder and select it, click OK.
  5. You should see your extension appear in the list on the Chrome Extensions page. Make sure Enabled is checked and there are no errors. If you make any changes to the code, you can come back here and click Reload to update the extension with your latest changes.
  6. You should now see the extension icon you added during Setup show up in the top right of the Chrome browser.
  7. Open Trello, browse to a board and open a card to test with. With the card open in the Chrome tab, click the extension icon. A small webpage will drop down with a Create Task button. When you click it, a new Chrome tab will open with your CRM task. Congratulations!


This is a relatively simple and bare-bones integration. It is not very configurable, has no error handling and doesn’t look great, but it gets the job done. If you are looking to expand on this or build something much more complex and you need some help, give us a call. Thanks for reading!

Ease into the cloud with Microsoft Dynamics Lifecycle Services

Topics: Microsoft Dynamics 365

Artificial Intelligence: All the Rage with CRM Today

Today's blog post was written by Kristie Reid, VP of Consulting at Sonoma Partners.

The latest CRM buzz is around Artificial Intelligence. No, not the kind of AI that allows computers to take over the world (yet) or the latest winner of Jeopardy.  I mean the kind of machine learning that can process your CRM data to reveal some pretty amazing things. Some of the use cases thought of so far include:

  • Predictive analytics for lead scoring
  • Proactive notification to identify deals going south before the sales person recognizes what is happening
  • Understanding customer sentiment without needing to pick up a phone to see if they are happy or dissatisfied
  • Automated email creation that inserts the content before you can even think of what to write Kr 1

Salesforce made a huge announcement a few weeks before Dreamforce about “Einstein.”
This is what they are coining as their machine learning technology built directly into the
Salesforce platform. In true Salesforce fashion, they went big at their annual conference with adorable Einsteins running around with the 170,000 attendees.

Kr 2Microsoft doesn’t have a catchy title like Salesforce (or a cute logo), but they do have Azure Machine Learning. This product is currently more of a platform which can be configured and incorporated into your Microsoft products, including Dynamics CRM (or Dynamics 365 for Sales these days).

No matter which product you use, this is exciting stuff for CRM applications which have historically been thought of as overhead. Imagine telling your sales team that CRM can now write their emails for them!

But before announcing how smart your CRM system is to your organization, here are some things you will want to consider:

  1. You must feed the beast: The predictive analytics engines that power these tools require data for them to analyze. So, for new CRM implementations, this may take a while. Be realistic about what you can expect and when.
  2. Garbage in makes even worse garbage out: Artificial Intelligence does not resolve the age old issue of "bad data in, bad data out." Except now, there may be more risks exposed since "bad data in" could lead to bad decisions.
  3. Who’s right, who’s wrong: What if the output from the machine learning algorithms doesn’t match your rules? These instances can be taken case by case, but this is something that should be monitored by a business sponsor who understands.

Questions/comments/concerns? Give us a shout.

New Call-to-action

Topics: CRM Best Practices

SmtpClient for Quick Notifications

Today's blog post was written by Matt Dearing, Principal Developer at Sonoma Partners.

From time to time we'll need to write a quick script to mass update records in a CRM environment through the API. One of the easiest ways for a developer to do this is through LinqPad. Depending on the number of records that need to be queried and updated, the script may need to run for an extended period of time and most likely at night or on the weekend to avoid impact to end users. An easy way to get periodic status updates, without sitting at your desk monitoring the progress, is to leverage SMTPClient. SMTPClient will allow you to send yourself, and others, email status updates and notifications of any error while your script is running.

At Sonoma Partners, we use exchange online, so an example would be: 

A similar call could be made to send an error message and stack trace. This gives you the ability to fire off your script and go do something else while being notified of progress and of any errors that may occur and require your attention.

Have a question for us on this or other CRM-related matters? Drop us a note.

Topics: CRM Best Practices