Power Platform

Trial Environments for Power Apps Training

Want to get hands on with the Power Platform but don’t have access to an environment? Well, you can sign up for a free trial instance that provides 30 days access to various Power Platform platform components. Once set up, you can build Canvas or Model Driven Power Apps, Power Automate Flows, or a Power Apps Portal.

If you want guided training, Microsoft offers a series of events called App In A Day (AIAD). Power Platform AIAD partner led training sessions have become very popular, and for good reason – they provide a hands on introduction to the Power Platform led by excellent trainers.  And it’s all free!  If you can’t attend a session in person, you can just download the labs from this link and work at your own pace. Whether attending or working at your own pace, a Trial environment is usually required if you don’t have access to a Power Platform sandbox.

This post will provide some instructions for setting up your Office 365 and Power Platform trial environment. This is not an overly complex process, but it can be a bit confusing, depending on your level of experience.

You may also take on the role of instructor, whether for the AIAD or for internal team training. This means ensuring your attendees have an environment where they can work on their labs or other projects.

One option here is to have each attendee set up their own trial. Again, not overly complex, but it will depend on the audience and your time available. So another option is pre-deploying a series of environments for your attendees.

So we will first take a look at the steps available to set up an individual Office and Power Platform trials. From there, we can look at how to set up a batch of 20 additional trial environments for a group of attendees.

Creating Office 365 PowerApps Trial Environments

The AIAD Trainer package includes instructions on setting up the free Trials. You will first set up a Office 365 E3 Trial and then add a Power Apps Plan 2 Trial to the new subscription. When I delivered the session, I reworked the notes a bit as the process had changed since originally written and some of the attendees were a bit confused. These notes also encourage attendees to use non work email address for the trial. This ensures they have the appropriate access to work the labs and protect from inadvertent changes to work subscriptions. The updated steps are:

Sign Up For Office 365 E3 Trial account at the following link – Office 365 E3 Trial

Welcome!
Welcome, let’s get to know you

We recommend you not use your work Office 365 email address to avoid confusion.  For example, I used an Outlook.com email address.  This email will be used to send you the new account details.

Create your user ID

Choose a unique name for your account. For a real instance, this would be your company name.  For our labs, you might choose something like admin and aiadrestontest:

User ID and password

You will receive a validation code to… Prove. You’re. Not. A. Robot.

Prove it!

Once you complete this step, you should receive an email with your account info.

Login to your new Office Trial account and use the following link set up the PowerApps trial – PowerApps Plan 2 Trial Offer

Check out

Here you just need to choose Try now!

Order receipt

From here, click the link to the Users page. It may take a moment or two to load.

Active users

Select the menu next to the key icon by clicking the vertical ellipses and choose Manage Product Licenses.

Licenses and Apps

In the Account details pane to the right, select Licenses and Apps.  Check the Microsoft PowerApps Plan 2 item and choose Save.

You should now be able login to the Power Apps portal at https://make.powerapps.com/home

Create additional Users and Environments

If you are just looking for an environment for your personal testing, you are all set! You have a new Office 365 and Power Apps trial with 25 available licenses. However, if you would like to set up multiple environments, you have more work to do. You could have each attendee repeat the steps above either before the session or as part of the lab prep. This can be challenging for some and may take time out of your training session.

Another option is to build out new Environments for each of the 24 additional licenses provided with this trial. You would need to do the following:

  • In the Office Admin center, Create a new User
  • Assign the Office and Power Apps licenses
  • In the Power Apps admin portal, create a new Environment
  • Assign the new User admin rights to the Environment
  • Provision a new Database for the environment

This can be a time consuming so the AIAD Trainer pack also included some Power Shell scripts to automate the steps above. The script creates new user accounts with random names, assign licenses, creates a new Environment and CDS database for each. Once created, your attendees can simply login with the new user credentials and work on their own labs or sample projects. I would love to find the original author to give them credit since it’s a big time saver and also a great example of some of the ALM capabilities.

The script does some really cool stuff but it came with a few issues. I’ve updated this script to fix a few issues found with the original version. When I first attempted to run the scripts, I was unable to authenticate to my new account. This was because of a few updates to the Power Platform and was relatively easy to fix.

Over the last few months, I’ve made some additional updates to address some issues and to make it easier to use. For example, you can pass the user name and password rather than being prompted each time. The script will now only create Trial environments by default, instead of the original Trial and Production Pairs. Recent changes cause failures when creating more than 10 Production environments, likely because of storage limitations being enforced.

I’ve also added exception handling. In the original script, errors were being swallowed inside of one of the loops. While testing, I thought the script was just running super slowly, but it was actually in a loop continuing to throw errors. I kicked off the script on one environment and stepped away from my machine, not realizing that I was basically hitting the server over and over again with exceptions. The cool thing? Microsoft watches for these errors!

The embarrassing bit? I know this because the engineering team saw the name of the account, realized that it was for AIAD, and reached out to the AIAD management team who contacted me. The engineering team wanted to know what the heck I was doing! I felt a bit silly but they were all very nice nice about it. I explained that I was running this script and I shared it with them, and they offered some advice on updates.

It turns out that I was seeing errors because before running any automated PowerShell scripts against a new tenant, you must login at least once to the https://make.powerapps.com site. The engineering team said that they planned on addressing this in the future, but for now, you must login at least once before running the PowerShell script.

You can find the script, and a document with the steps outlined in the previous section, on Github at http://github.com/jamesnovak/setupaiad. I’ve included some notes to the ReadMe page outlining the command line parameters and their usage. Running the script should be fairly straightforward even if you are not a developer:

  • Download the entire repository, including child folder scripts
  • Open PowerShell and change directory to the download folder
  • Run the SetupAIAD.ps1 with the relevant parameters

For example, using the account described in the Trial instructions:

.\SetupAIAD.ps1 -TargetTenant 'aiadrestontest' -UserName 'admin' -Password 'password' -TenantRegion 'US' -NewUserPassword 'Password!' -UserCount 20 -MaxRetryCount 3 -SleepTime 5

In this example, the script would create 20 user accounts, user1 through user20. Each account will have an Office and Power Apps license assigned and login credentials like: user1@aiadrestontest.onmicrosoft.com and Password! as their password. Once the users login, they will see an Environment corresponding to their username, such as User1-Dev with a provisioned CDS database. The new user should have administrative access to the Environment. I’ve found this to be a nice time saver when setting up a many accounts at once!

NOTE: If you re-run the script, it will DELETE these existing environments. I have not added logic to check for the accounts and add missing. That might be a nice feature for a future version, but for now, my intention with this script an clean setup. This method also restarts the 30 day Trial period on the Environments.

I have a few additional parameters to add, such as whether to create Production environments. You can make some simple updates to the scrip to enable this now, but you may hit limits at 10 user accounts.

I plan on keeping this script and attached document up to date, but if you have any other suggestions, feel free to add an Issue or drop me an email!

Summing up

We now have instructions on how to create a new Office 365 and Power Apps Trials that provide full access to Power Platform components. We can also quickly create batches of 20 or so logins at a time on these new trials with a PowerShell script.

So if you have a group of 40 people attending an AIAD session or an internal training session, you can create two Trials and run the PowerShell script twice. Last week, I was providing some training for a group and created 80 accounts (4 trials, 4 runs of the script) in about 20-25 minutes. Much faster than manually setting all this up!

I hope this helps you kick off your own AIAD session or internal Power Platform brown bag sessions for your team! As always, any comments, questions, and suggestions are welcome.

Liquid Templates in Dynamics Portals – Part 3

It’s been a while since the last post in this series Liquid Templates in Dynamics Portals – Part 2. Since then, I’ve had the opportunity to deliver a presentation on Liquid Templates a few times and receive some great feedback. You can check out a recording of my Power Platform 24 virtual session titled Working with Liquid Templates in Power Portals hosted by xrmVirtual. This was an excellent event all around so check out the other videos available!

Let’s get back to it with a look at data collections in Liquid templates.

Working with lists of data

Displaying CDS data from a single record is a powerful feature, but working with data collections is a common requirements for Power Portal solutions. For example, we may need to display a list of Open Cases as demonstrated on the Customer Self Service Portal or display a list of open Application records on a custom Portal handling some variety of applications submissions.

We can usually handle these situations via configuration but sometimes requirements go beyond configuration. Fortunately, Liquid templates offers tools for working with data collections – Objects of type Array and Dictionary that contain a list of items, Tags that allow iterating over the list, and Filters that offer additional list refinement.

Arrays and Dictionary Types

Liquid Objects of type Array and Dictionary are similar in that we can iterate over the items list using Liquid Tags. The primary difference between the two is that with Arrays, we can access a data element using the item position while the Dictionary allows accessing item via key. Some examples of Arrays and Dictionaries:

  • page.children – collection of child pages for the current Portal Page
  • entityview.records – the current page of record results returned when requesting an entitylist
  • entity attributes – a list of attribute values accessible by attribute name

These objects provide access to the Portal configuration details and CDS data within our system.

Iteration tags

Liquid Iteration Tags allows us to loop over the items in our Array or Dictionary. I like the definition provided in the online documentation with the for Tag:

Executes a block of code repeatedly

The for Iteration tag

It seemed an odd phrase at first but it makes sense – for each item in the list, execute the block of code within the Tag definition.

When using the for Tag, you also gain access to the forloop object. While iterating over a list of elements, this object provides information about the loop itself through several forloop attributes. For example, you can check the list length, the current index, whether the current index is the first or last item, or the number of remaining items via rindex. This can be helpful depending on the complexity of the work being done in the loop.

The for Tag seems to be the most common, but we also have a few other iteration options:

  • cycle – loop on a group of strings. This is used within a for tag (and I can’t actually think of a real world scenario where it might be used).
  • tablerow – generate an HTML table row for each item in an array. This is a nice option to minimize your coding if you know that you will be using HTML tables

Collection Filters

As discussed in part 2Filters allow manipulation of our Liquid Objects when rendering the data. Liquid Templates also offer Array filters available that allow for processing Arrays and Dictionaries. Here are some of the filters that when applied to an existing array, they return a new array or object:

  • batch – divides an array into smaller arrays of a given size. An example where this is useful is rendering a list of items on more than one row in a table layout, breaking the batches into the number of columns you would like to display.
  • concat – combine two arrays into a new array. This might be handy of you need to run multiple queries, or combine arrays from two template calls
  • where / except – query for items where an attribute value matches a given criteria, or all items that do NOT match the criteria
  • select – select a given attribute value from each item in a list. This can be handy if you want to pull out a single attribute value for
  • first / last – return the first or last element of the list. This can be helpful either when filtering items or while iterating on items in a for loop, such as closing out an element on the last item.
  • group_by / order_by – reorder your list, grouping or sorting by a given element attribute. You might be able to group and order in your initial list retrieval, such as with an Entity View or FetchXml query. But this filter might be required after you apply a concat filter to reorder your combined results.
  • join – combine the elements in the array into a delimited string

This is not the complete list, so be sure to check out the links for additional filters that might suit a particular need. For example, I had not seen batch recently and it allowed me to greatly simplify an exiting Web Template.

Now that we have reviewed Arrays and Dictionaries tools in Liquid, let’s take a look at an example to illustrate these capabilities working together.

An example

The Power Portals platform offers a variety of methods for displaying CDS data – Entity Lists, Entity Forms, and Web Forms just to name a few. But if you’ve worked on any extended Portals projects, you have likely hit a limitation to what is available. This is where Liquid can help.

Here’s a fairly simple scenario as an example – let’s display a list of Contacts on a Web Page and include a clickable link for a website URL field on your Portal that is not just the URL text. A standard grid view in the Portal will render text fields tagged as URL or Email as clickable links. For example, here is a grid showing sample data with the emailaddress1 and websiteurl attributes from the Contact, the blue text indicating a clickable link:

Here the Portal uses the Entity Attribute metadata to render links which is really nice for maintainability. What if your requirements are to display alternate text for the website to the end user? Currently there is method to swap out the text value of the websiteurl attribute that is displayed as the text of the anchor tag.

This example was inspired by a discussion around an article by Nick Doleman on his readyxrm.blog titled Dynamics 365 Portals – Overcoming Entity List Roadblocks with HTML and Liquid. Nick’s scenario outlines how to apply custom formatting to table cells and adding links to related records his Event and Event Session entities. As usual, it’s an very detailed article and I highly recommend reading it to see the steps to set up a Web Page with the custom Web Template.

Some Setup…

Our example will be simpler than Nick’s – we are going to display a list of Contacts and display a link to their Website (websiteurl) using alternate text instead of the raw URL. If the Contact record has a parent Account, then we will use the name of the Account, otherwise we will provide some alternate placeholder text.

We will assume a few things so that we can focus on the Liquid Objects, Tags, and Filters. Let’s assume that we have:

  1. We have configured an Entity List named Active Contacts that displays the system view Active Contacts, with Website (websiteurl) and Email address (emailaddress1) attributes as additional columns.
  2. We created a new Web Page named Entity List Sample configured to display an Entity List named Active Contacts.
  3. Our Page uses a custom Page template of type Web Template bound to a custom Web Template, each named Entity List Sample and Liquid – Entity List Sample respectively.
  4. We have the correct Entity Permissions to see the list of Contacts in the system.

Here is a quick look at our Web Page settings:

Web Page – Entity List Sample

Our Web Template

Now that we have all that Portal configuration set up, we start building our custom Web Template. To keep things simple, we are extending the out of the box Web Template named ‘Layout 1 Column’ and overriding the block named ‘main’. (More on extends and blocks in the next post!). Our starting Web Template looks like this:

{% extends 'Layout 1 Column' %
{% block main %}
    {% include 'Page Copy' %}
{% endblock %}

Our custom Liquid for the Entity List just below the statement {% include 'Page Copy' %}. So when the Page Copy on the parent Web Page is updated, it will be rendered as well.

Using the Entity List

We configured our Web Page to include an Entity List, so now we can use Liquid to access and render the results of the View behind the Entity List. To access this data, we will include the entitylist and entityview Tags in our Web template. Our updated Web Template looks like:

{% extends 'Layout 1 Column' %}
 {% block main %}
     {% include 'Page Copy' %}
     {% entitylist id:page.adx_entitylist.id %}
       {% entityview id:params.view %}
          {% for contact in entityview.records %}
            ... our code here ...
         {% endfor %}
       {% endentityview %}
    {% endentitylist %}
 {% endblock %}

The new entitylist Tag has a single parameter named id that passes the id of the Entity List we have included in our page. This can be important for maintainability if you want to change the Entity List in the Web Page record without updating the Web Template.

The entityview Tag is a child of entitylist and includes its own id parameter. The entityview tag include several additional parameters such as sorting, searching, setting page size. These options can make our display even more dynamic but let’s save all that for a follow up post on the entityview and its capabilities! For now, we will just render our list of Contacts.

Within the entityview, we have included the for Tag and finally get to the iterating on the collection of CDS records. Within this for Tag, we will access the individual Contact records and display our data.

Display the Contact info

We added the Entity List which includes a View definition to our Web Template and when the Portal engine renders the page, it retrieves the records as defined by underlying query. Behind the scenes, Liquid is essentially executing a FetchXml query for us. The list of returned records are accessible via the entityview.records object and attribute, as we see in the for tag:

{% for contact in entityview.records %}
 ... our code here ...
{% endfor %}

When Liquid processes the for Tag, it executes the code within once for each record in the entityview.records list, placing a reference to the record in the variable named contact. It’s essentially performing an {% assign %} operation behind the scenes for us. Now that the record is available via the contact variable, we can access the Contact attributes. For example, we can retrieve the Contact Website using the following syntax:

{{contact['websiteurl']} or {{contact.websiteurl}

Now we have full control over how we want to render each of the Contact fields.

Bootstrap tables!

Within the for loop, we want to render a new table row for each contact. Instead of an HTML table element, we will use div tags and leverage the Bootstrap styles made available with the Portal. This means that for each contact, we will have a new div tag flagged as a row, and for each Contact attribute, we will add a div tag flagged as a column. A general bootstrap row and cell look like the following:

<div class="row text-wrap">
    <div class="col-md-2">
    <div>
</div>

The row class declaration identifies the row while the col-md-2 class indicates a table column. We will include a div tag for each of our Contact fields, filling in the data using the data from the contact Object. You can read more about Bootstrap tables and the grid system at this link: Bootstrap Grid System.

Most fields are simple and we can simply display the contact attribute value as is. For example,

<div class="row text-wrap">
     <div class="col-md-2">
        {{contact.fullname}}
     <div>
</div>

With the website field, we get to the fun stuff. We will need to add the Liquid code to render the anchor tag for the URL with alternate text. The logic is straightforward: if we have a website value for the contact, check for the parent Account, and if present, use the parent Account name as the anchor tag text. Otherwise, use the placeholder text.

So our Liquid code for rendering the websiteurl looks like this:

{% capture contact_website %}
     {% if contact.websiteurl != nil %}
        {% assign link_label="Check out my website!" %}
        {% if contact.parentcustomerid != nil %}
           {% assign link_label={{contact['parentcustomerid'].name}} %}
         {% endif %}
          <a href="{{contact.websiteurl}}">{{link_label}}</a>
      {% else %}
          (No Website)
      {% endif %}
  {% endcapture %}

<div class="row text-wrap">
     <div class="col-md-2">
        {{contact_website}}
     <div>
</div>

Note that we pre-populated the value of link_label in our first assign tag. This is to ensure that the variable has a value when evaluated because of some odd behavior rendering uninitialized variables. We can also added a simpler version of the anchor tag logic to the emailaddress1 column to include a clickable link.

An updated version of the table row logic for all fields will now look like this:

{% capture myWebsite %}
   {% if contact.websiteurl != nil %}
      {% assign link_label="Check out my website!" %}
      {% if contact.parentcustomerid != nil %}
         {% assign link_label={{contact['parentcustomerid'].name}} %}
      {% endif %}
      <a href="{{contact.websiteurl}}" target="_li" >{{link_label}}</a>
   {% else %}
      (No Website)
   {% endif %}
{% endcapture %}

<div class="col-md-2">
    {{contact.fullname}}
</div>
<div class="col-md-2">
    {{contact["parentcustomerid"].name}}
</div>
<div class="col-md-2">
    {{contact.jobtitle}}
</div>
<div class="col-md-2">
   {% if contact.emailaddress1 != nil %}
      <a href="mailto:{{contact['emailaddress1']}}">{{contact['emailaddress1']}}</a>
   {% endif %}
</div>
<div class="col-md-2">
   {{myWebsite}}
</div>
<div class="col-md-2">
   {{contact.adx_publicprofilecopy}}
</div>

This block of Liquid now renders our contact attributes, formatting both the website and email anchor tags. We have the option of additional logic for each. For example, if the contact.parentcustomerid is null, we can add some alternative text for that column too.

Note that we are using the capture Tag to build the website anchor element. This Tag allows saving complex HTML in a variable for reference later within the div. Using capture is not absolutely required because as we can see when rendering the email link, we can place the formatting logic inline in the attribute column div. I feel it makes things more readable when the logic is more complex.

Wrapping up the Web Template

Now that we have the logic ready, make sure our table fits nicely into the parent page, so we will wrap it up in div decorated with another Bootstrap CSS class available with with default Portal styles. We also want to make sure that our table has a header and is formatted correctly, so we include a few additional Bootstrap CSS classes in the header columns. We can also add a quick check for no records returned with the entityview. It would look odd to have a table header with no rows, so we can just display a friendly message.

We will place the container div and the table header within the entityview Tag and outside the for Tag. This will ensure that no stray HTML is left around if the entitylist is removed from the Web Page record. Our updated template now looks like this:

{% extends 'Layout 1 Column' %}
{% block main %}
{% include 'Page Copy' %}
{% entitylist id:page.adx_entitylist.id %}
   {% entityview id:params.view %}
   <div class="content">
      {% if entityview.records.first == nil %}
         <h4 class="text-center">No contacts found!</h4>
      {% else %}
         <div class="row alert">
            <div class="col-md-2 active alert-link">Full Name</div>
            <div class="col-md-2 alert-link">Company</div>
            <div class="col-md-2 alert-link">Job Title</div>
            <div class="col-md-2 alert-link">Email</div>
            <div class="col-md-2 alert-link">Website</div>
            <div class="col-md-2 alert-link">Public Profile</div>
         </div>
         {% for contact in entityview.records %}
            ... our code here ...
         {% endfor %}
      {% endif %}
         </div>
      {% endentityview %}
   {% endentitylist %}
{% endblock %}

We have the Liquid ready for each Contact in the Entity List, we’ve updated the parent container, and we’ve included a header for our table. So with everything combined, our Web Template looks like this:

{% extends 'Layout 1 Column' %}
{% block main %}
   {% include 'Page Copy' %}
   {% entitylist id:page.adx_entitylist.id %}
      {% entityview id:params.view %}
      <div class="content">
         {% if entityview.records.first == nil %}
            <h4 class="text-center">No contacts found!</h4>
         {% else %}
            <div class="row alert">
               <div class="col-md-2 active alert-link">Full Name</div>
               <div class="col-md-2 alert-link">Company</div>
               <div class="col-md-2 alert-link">Job Title</div>
               <div class="col-md-2 alert-link">Email</div>
               <div class="col-md-2 alert-link">Website</div>
               <div class="col-md-2 alert-link">Public Profile</div>
            </div>
            {% for contact in entityview.records %}
               {% capture myWebsite %}
                  {% if contact.websiteurl != nil %}
                     {% assign link_label="Check out my website!" %}
                     {% if contact.parentcustomerid != nil %}
                        {% assign link_label={{contact['parentcustomerid'].name}} %}
                     {% endif %}
                     <a href="{{contact.websiteurl}}" target="_li" >{{link_label}}</a>
                  {% else %}
                     (No Website)
                  {% endif %}
               {% endcapture %}
               <div class="col-md-2">
                  {{contact.fullname}}
               </div>
               <div class="col-md-2">
                  {{contact["parentcustomerid"].name}}
               </div>
               <div class="col-md-2">
                  {{contact.jobtitle}}
               </div>
               <div class="col-md-2">
                  {% if contact.emailaddress1 != nil %}
                    <a href="mailto:{{contact['emailaddress1']}}">{{contact['emailaddress1']}}</a>
                  {% endif %}
               </div>
               <div class="col-md-2">
                  {{myWebsite}}
               </div>
               <div class="col-md-2">
                  {{contact.adx_publicprofilecopy}}
               </div>
            {% endfor %}
         {% endif %}
      </div>
      {% endentityview %}
   {% endentitylist %}
{% endblock %}

A quick summary of what we’ve just built. In our updated Web Template:

  • inherit from the Web Template named Layout 1 Column and override only the block named main
  • included the Page Copy from the Web Page to ensure it is rendered
  • set reference the Entity List configured with the Web Page record by it’s id available on the page.adx_entitylist object atribute value
  • render a new Bootstrap based table with a row for each Contact returned with the Entity List and Entity View
  • for each Contact websiteurl attribute, if available, we display the name of the parent Account instead of the website URL value

How does our final product look?

Formatted Contact View

We’ve kept things simple to focus on the logical flow control and filters available with Liquid Templates in Power Portals, but we can easily see lots of options for formatting – sortable columns, paging, filtering, highlighting special Contacts, etc. Or we can make some logical changes such as displaying the website URL of the parent customer and not the Contact.

Reusing Liquid templates

So what happens if we need to extend and change this example a bit? For example, instead of an entitylist, we want to return and format similar data using FetchXml? I really don’t want to copy this code in several locations, so in our next post, we can take a look at some of the Liquid components that enable reuse, such as includeextends, and block Tags.

Until then, as always, all comments, questions, and suggestions are welcome!

How to detect car engine anomaly by analyzing engine noise?

Following to my article on “Starting your exciting journey of Connected Field Service and Azure IoT Hub“, I started working on a practical scenario about measuring noise in your surrounding and generating alerts in #PowerPlatform. In this article I want to summarize all resources required to implement such a scenario and my learnings. I hope this will add on the resources available in the community so you can use it as a walkthrough to implement a practical scenario.

In this article, you will see what architectural components are required to implement this simple scenario using Azure Iot and Connected Field Service. This article focuses on what is happening underhood of the platform. Luckily with Connected Field Service Application, you have everything managed behind the scene and you don’t need to worry much but this walkthrough enables to you understand what options you have in such scenarios.

Scenario

The scenario is about connecting MXChip IoT DevKit to your car or any place with noise and analyze the noise level by recording and sending the noise in form of Wave stream to an Azure IoT Hub. The Azure IoT Hub sends the data to an #Azurefunction which calculates the noise level using a simple formula and the function calls a #MicrosoftFlow to create alerts in #PowerPlatform. This can lead to number of endless scenarios.

  • The function for calculating the noise level from a wave file is extremely simple as well. There are so many scientific information which you can read here, here, here and here.
  • Calculating the noise level is not an easy task. There are many considerations involved and if you want to have the real working model, you will need to work on analyzing audio files which is beyond the scope of this demo.
  • It is possible and desirable to calculate the noise level in the device and send only the alerts to Azure IoT. This will reduce the traffic and the load on your Azure. However, for the sake of experiment I am sending all the noise data to Azure and calculate the noise level in Azure function.
  • In this demo, I am not listening to the noise all the time. I start recording on press of button A. I send the noise data to Azure on press of button B. I made this change to the scenario to demonstrate working with buttons in MX Chip and also reduce the traffic to Azure.

Architecture

The architecture of this sample is very simple. I am using an IoT Hub and Azure function to calculate and propagate the IoT events to the #PowerPlatform. On the device side, there is an Arduino application running which listens to noises and sends the noise to the Azure function.

A very comprehensive architecture of a connected field service is created in the below diagram which can simply be implemented using the #ConnectedFieldService application. However, I just wanted to implement it in a simpler way. Full details of the #ConnectedFieldService architecture can be seen in this documentation.

Components

The logical diagram of components is demonstrated below:

Ardiuno App

This component is a very program which reads the input from Audio, ButtonA and ButtonB of the device and does the following:

  1. On startup, it initializes the device and gets ready to listen to surrounding noise. It also checks the connectivity to Azure.
  2. On press of ButtonA , it records and surrounding noise and stores the stream in a buffer.
  3. On press of ButtonB, it sends the stream in the buffer to Azure.

To implement this part of the application, you will need to take following actions:

  1. Setup your device MXChip device. Please refer to this link to start.
  2. Setup your Visual Studio environment. Please refer to this link.
  3. You will need to learn how to deploy your code to the MXChip device. The simple way to upload your code your code to the device is to bring your MXChip device to Configuration mode. This means everytime you want to upload your updated code, Press A (and keep pressing) and then press reset (while still pressing A). Then release reset (While still pressing A) and then release A. Now you are ready to upload your code.
  4. If you want to debug your code in the device, you can refer to this link.

Here is my sample code:


#include "AZ3166WiFi.h"
#include "DevKitMQTTClient.h"
#include "AudioClassV2.h"
#include "stm32412g_discovery_audio.h"
#define MFCC_WRAPPER_DEFINED
#define MODEL_WRAPPER_DEFINED
//Constants and variables- Start//
enum AppState
{
APPSTATE_Init,
APPSTATE_Error,
APPSTATE_Recording,
APPSTATE_ButtonAPressed,
APPSTATE_ButtonBPressed
};
// variables will change:
static AppState appstate;
static int buttonStateA = 0;
static int buttonStateB = 0;
static bool hasWifi = false;
static bool hasIoTHub = false;
AudioClass &Audio = AudioClass::getInstance();
const int AUDIO_SIZE = 32000 * 3 + 45;
char *audioBuffer;
int totalSize;
int monoSize;
static char emptyAudio[AUDIO_CHUNK_SIZE];
RingBuffer ringBuffer(AUDIO_SIZE);
char readBuffer[AUDIO_CHUNK_SIZE];
bool startPlay = false;
void SendMessage(char *message)
{
// Send message to Azure
if (hasIoTHub && hasWifi)
{
char buff[512];
// replace the following line with your data sent to Azure IoTHub
snprintf(buff, 512, message);
if (DevKitMQTTClient_SendEvent(buff))
{
Screen.print(1, "Sent...");
}
else
{
Screen.print(1, "Failure...");
}
delay(2000);
}
else
{
// turn LED on-off after 2 seconds wait:
Screen.print("NO BUTTON DETECTED");
delay(1000);
Screen.clean();
}
}
void setup()
{
// put your setup code here, to run once:
memset(emptyAudio, 0x0, AUDIO_CHUNK_SIZE);
if (WiFi.begin() == WL_CONNECTED)
{
hasWifi = true;
Screen.print(1, "Running!!!");
if (!DevKitMQTTClient_Init(false, true))
{
hasIoTHub = false;
return;
}
hasIoTHub = true;
// initialize the pushbutton pin as an input:
pinMode(USER_BUTTON_A, INPUT);
pinMode(USER_BUTTON_B, INPUT);
appstate = APPSTATE_Init;
}
else
{
hasWifi = false;
Screen.print(1, "No Wi-Fi");
}
}
void loop()
{
// put your main code here, to run repeatedly:
Screen.clean();
// while(1)
{
// read the state of the pushbutton value:
buttonStateA = digitalRead(USER_BUTTON_A);
buttonStateB = digitalRead(USER_BUTTON_B);
if (buttonStateA == LOW && buttonStateB == LOW)
{
//SendMessage("A + B");
}
else if (buttonStateA == LOW && buttonStateB == HIGH)
{
// WAVE FORMAT
Screen.clean();
Screen.print(0, "start recordig");
record();
while (digitalRead(USER_BUTTON_A) == LOW && ringBuffer.available() > 0)
{
delay(10);
}
if (Audio.getAudioState() == AUDIO_STATE_RECORDING)
{
Audio.stop();
}
startPlay = true;
}
else if (buttonStateA == HIGH && buttonStateB == LOW)
{
// WAVE FORMAT
if (startPlay == true)
{
Screen.clean();
Screen.print(0, "start playing");
play();
while (ringBuffer.use() >= AUDIO_CHUNK_SIZE)
{
delay(10);
}
Audio.stop();
startPlay = false;
SendMessage(readBuffer);
}
else if (buttonStateA == HIGH && buttonStateB == HIGH)
{
Screen.clean();
}
}
delay(100);
}
}
void record()
{
Serial.println("start recording");
ringBuffer.clear();
Audio.format(8000, 16);
Audio.startRecord(recordCallback);
}
void play()
{
Serial.println("start playing");
Audio.format(8000, 16);
Audio.setVolume(80);
Audio.startPlay(playCallback);
}
void playCallback(void)
{
if (ringBuffer.use() < AUDIO_CHUNK_SIZE)
{
Audio.writeToPlayBuffer(emptyAudio, AUDIO_CHUNK_SIZE);
return;
}
int length = ringBuffer.get((uint8_t *)readBuffer, AUDIO_CHUNK_SIZE);
Audio.writeToPlayBuffer(readBuffer, length);
}
void recordCallback(void)
{
int length = Audio.readFromRecordBuffer(readBuffer, AUDIO_CHUNK_SIZE);
ringBuffer.put((uint8_t *)readBuffer, length);
}

Azure function

This is the simplest of all. All you have to do is to receive the stream and calculate the noise level. This can be very sophisticated but it is out of scope of this article.


using IoTHubTrigger = Microsoft.Azure.WebJobs.EventHubTriggerAttribute;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.EventHubs;
using System.Text;
using System.Net.Http;
using Microsoft.Extensions.Logging;
using System;
namespace IoTWorkbench
{
public static class IoTHubTrigger1
{
private static HttpClient client = new HttpClient();
[FunctionName("IoTHubTrigger1")]
public static void Run([IoTHubTrigger("%eventHubConnectionPath%", Connection = "eventHubConnectionString")]EventData message, ILogger log)
{
log.LogInformation($"C# IoT Hub trigger function processed a message: {Encoding.UTF8.GetString(message.Body.Array)}: " + System.Text.Encoding.Default.GetString(message.Body));
byte[] buffer = message.Body.ToArray();
short sample16Bit = BitConverter.ToInt16(buffer, 0);
double volume = Math.Abs(sample16Bit / 32768.0);
double decibels = 20 * Math.Log10(volume);
log.LogInformation(decibels.ToString());
}
}
}

Handshaking

In order the device to send messages to the Azure function, the device must know the endpoint in which it should send the data. You can take steps in the this link to register your device with Azure function. It is all about using Azure IoT Workbench.

References

https://docs.microsoft.com/en-us/dynamics365/field-service/developer/connected-field-service-architecture

Photo by Steinar Engeland on Unsplash

Power Platform 24 Live!

We recently completed the first ever 24 hour event exclusively focused on the Power Platform – Power Platform 24. The Dynamics and Power Platform community has fantastic events literally across the globe. As amazing as these in person events are, not everyone can attend as either a speaker or attendee. We wanted to remove geography as an obstacle!

Moving to a virtual format allows the team to include both speakers and attendees who may otherwise miss the opportunity to share with and learn from the community. For organizers, the added benefit is lower overhead because you don’t need to secure a venue, coordinate speaker travel, provide prizes, or feed anyone. All said, lots of pros for this virtual event format.

Our first event went off without a hitch! So maybe we had a very minor glitch or two, but the sessions were fantastic and the event ran like a well oiled machine all because of amazing speakers and a group of dedicated organizers.

I personally learned much from the experience, so I wanted to share some thoughts for those that might want to organize a similar event. This post covers a bit about the approach to organizing the event and some tools used to run the show.

Organizing the Event

The event came together fairly quickly after the idea was thrown out in a group conversation: anyone interested in hosting a virtual Power Platform event? The response was of course, “Heck yeah!” The volunteers immediately began throwing around ideas. We had a lot to discuss but most of it boiled down to these main topics:

  • How do we choose speakers?
  • How do we host each presentation?
  • How do we register attendees?

Most of the organizers have some experience running in person events, virtual events, or both so we started with some best practices in mind. We also brought experience with a variety of tools based from past events. This experience gave us a nice head start, we then simply needed to choose what works best for a virtual event spanning a full 24 hours!

Choosing speakers

The team chose sessionize.com as the platform for a call for speakers and vetting the submissions. If you have not used the platform, definitely check it out. Sessionize is offers excellent tools for both organizers and speakers to to organize event submissions and manage sessions across multiple events. Another huge plus is that for the service is free for free community events. Sessionize features alone could take up a full post!

Once you lock down your call for speakers, we used sessionize to categorize, review, and rate submissions. The organizing team reviewed each of the more than 70 submissions, ranking them based on the information provided by the speaker. This was honestly one of the hardest part of the process because we received so many excellent submissions.

We considered multiple tracks because of the number of submissions, meaning we could run two or three concurrent hour long sessions. This was our first 24 hour event, so we chose a single track of 24 one hour sessions, starting at 8:30 AM EST and running through 8:30 AM the next day.

Hosting the event

This was the big decision: What platform do we use to host the event? We can all list a dozen virtual event platforms in just a few minutes, but that doesn’t actually make things easier. We ended up choosing Teams and a Teams Live Event. This makes sense as this is a Microsoft Power Platform event, but here is an excellent article that Purvin Patel shared which helped make our decision: Produce a live event using Teams. This article outlines how to set up a Team Live Event and details around Producer roles.

The Teams Live event setup means assigning users to a producer role where they can monitor a control the live stream, Q&A channel, manage the event notes, and start/stop the event. Another important feature is the ability to record each session. This may not be a requirement for other events, but we wanted to provide recordings for both attendees and speakers. This is an excellent feature but the limitation of 4 hours per recording is something to keep in mind if you choose this platform. We needed to keep this limitation in mind when scheduling the sessions and producers.

Using a Teams Live event requires Office 365 and Teams licenses. Fortunately, the XrmVirtual crew is already delivering live events using Teams, so they offered to run the event for us. We now had a chosen platform, so we needed to decide how to run the sessions.

Delivering the Sessions

We broke the 24 hours into six blocks of 4 hours and we took volunteers as producers for each segment, which worked perfectly with our 4 hour cap on recording. A producer was logged in the speaker during the session to handle connected issues, answer or raise questions, and manage transitions between speakers.

This meant that we posted six 4 hour Team Live events that ran in sequence. Once these were established, individual invites were sent to each speaker with a link for the correct block of time. This was all handled by the XrmVirtual team and I felt it worked out great as both a speaker and a producer. It was easy for me but I know it took a lot of time to set up!

At the start of each session, a producer logged in to Teams with the correct account, share any slides that were required at the time, and kick off the session, and began recording. The speaker could then just shared their screen and delivered the session. Once the session was complete, the producer shut down the event to end recording while the next producer was already up an running with the next speaker.

Registering event attendees

Registering event attendees seems pretty important, so why is it last in the list?

Well our solution for registering users for the event was pretty simple: we didn’t register users. Fortunately, the Teams Live event platform allows users to connect without a prior registration and post questions anonymously. We had no need to track any user info, manage cancellations, etc. Attendees could jump on to catch a session and disconnect when done.

This could be an issue with different virtual delivery platforms but it did not seem to be an issue for us. I believe we averaged about 100 attendees per session which is a pretty nice number. I’ve had in person sessions with only 5 people, so 100 is pretty nice! We had some excellent questions by attendees which really adds to the delivery. And of course, attendees who missed the live session can jump online and view the recorded sessions on demand!

Testing, 1…2…

One practice that made this event run so smoothly was… practice! The week prior to the event, XrmVirtual team set up test sessions to ensure speakers could connect without issue. Each speaker jumped on to the Team event, shared their screen, and tested their audio. It sounds simple, and it was, but it saved us from potential issues on the day of the event.

We also made sure that producers understood the Teams setup operates. The XrmVirtual team provided a new account from their Office organization for each 4 hour block. Each account was granted producer rights on their respective Team Live events. Having enough accounts is another item to consider if you choose a Teams Live event as a platform.

I was not the one that set up the Teams Live event for all of the sessions, but from an end user perspective, I found this event went smoothly and the Teams platform is fairly easy to use.

Thanks once again!

I will call out the organizing team here in case you want to reach out and say thanks! For me, I wanted to say thanks once again to the organizing team for gathering and vetting the speakers, setting up the infrastructure, communicating with speakers and attendees, taking time to act as producers (at really crazy hours!), processing all of the recorded videos, and advertising the event.

Thanks for simply giving up a chunk of your free time to make this event happen.

Julie Yack
David Yack
Joel Lindstrom
Aiden Kaskela
Beth Burrell
Michael Ochs
Sarah Jelinek

Everyone on the team pitched in, but I think a few special shout outs are in order – thanks to Julie for owning the meetings and technical bits with the producer setup and being online for 16 or so hours monitoring the event real time. And thanks to David Yack for spending his weekend breaking down all of the videos and hosting them for our viewing pleasure.

And thanks to all of the speakers that gave up their time to plan and provide some excellent sessions for the community! Check out the full list of speakers at the Power Platform 24 site! You can check out the recorded events now that they have been posted here!

I am looking forward to another Power Platform 24 event… Keep an eye out for the next event announcement!

How to call #webapi from #PowerPlatform #Portals

In almost all of my portal projects I get a question from my clients: “How to call an external api from #portals”? This question has been common that I decided to write about my experience on this topic which might be helpful for the community. This post will focus on two main areas:

  1. The available options to integrate #portals with external
  2. A step by step guide on one of the least discussed options which is using Oauth Implicit Grant flow and how I created a simple demo for one of my customers

Scenario

I would like to give a business context to this scenario. Any enterprise solution requires integration and interaction of multiple systems which #portals could be one of them. Imagine a scenario where a customer is looking for a quote on a product in the company portal. In this case the #portal is required to bring quote details from CPQ (Configure Price Quote) system to the portal. In another scenario, a #portal is required to integrate with a core banking system to get the customer’s latest balances. In these scenarios and similar ones, we will require the #portal to integrate with an external api to get information.

In order to enable such integrations, the #portal must be able to make calls in a secure way as most of the internal systems require authentication before anything can happen. So what are the options available?

Solutions

Since #powerplatform #portals are tightly integrated with #powerplatform, in most cases the integration is done through the #powerplatform itself. However, the integration through these #powerplatform has three flavors.

  1. The first one is creating actions in the platform which communicated with external API and manages the requests and responses; then calling the actions through a workflow where the workflow is triggered using Entity Form or Entity List events. 
Portal Integration with Web Api
Portal Integration with Web Api using Actions

 

  • The second option is to use #MicrosoftFlow to encapsulate the Workflow and Action part in a Flow. The benefit of this solution is that you won’t need to write code (in most cases but not guaranteed) to call #webapi

    Portal Integration using Flow
    Portal Integration using Flow
  • The above two options, use #PowerPlatform to facilitate the integration and all calls are routed through the platform. However, going through the server is not always feasible. There are situations in which you would like to make client side calls from javascript using Ajax from #portals to call external API. However, the main concerns in these scenarios are authentication. And the solution provided by the platform is “Oauth Implicit Grant Flow“.If you would like to learn more about what is the ”

    Oauth Implicit Grant Flow” beyond the #PowerPlatform, you can read more here.

 

There are concerns over the Oauth Implicit Grant flow and the recommendation is to use “Oauth code grant flow”. According to the Oauth working group, “t is generally not recommended to use the implicit flow (and some servers prohibit this flow entirely). In the time since the spec was originally written, the industry best practice has changed to recommend that public clients should use the authorization code flow with the PKCE extension instead.”. Microsoft is aware of this restriction however, it is believed Oath implicit grant flow is still ok to use.

I have proposed an idea to implement the Oauth code grant flow in this IDEA. Please vote for it.

Now getting back to the topic: How to Integrate:

Portal Integration with Oauth Implicit Grant Flow
Portal Integration with Oauth Implicit Grant Flow

In this scenario, there is no server side calls are required. A complete documentation is available here. However, the documentation is not very helpful if you want to do things quickly since there is a learning cycle involved. OAuth 2.0 implicit grant flow supports endpoints that a client can call to get an ID token. Two endpoints are used for this purpose: authorize and token. I will not go to the details of these calls and I assume you already know what these are.

So here is what you will have to do:

  1. Create your web api. You can download the sample api from this Github project. This website is no different than any MVP website. So you can create your own with Web APIs. 
  2. Next is to register your application in Azure Active Directory. This is a free service which you can use to provide authentication to your web api. A step by step details of the registration process is in this link.The REDIRECT URL must be the direct link to the page you created in the step # 2. You will need to note the following after this step:

    – Client ID
    – Redirect URL

  3. Let’s say you have a Quote page in your portal and you would like to place a button on the portal page to get Quotations from your internal website. You will have to put a custom HTML in your “Content Page” (not the main page) of the portal. This custom HTML will be used to add a QUOTE button to the portal and also retrieve the Quotation by use of a custom javascrtip code.
<h2>The QUOTE BUTTON</h2>

<button type="button" onclick="callAuthorizeEndpoint()">Give me a Quote!</button>

 <script>
//Remove this line to avoid State validation
$.cookie("useStateValidation",1);
function callAuthorizeEndpoint(){
//Used for State validation
var useStateValidation = $.cookie("useStateValidation");
var appStateKey = 'p07T@lst@T3';
var sampleAppState = {id:500, name:"logic"};
//Replace with Client Id Registered on CRM
var clientId = "CLIENT ID OBTAINED FROM AZURE ACTIVE DIRECTORY";
//Replace with Redirect URL registered on CRM
var redirectUri = encodeURIComponent("https://MYPORTAL.powerappsportals.com/REDIRECT_PAGE/");
//Authorize Endpoint
var redirectLocation = `/_services/auth/authorize?client_id=${clientId}&redirect_uri=${redirectUri}`;
//Save state in a cookie if State validation is enabled
if(useStateValidation){
$.cookie(appStateKey, JSON.stringify(sampleAppState));
redirectLocation = redirectLocation + `&state=${appStateKey}`;
console.log("Added State Parameter");
}

//Redirect
window.location = redirectLocation;
}
</script>


  1. Modify the source code in the web api website to use the Client ID and Redirect URL in its startup page.
public virtual Task ValidateIdentity(OAuthValidateIdentityContext context)
        {
try
{
if (!context.Request.Headers.ContainsKey("Authorization"))
{
return Task.FromResult<object>(null);
}

// Retrieve the JWT token in Authorization Header
var jwt = context.Request.Headers["Authorization"].Replace("Bearer ", string.Empty);
var handler = new JwtSecurityTokenHandler();
var token = new JwtSecurityToken(jwt);
var claimIdentity = new ClaimsIdentity(token.Claims, DefaultAuthenticationTypes.ExternalBearer);
var param = new TokenValidationParameters
{
ValidateAudience = false, // Make this false if token was generated without clientId
ValidAudience = "CLIENT ID", //Replace with Client Id Registered on CRM. Token should have been fetched with the same clientId.
ValidateIssuer = true,
IssuerSigningKey = _signingKey,
IssuerValidator = (issuer, securityToken, parameters) =>
{
var allowed = GetAllowedPortal().Trim().ToLowerInvariant();

if (issuer.ToLowerInvariant().Equals(allowed))
{
return issuer;
}
throw new Exception("Token Issuer is not a known Portal");
}
};

SecurityToken validatedToken = null;
handler.ValidateToken(token.RawData, param, out validatedToken);
var claimPrincipal = new ClaimsPrincipal(claimIdentity);
context.Response.Context.Authentication.User = claimPrincipal;
context.Validated(claimIdentity);
}
catch(Exception exception)
{
System.Diagnostics.Debug.WriteLine(exception);
return null;
}
return Task.FromResult<object>(null);

}
  1. The next step is to use Custom HTML on the Redirect PAGE so that you can make the call to the Web API by the token obtained in this step.
function getResultInUrlFragment(hash){
    if(hash){
        var result = {};
        hash.substring("1").split('&').forEach(function(keyValuePair){
            var arr = keyValuePair.split('=');
//  Add to result, only the keys with values
            arr[1] && (result[arr[0]] = arr[1]);
        });
return result;
    }
else{
return null;
    }
}
//Validate State parameter
//Returns true for valid state and false otherwise
function validateState(stateInUrlFragment){
if(!stateInUrlFragment){
console.error("State Validation Failed. State parameter not found in URL fragment");
return false;
    }

// State parameter in URL Fragment doesn't have a corresponding cookie.
if(!$.cookie(stateInUrlFragment)){
console.error("State Validation Failed. Invalid state parameter");
return false;
    }
return true;
}

var useStateValidation = $.cookie("useStateValidation");
var appState = null;

//Fetch the parameters in Url fragment
var authorizeEndpointResult = getResultInUrlFragment(window.location.hash);

//Validate State
if(useStateValidation){
if(!validateState(authorizeEndpointResult.state)){
authorizeEndpointResult = null;
    }
else{
appState = $.cookie(authorizeEndpointResult.state);        
console.log("State: "+appState);
    }
}

//Display token
if(authorizeEndpointResult){
    var data = authorizeEndpointResult.token;
console.log("Token:" + data);
   $.ajax({
type: "GET",
url: "https://URL_TO_THE_WEB_API.azurewebsites.net/api/external/ping",
contentType: "application/json; charset=utf-8",
dataType: "json",
headers: {
Accept:"text/plain; charset=utf-8",
        "Authorization": "Bearer "+data
},
success: function (data) {
alert(JSON.stringify(data));
console.log(data);
}, //End of AJAX Success function
failure: function (data) {
alert(data.responseText);
}, //End of AJAX failure function
error: function (data) {
alert(data.responseText);
} //End of AJAX error function
});
}

I hope this post helps you a bit to make your portals connect to the outside world!

Liquid Templates in Dynamics Portals – Part 2

Liquid Templates in Dynamics Portals – Part 1 is a high level overview and a bit of history of Liquid Templates. Now we can take a closer look at Liquid Templates support in Dynamics Portals. While this capability offers flexibility to Portal developers, it can be overwhelming when getting started.

The Microsoft Documentation on Liquid is fairly solid but I think it skips many Liquid fundamentals offered by the Shopify Liquid online documentation. This is understandable since I assume they want to focus on Portals specific extensions, but the foundation is important for newcomers.

In this post, we will cover language features not outlined in the Microsoft Portals documentation:

  • Liquid Templates fundamentals – a quick overview of the main capabilities of Liquid Templates
  • Dynamics specific extensions – what extensions has Microsoft added to Liquid for Portals

Once we cover the fundamentals, we can dig deeper into:

  • Liquid Template usage in Dynamics Portals – where can we use Liquid Templates
  • Common usage patterns – examples of when and how to leverage Liquid Templates in Portals
  • Complex examples – more extensive customization using Liquid

We won’t cover all items in this post but we can get things off to a good start.

Liquid Templates fundamentals

Liquid Templates (or Liquid) is a ‘templating language’ but this phrase can be a bit confusing. When I think of templates, Email templates, Document templates, or even reports come to mind. I see language, I think Java, C#, C++, or Visual Basic with which I can build standalone applications for the desktop, mobile, or the web.

Liquid offers features traditionally associated with templates. With Email or Document Templates in Dynamics 365 CE (CRM), we enter placeholders or slugs for data fields inline with our content. The system fills in the slugs with the selected CRM record data when generating the Email or Document. We can do the same with Liquid and Power Portals – placeholders for Common Data Service (CDS) data that render when the page loads.

Liquid goes beyond templates with capabilities similar to programming languages like JavaScript. For example, you can implement conditional logic using Control Flow operators like the if tag, capture variable values using the assign tag, or we can iterate on collections of items using the loop tag.

So with Liquid, we inject CDS data elements into our Portal content (Templates) and provide some logical control over presentation (Language). These two capabilities make Liquid a powerful means of extending a Portals solution. Let’s take a look at some of the specifics of the Liquid language.

Liquid Template Building Blocks

We know the basics of JavaScript, so what are the basics of Liquid? Right on the Liquid documentation Introduction:

Liquid code can be categorized into objectstags, and filters.

Introduction

I won’t repeat the documentation since it’s well done and pretty extensive. But I think it’s important to understand these three categories:

  • Objects grant access to data and structures from CDS and Portals
  • Tags provide logic and flow to the Liquid Template
  • Filters allow us to process or transform data available in objects

We can access objects using double curly braces. The example below tells Liquid to render the value of the name object:

{{ name }}

Filters extend how we access the object, transforming or processing the data returned with the object. Adding to the example above, we can convert the value of name to upper case with the ucase filter:

{{ name | upcase }}

We access tags using a combination of curly brace and percent sign. Here, we can show some conditional logic using the if tag. This snippet will check to see if the name object is null, and if not, display the text within:

{% if name %}
  Hello {{ name }}!
{% endif %}

Liquid recognizes the elements between the curly braces as our instructions, executes them on the server, and returns the results as our rendered Portal content. This means that objects, tags, and filters are pretty important building blocks for the Liquid Template language!

Liquid Objects and Types

Objects mean data, but what kind of data can we expect? In the standard Liquid documentation, objects have a variety of Types

  • String
  • Number
  • Boolean
  • Nil
  • Array

An object in Liquid can be one of these Types. String, Number, and Boolean should be familiar. Nil is the equivalent of Null and can be important: null values mean something very different than empty strings in CDS. An Array allows working with collections or lists of values, such as a list of strings or numbers. You can iterate on Arrays or access individual items using their position or index.

Right away, we see extensions to the Liquid language by the Power Portals framework. Power Portals supports a few additional Types, as described in the online documentation:

  • Dictionary
  • DateTime

A Dictionary is similar to an Array as it holds collections of values, but items in a Dictionary can be accessed using a string key, not just the index. This will be important when retrieving CDS data. For example, if you have a list of Contacts, you can access a contact by their Id directly:

{{ contact[Id] }}

Also note that an Object can also be a comprised of several of these Types. If we think back to CDS objects, we have Entities with Attributes. For example, a Contact has Attributes like First Name, Last Name, and Created On. An object that represents a Contact and these attributes would then be a combination of String and DateTime types. We could represent these Contact Entity Attributes in Liquid with something similar to the following:

  • {{ contact.firstname }}
  • {{ contact.lastname }}
  • {{ contact.createdon }}

Here we can access a Contact Entity record and Attributes through a Liquid object named contact with String and DateTime properties in a fashion very similar to what we see in JavaScript.

This is another example of CDS extensions to the Liquid language, but standard Liquid objects work in a similar manner. For example, Shopify provides an object called page and from this object, you can access the title like so:

{{ page.title }}

With the basics of accessing data down, how can we work with this data in our Portal?

Operators and Conditional Tags!

Operators allow us to compare or evaluate objects, and the Power Portals support for operators includes a few additions to the Shopify foundation. The standard operators should be familiar to both developers and non-developers alike from math class: Equals (==), Not Equals (!=), Less Than (<), etc.

Operators are used when coupled with conditional logic tags, such as the if tag we have seen in the snippet above. We can update the snippet to use the Equals operator. For example, if we had a Boolean flag indicating whether someone is logged in, we can say hello to the user:

{% if logged_in == true %}
  Hello {{ user.fullname }}!
{% endif %}

More interesting operators worth note are Condition And (and), Condition Or (or). These two operators allow us to check for multiple conditions in one if statement. What if we want to be sure their full name has a value?

{% if logged_in == true and user.fullname != nil %}
  Hello {{ user.fullname }}!
{% endif %}

A few more conditionals that allow us to evaluate strings are contains, startswith, and endswith. We can also check for the page, only showing the Hello message on the Home page:

{% if logged_in == true and user.fullname != nil and page.title startswith 'Home' %}
  Hello {{ user.fullname }}!
{% endif %}

Another conditional tag is unlesswhich is the opposite of if. We could rework this last snippet using unless:

{% unless page.title startswith 'Profile' %} 
   {% if logged_in == true and user.fullname != nil %}
     Hello {{ user.fullname }}! 
   {% endif %}
{% endunless %}

Now, we will show the Hello for all pages except their Profile page. This also shows how we can nest these statements. Here, the if conditionals will never be evaluated until the unless conditional is met.

These are simple examples, but we can easily add some conditional logic inline that will be rendered on the server rather than building a lot of JavaScript that needs execute this logic the content on the client.

Filters for transformation

Now we know how to display data using objects and we can decide when to display data using conditional tags, we can transform or manipulate the data using Filters. We add filters to our object display syntax using a pipe “|” delimiter and Filters can be added by platforms extending liquid, but the default filters we get with the base Liquid implementation are pretty extensive.

Many filters offer simple formatting, such as converting the case of a string. Continuing our previous snippet, maybe we want to show how excited we are and make the name upper case by adding | upcase as a filter:

{% unless page.title startswith 'Profile' %} 
   {% if logged_in == true and user.fullname != nil %}
     Hello {{ user.fullname | upcase }}! 
   {% endif %}
{% endunless %}

Another common example is a format mask for dates. Lets tell the user what day and time it is when they logged into the site:

{% unless page.title startswith 'Profile' %} 
   {% if logged_in == true and user.fullname != nil %}
     Hello {{ user.fullname | upcase }}! Today is {{ "now" | date: "%Y-%m-%d %H:%M" }}.
   {% endif %}
{% endunless %}

Here we use a special object called now that will return the current date and time, and then we use the date filter that will format the date according to the mask provided. In this example the final result with the date would look something like:

Hello JIM! Today is 2019-08-06 23:24

Liquid provides many other filters for dealing with numbers and collections, such as adding numbers (| plus) or converting . These filters are more useful in more complex situations than just formatting dates or strings. You might be capturing variables while iterating on a loop, or you might be filtering items out of a collection for display on a custom page.

We will dive into some of these more complex filters when we cover collections and additional Portal specific extensions in more detail. We will use some more realistic examples to demonstrate how filters can be extremely useful!

Up next…

This was a fairly long post but we covered objects and data types, conditional tags, operators, and some powerful filters. Understanding these topics are critical because these are used throughout the more complex scenarios offered with Liquid Templates

Next we will take a look at Collections and their related control flow tags in more detail and how they related to more of the Power Portals specific extensions.

As always, comments, questions, and corrections are all welcome!

Improve efficiency of Call centers using Dynamics 365 and Azure cognitive services

Photo by Hrayr Movsisyan on Unsplash

I am Fascinated by sophistication of Azure services and how they help us to improve our solutions and extend the way we can solve customer problems. Recently I had a requirement to implement  a dynamics 365 solution to enable a call center to capture cases while their operators are offline.

One solution was to provide a self-service portal to customers to log the cases when Call center operators are offline. But in this case the customer was looking for something very quick to implement and having the ability to link incoming cases with their call center channel and derive some reporting based on it.

Approach

I started looking at Azure services and see how I can use Azure cognitive services and speech recognition to help me solve this requirement and like always I Azure did not disappoint me. In this post I would like to share my experience with you and take you to the steps that you would need to create such a solution. Of course possibilities are endless. However, this post will give you a starting point to begin your journey.

I have seen solutions where telephony systems send voice recordings of callers as an email attachment to a queue in CRM. The CRM then converts that queue item to a case and attaches the voice recording as note to the case. The challenge with this solution is the call center operators have to open attachments manually and have to write the description of the case after listening to the audio file. This means their time is spent on inefficient activities whereas they should be utilize in better ways.

Another problem with this approach is size of attachments. As time goes by, audio attachments will increase the database size impacting the maintenance of solution.

Scenario

Our scenario is based on the fact that call center agents are not working 24 hours a day.

While agents  are offline customer should still be able to contact call center record the voice messages to create cases.

We will use the following components:

  1. Azure Blob to receive recorded audio files from telephony system.
  2. Azure cognitive services to listen to recorded audio files and translate the content to a text message. The audio file will be saved in  Azure blob (which is cheaper than CRM database storage).
  3. Azure function (with Azure Blob Binding) to recognize the text from the audio file and extracts the case description.
  4. Dynamics 365 Web API to create a case in CRM using the description extracted from Azure Cognitive services.  We can also add blob metadata like filename, etc. to case properties.
Solution Architecture

The full source code is available at GitHub

However, the main code snippet to perform conversion is below:

 public static async Task <string> RecognitionWithPullAudioStreamAsync ( string key, string region, Stream myBlob , ILogger log )

        {

            // Creates an instance of a speech config with specified subscription key and service region.

            // Replace with your own subscription key and service region (e.g., "westus").

            var config = SpeechConfig.FromSubscription(key, region);

            string finalText = string.Empty;

            var stopRecognition = new TaskCompletionSource<int>();

            // Create an audio stream from a wav file.

            // Replace with your own audio file name.

            using ( var audioInput = Helper. OpenWavFile ( myBlob ) )

            {

                // Creates a speech recognizer using audio stream input.

                using ( var recognizer = new SpeechRecognizer ( config , audioInput ) )

                {

                    // Subscribes to events.

                    recognizer. Recognizing += ( s , e ) =>

                    {                       

                    };

                    recognizer. Recognized += ( s , e ) =>

                    {

                        if ( e. Result. Reason == ResultReason. RecognizedSpeech )

                        {

                            finalText += e. Result. Text + " ";

                        }

                        else if ( e. Result. Reason == ResultReason. NoMatch )

                        {

                            log.LogInformation ( $"NOMATCH: Speech could not be recognized." );

                        }

                    };

                    recognizer. Canceled += ( s , e ) =>

                    {

                        log. LogInformation ( $"CANCELED: Reason={e. Reason}" );

                        if ( e. Reason == CancellationReason. Error )

                        {

                            log. LogInformation ( $"CANCELED: ErrorCode={e. ErrorCode}" );

                            log. LogInformation ( $"CANCELED: ErrorDetails={e. ErrorDetails}" );

                            log. LogInformation ( $"CANCELED: Did you update the subscription info?" );

                        }

                        stopRecognition. TrySetResult ( 0 );

                    };

                    recognizer. SessionStarted += ( s , e ) =>

                    {

                        log. LogInformation ( "\nSession started event." );

                    };

                    recognizer. SessionStopped += ( s , e ) =>

                    {

                        log. LogInformation ( "\nSession stopped event." );

                        log. LogInformation ( "\nStop recognition." );

                        stopRecognition. TrySetResult ( 0 );

                    };

                    // Starts continuous recognition. Uses StopContinuousRecognitionAsync() to stop recognition.

                    await recognizer. StartContinuousRecognitionAsync ( ). ConfigureAwait ( false );

                    // Waits for completion.

                    // Use Task.WaitAny to keep the task rooted.

                    Task. WaitAny ( new [ ] { stopRecognition. Task } );

                    // Stops recognition.

                    await recognizer. StopContinuousRecognitionAsync ( ). ConfigureAwait ( false );

                    return finalText;

                }

            }

        }

Important considerations:

  1. [This point is optional, if you use Web API to create cases in CRM] You will need use Multi-tenant configuration, if your Azure Function Tenant and the tenant in which your CRM API is registered, are different. If your Azure function tenant and the tenant in which your CRM API is registered, you can use Single Tenant configuration.
  2. The input file from the telephony to Azure blob must be in a specific format. The required format specification is:
Property Value
File Format RIFF (WAV)
Sampling Rate 8000 Hz or 16000 Hz
Channels 1 (mono)
Sample Format PCM, 16-bit integers
File Duration 0.1 seconds < duration < 60 seconds
Silence Collar > 0.1 seconds

 

4. You can use ffmpeg tool to convert your recording to this specific format. For your testing, you can download and use the tool as below:
Download ffmpeg from this link.
Use the command: ffmpeg -i “<source>.mp3” -acodec pcm_s16le -ac 1 -ar 16000 “<output>.wav”
5. My sample in GitHub covers input in one single chunk of audio. However, if you wish to have continuous streaming, you will need to implement the         StartContinuousRecognitionAsync method.
6. The azure function should be configured to be blob trigger.

PowerApp Admin Tools

I’ve been working on XrmToolBox Tools for a bit now, both speaking and posting on the huge number of cool tools and how we can build our own. I’m still working on a few XrmToolBox related projects, but when I started diving into Power Apps, I immediately wondered if I could replicate some Tools in Canvas or Model Driven Apps. Wouldn’t it be cool if we could build out a suite of some administrator or developer tools as Power Apps?

In my post Building XrmToolBox Tools, Part 2, we build an example Tool that allows admins to view the list of Users with a Security Role assignment. It’s a relatively simple tool but it can be pretty useful if you need a quick check on a Role for migrations, troubleshooting, etc. This seemed like as good a candidate as any for a new ‘Admin Tool’ Power App.

Security Role Member Manager!

The proposed functionality for the tool is pretty simple: provide a list of Security Roles in the system, and when the user selects a Role, show the list of assignments. This was meant to be written in about an hour, so that was the extent of the capabilities. Since we have a bit more time, we can add some features. How about we display the list of Teams that have the selected Role assignment and allow Adding or Removing a User or Team to the selected Role.

These requirements are a pretty good candidate for a Canvas App. We can build this using the following components:

  • Common Data Service (CDS) connector – provides the list of Security Roles
  • Office 365 Users Connector – provides the user picture
  • Gallery – bound to the Security Roles list
  • Gallery – bound to the list of Users related to the Security Role
  • Gallery – bound to the list of Teams related to the Security Role

The main screen layout and functionality is also fairly simple. Bind the main grid to to the Security Roles list, and on the select event, bind the secondary lists to the related Users and Teams. No real code behind, just some simple data binding to galleries.

Here is the initial main screen for the Power App:

Security Role Member Manager

Some challenges, some solutions

With the XrmToolBox Tool, I wrote some code to retrieve Security Roles and and User Security Role assignments using the standard SDK Query Expression methods. The list of Security Roles was a simple Retrieve Multiple while the User Role Assignment is a many-to-many relationship.

The many to many is where I stumbled a bit. When I select my Security Role, I want to retrieve the Users and Teams which are both many-to-many relations to Security Roles. The CDS connector does not list the join table as an Entity, so I couldn’t simply add a new Data Source for User or Team and filter by the selected Role. Fortunately, support has been added for many-to-many relationships in the CDS connector. Here is an excellent blog post on the feature by Greg Lindhorst, Principal Program Manager at Microsoft: Relate records in Many-to-Many relationships

So to render the list of Users and Teams, I can bind the galleries using a simple formula. The main screen gallery from which you select a Security Role is named ‘Security Roles List’. So the User and Teams gallery Items property can be set using these simple formulas respectively:

'Security Roles List'.Selected.Users
'Security Roles List'.Selected.Teams

As you can see in Greg’s post, adding and removing Users and Teams are fairly easy too. To remove a User from the selected Security Role, we need a single line formula added to our gallery button:

Unrelate('Security Roles List'.Selected.Users, ThisItem)

That statement passes the selected Security role and the currently selected User to the Unrelate formula, and we’re done!

Next Steps

I plan on a follow up post with a bit more functionality. For example, I like the inline model for selecting a User shown in the post by Greg above, but I think selecting multiple Users and Teams works better. Another nice feature will be to distinguish between Business Units. Right now, this pulls all Security Roles for the entire organization.

This sounds like an obvious one, but I also plan on adding a confirmation dialog before removing the User or Team from a Security Role. This was a bit more complicated than I had expected, so I will write up more detail on how this will be implemented.

As I was working on this sample Power App, I came across a great post User Admin PowerApp (Part 4) by d365Cooky that proposes a similar tool, but for managing Security Roles for a selected User. I like the idea of embedding this into Dynamics 365 CE. By the way, I saw the link via Guido Preite’s dynamicsweekly.com newsletter. If you have not already signed up for this, get to it!

I’ll post notes on all these updates with more detail on how it was built, including the full solution for download in a follow up post.

Powerful stuff!

I feel like I have said that a LOT recently! In a relatively short time, I built a functioning Power App that will allow administrators to manage the Security Role Users and Teams. I put this together in a few hours, including some reading on the CDS connector capabilities and designing a few screens. All of this was done using the existing connectors and no custom code outside of the standard Canvas App formulas.

This is not as complex a tool as you may find in the XrmToolBox, and I am definitely going to continue any contribution I can to the XrmToolBox! But I think this once again demonstrates how the Power Platform allow us to provide low code/no code solutions to your users.

In the meantime, as always, any comments, suggestions, or questions are appreciated!

Liquid Templates in Dynamics Portals – Part 1

I’ve been working on Dynamics Portals projects over the last few years.  My first project involved heavy use of Web Forms and Web Form Steps and customizing forms using JavaScript. On my current project, I’ve had the opportunity to dive into Liquid Templates.

In this series of posts, I hope shed some light on one question that I had when first diving into Portals: What the heck are Liquid Templates and how can we use them in Portals?

Some Portals background

The documentation for Portals on docs.microsoft.com is pretty solid and includes a good definition Liquid Templates with regards to Portals:

Liquid is an open-source template language integrated into portals. It can be used to add dynamic content to pages, and to create a wide variety of custom templates.

Liquid Template support allows developers to customize their Portal past standard record based configuration. This includes customizing Portal page design and layout, adding custom data elements, or changing the logical flow of a Portals based application.

Liquid Template code executes on the server. This means that your template logic will run before the page is ever rendered, giving developers access to powerful capabilities not available via client side scripting.

Liquid Templates support has been available in Portals for some time. Adxstudios was acquired by Microsoft and the their Portals product has been re-branded into Dynamics Portals. While changes have been made to the platform in the few years since the acquisition, much of the functionality remains the same, including the Liquid Templates support.

When the support for Adxstudios Portals ended, we saw the release of Adoxio xRM Portals Community Edition:

Microsoft has released the Portals Source Code to the Microsoft Download Center under MIT license for developers to download

So now you can download a copy of the community version of xRM Portals and host in your own environment. This also allows developers a bit of insight into how the Portals solution works.

Open Source?

The Microsoft documentation does not provide much background on the Open Source template language being used by Portals. We could dive right in and start using Liquid Templates in our Portal, but I like to have some background as I learn something new. Specifically, I am looking for a better understanding on core features and what extensions have been added by Microsoft.

Now that we have the Community Edition code, we can take a peek under the hood. In the solution, we see a reference to the DotLiquid NuGet package. This project is a .NET port of the original Ruby based Liquid Template language created by Shopify. As their site notes, this templating language has been around since 2006 and it’s been proven by its use by many companies on a variety of products.

These additional sites provide some documentation as we start digging into Liquid Templates. While solid, the documentation provided by Microsoft is not complete, focusing on their Liquid Template extensions and Portals specific examples.

Portal building blocks!

Back to the note from the Microsoft documentation. What do they mean by “integrated into portals”? First, we can take a step back and look at the overall Dynamics Portals solution.

I like to view Portals as a content management system hosted on the Power Platform. Of course, Portals offers more than simple content management, such as exposing data from and collecting data for Power Apps like Dynamics 365 CE. When installing Portals in a new environment, we can install fully functioning solutions, such as a Customer Self-Service Portal that offers access to Knowledge base articles, trouble ticket submission and management, etc. You can see a quick breakdown of the pre-built Portal features at the Provision a Portal article. Pretty powerful stuff!

We can configure our Portal by creating or editing Entity records related to Portal components, such as Website, Web Pages, site navigation, Entity, Lists, and Entity forms.

This is all standard Portals work… How does this relate to Liquid Templates?

Well, the Microsoft Portals development team uses Liquid Templates when building the out of the box Portals. Looking at the Customer Self-Service Portal, we can see an example of Liquid Templates right in the Home page. Looking at the Web Page record, we can open the Page Template named Home, and see that it is configured to use a Web Template, also named Home.

Page Template – Home

All configuration records so far.

But once we open up the Home Web Template record and view the Source attribute, we can see Liquid Templates in action.

Web Template – Home, Source

In the source, the first bit we see are some curly braces with an ‘assign’ keyword, followed by some HTML and then a few more curly braces and keywords. So what are we looking at in the source?

Liquid Templates Markup

Liquid is a templating language which means that we provide an overall output template mixed with Liquid tags that the templating engine can process. For developers, should be somewhat familiar. It actually reminds me a bit of the Razor syntax used by ASP.NET MVC Views. If you are not a developer, the curly braces enclose our Liquid code allowing the Liquid Template engine to separate it from the rest of the static HTML.

With Liquid Templates, we can work with data as objects and object collections, conditionals such as if/then, and programmatic flow with for loops. In the Home page example, we can see the assignment of a value in the very first line:

{% assign forums_sm = sitemarkers["Forums"] %}

This one line is pulling a value out of the sitemarkers object collection and assigning it to a value to be used later in the template. Also in the template, we can see the following line:

{% include 'Search' %}

This simple statement is using another template named Search within the Home page. This seems pretty small but it’s a great example of code reuse offered up by the Liquid Template engine. Microsoft built their Search template code once and they can reuse it as needed. In this case, the Search Liquid Template is a 60+ line Liquid Template that can be reused with a single line of code!

Digging deeper

So this is a pretty high level overview with a simple example that only just scratches the surface of the Liquid Templates capabilities. For example, how can I loop on data returned from the Portal? What if I want to render something that is not HTML? What if I have some conditional logic that I need to check?

In the next few posts, we are going to dive into the core Liquid Template functionality and cover some of the Portals specific extensions available with Dynamics Portals. We are also going to look at some real world examples of using Liquid Templates in Portals.

In the meantime, questions and comments are all welcome!