Power Portals

How to call #webapi from #PowerPlatform #Portals

In almost all of my portal projects I get a question from my clients: “How to call an external api from #portals”? This question has been common that I decided to write about my experience on this topic which might be helpful for the community. This post will focus on two main areas:

  1. The available options to integrate #portals with external
  2. A step by step guide on one of the least discussed options which is using Oauth Implicit Grant flow and how I created a simple demo for one of my customers

Scenario

I would like to give a business context to this scenario. Any enterprise solution requires integration and interaction of multiple systems which #portals could be one of them. Imagine a scenario where a customer is looking for a quote on a product in the company portal. In this case the #portal is required to bring quote details from CPQ (Configure Price Quote) system to the portal. In another scenario, a #portal is required to integrate with a core banking system to get the customer’s latest balances. In these scenarios and similar ones, we will require the #portal to integrate with an external api to get information.

In order to enable such integrations, the #portal must be able to make calls in a secure way as most of the internal systems require authentication before anything can happen. So what are the options available?

Solutions

Since #powerplatform #portals are tightly integrated with #powerplatform, in most cases the integration is done through the #powerplatform itself. However, the integration through these #powerplatform has three flavors.

  1. The first one is creating actions in the platform which communicated with external API and manages the requests and responses; then calling the actions through a workflow where the workflow is triggered using Entity Form or Entity List events. 
Portal Integration with Web Api
Portal Integration with Web Api using Actions

 

  • The second option is to use #MicrosoftFlow to encapsulate the Workflow and Action part in a Flow. The benefit of this solution is that you won’t need to write code (in most cases but not guaranteed) to call #webapi

    Portal Integration using Flow
    Portal Integration using Flow
  • The above two options, use #PowerPlatform to facilitate the integration and all calls are routed through the platform. However, going through the server is not always feasible. There are situations in which you would like to make client side calls from javascript using Ajax from #portals to call external API. However, the main concerns in these scenarios are authentication. And the solution provided by the platform is “Oauth Implicit Grant Flow“.If you would like to learn more about what is the ”

    Oauth Implicit Grant Flow” beyond the #PowerPlatform, you can read more here.

 

There are concerns over the Oauth Implicit Grant flow and the recommendation is to use “Oauth code grant flow”. According to the Oauth working group, “t is generally not recommended to use the implicit flow (and some servers prohibit this flow entirely). In the time since the spec was originally written, the industry best practice has changed to recommend that public clients should use the authorization code flow with the PKCE extension instead.”. Microsoft is aware of this restriction however, it is believed Oath implicit grant flow is still ok to use.

I have proposed an idea to implement the Oauth code grant flow in this IDEA. Please vote for it.

Now getting back to the topic: How to Integrate:

Portal Integration with Oauth Implicit Grant Flow
Portal Integration with Oauth Implicit Grant Flow

In this scenario, there is no server side calls are required. A complete documentation is available here. However, the documentation is not very helpful if you want to do things quickly since there is a learning cycle involved. OAuth 2.0 implicit grant flow supports endpoints that a client can call to get an ID token. Two endpoints are used for this purpose: authorize and token. I will not go to the details of these calls and I assume you already know what these are.

So here is what you will have to do:

  1. Create your web api. You can download the sample api from this Github project. This website is no different than any MVP website. So you can create your own with Web APIs. 
  2. Next is to register your application in Azure Active Directory. This is a free service which you can use to provide authentication to your web api. A step by step details of the registration process is in this link.The REDIRECT URL must be the direct link to the page you created in the step # 2. You will need to note the following after this step:

    – Client ID
    – Redirect URL

  3. Let’s say you have a Quote page in your portal and you would like to place a button on the portal page to get Quotations from your internal website. You will have to put a custom HTML in your “Content Page” (not the main page) of the portal. This custom HTML will be used to add a QUOTE button to the portal and also retrieve the Quotation by use of a custom javascrtip code.
<h2>The QUOTE BUTTON</h2>

<button type="button" onclick="callAuthorizeEndpoint()">Give me a Quote!</button>

 <script>
//Remove this line to avoid State validation
$.cookie("useStateValidation",1);
function callAuthorizeEndpoint(){
//Used for State validation
var useStateValidation = $.cookie("useStateValidation");
var appStateKey = 'p07T@lst@T3';
var sampleAppState = {id:500, name:"logic"};
//Replace with Client Id Registered on CRM
var clientId = "CLIENT ID OBTAINED FROM AZURE ACTIVE DIRECTORY";
//Replace with Redirect URL registered on CRM
var redirectUri = encodeURIComponent("https://MYPORTAL.powerappsportals.com/REDIRECT_PAGE/");
//Authorize Endpoint
var redirectLocation = `/_services/auth/authorize?client_id=${clientId}&redirect_uri=${redirectUri}`;
//Save state in a cookie if State validation is enabled
if(useStateValidation){
$.cookie(appStateKey, JSON.stringify(sampleAppState));
redirectLocation = redirectLocation + `&state=${appStateKey}`;
console.log("Added State Parameter");
}

//Redirect
window.location = redirectLocation;
}
</script>


  1. Modify the source code in the web api website to use the Client ID and Redirect URL in its startup page.
public virtual Task ValidateIdentity(OAuthValidateIdentityContext context)
        {
try
{
if (!context.Request.Headers.ContainsKey("Authorization"))
{
return Task.FromResult<object>(null);
}

// Retrieve the JWT token in Authorization Header
var jwt = context.Request.Headers["Authorization"].Replace("Bearer ", string.Empty);
var handler = new JwtSecurityTokenHandler();
var token = new JwtSecurityToken(jwt);
var claimIdentity = new ClaimsIdentity(token.Claims, DefaultAuthenticationTypes.ExternalBearer);
var param = new TokenValidationParameters
{
ValidateAudience = false, // Make this false if token was generated without clientId
ValidAudience = "CLIENT ID", //Replace with Client Id Registered on CRM. Token should have been fetched with the same clientId.
ValidateIssuer = true,
IssuerSigningKey = _signingKey,
IssuerValidator = (issuer, securityToken, parameters) =>
{
var allowed = GetAllowedPortal().Trim().ToLowerInvariant();

if (issuer.ToLowerInvariant().Equals(allowed))
{
return issuer;
}
throw new Exception("Token Issuer is not a known Portal");
}
};

SecurityToken validatedToken = null;
handler.ValidateToken(token.RawData, param, out validatedToken);
var claimPrincipal = new ClaimsPrincipal(claimIdentity);
context.Response.Context.Authentication.User = claimPrincipal;
context.Validated(claimIdentity);
}
catch(Exception exception)
{
System.Diagnostics.Debug.WriteLine(exception);
return null;
}
return Task.FromResult<object>(null);

}
  1. The next step is to use Custom HTML on the Redirect PAGE so that you can make the call to the Web API by the token obtained in this step.
function getResultInUrlFragment(hash){
    if(hash){
        var result = {};
        hash.substring("1").split('&').forEach(function(keyValuePair){
            var arr = keyValuePair.split('=');
//  Add to result, only the keys with values
            arr[1] && (result[arr[0]] = arr[1]);
        });
return result;
    }
else{
return null;
    }
}
//Validate State parameter
//Returns true for valid state and false otherwise
function validateState(stateInUrlFragment){
if(!stateInUrlFragment){
console.error("State Validation Failed. State parameter not found in URL fragment");
return false;
    }

// State parameter in URL Fragment doesn't have a corresponding cookie.
if(!$.cookie(stateInUrlFragment)){
console.error("State Validation Failed. Invalid state parameter");
return false;
    }
return true;
}

var useStateValidation = $.cookie("useStateValidation");
var appState = null;

//Fetch the parameters in Url fragment
var authorizeEndpointResult = getResultInUrlFragment(window.location.hash);

//Validate State
if(useStateValidation){
if(!validateState(authorizeEndpointResult.state)){
authorizeEndpointResult = null;
    }
else{
appState = $.cookie(authorizeEndpointResult.state);        
console.log("State: "+appState);
    }
}

//Display token
if(authorizeEndpointResult){
    var data = authorizeEndpointResult.token;
console.log("Token:" + data);
   $.ajax({
type: "GET",
url: "https://URL_TO_THE_WEB_API.azurewebsites.net/api/external/ping",
contentType: "application/json; charset=utf-8",
dataType: "json",
headers: {
Accept:"text/plain; charset=utf-8",
        "Authorization": "Bearer "+data
},
success: function (data) {
alert(JSON.stringify(data));
console.log(data);
}, //End of AJAX Success function
failure: function (data) {
alert(data.responseText);
}, //End of AJAX failure function
error: function (data) {
alert(data.responseText);
} //End of AJAX error function
});
}

I hope this post helps you a bit to make your portals connect to the outside world!

Liquid Templates in Dynamics Portals – Part 2

Liquid Templates in Dynamics Portals – Part 1 is a high level overview and a bit of history of Liquid Templates. Now we can take a closer look at Liquid Templates support in Dynamics Portals. While this capability offers flexibility to Portal developers, it can be overwhelming when getting started.

The Microsoft Documentation on Liquid is fairly solid but I think it skips many Liquid fundamentals offered by the Shopify Liquid online documentation. This is understandable since I assume they want to focus on Portals specific extensions, but the foundation is important for newcomers.

In this post, we will cover language features not outlined in the Microsoft Portals documentation:

  • Liquid Templates fundamentals – a quick overview of the main capabilities of Liquid Templates
  • Dynamics specific extensions – what extensions has Microsoft added to Liquid for Portals

Once we cover the fundamentals, we can dig deeper into:

  • Liquid Template usage in Dynamics Portals – where can we use Liquid Templates
  • Common usage patterns – examples of when and how to leverage Liquid Templates in Portals
  • Complex examples – more extensive customization using Liquid

We won’t cover all items in this post but we can get things off to a good start.

Liquid Templates fundamentals

Liquid Templates (or Liquid) is a ‘templating language’ but this phrase can be a bit confusing. When I think of templates, Email templates, Document templates, or even reports come to mind. I see language, I think Java, C#, C++, or Visual Basic with which I can build standalone applications for the desktop, mobile, or the web.

Liquid offers features traditionally associated with templates. With Email or Document Templates in Dynamics 365 CE (CRM), we enter placeholders or slugs for data fields inline with our content. The system fills in the slugs with the selected CRM record data when generating the Email or Document. We can do the same with Liquid and Power Portals – placeholders for Common Data Service (CDS) data that render when the page loads.

Liquid goes beyond templates with capabilities similar to programming languages like JavaScript. For example, you can implement conditional logic using Control Flow operators like the if tag, capture variable values using the assign tag, or we can iterate on collections of items using the loop tag.

So with Liquid, we inject CDS data elements into our Portal content (Templates) and provide some logical control over presentation (Language). These two capabilities make Liquid a powerful means of extending a Portals solution. Let’s take a look at some of the specifics of the Liquid language.

Liquid Template Building Blocks

We know the basics of JavaScript, so what are the basics of Liquid? Right on the Liquid documentation Introduction:

Liquid code can be categorized into objectstags, and filters.

Introduction

I won’t repeat the documentation since it’s well done and pretty extensive. But I think it’s important to understand these three categories:

  • Objects grant access to data and structures from CDS and Portals
  • Tags provide logic and flow to the Liquid Template
  • Filters allow us to process or transform data available in objects

We can access objects using double curly braces. The example below tells Liquid to render the value of the name object:

{{ name }}

Filters extend how we access the object, transforming or processing the data returned with the object. Adding to the example above, we can convert the value of name to upper case with the ucase filter:

{{ name | upcase }}

We access tags using a combination of curly brace and percent sign. Here, we can show some conditional logic using the if tag. This snippet will check to see if the name object is null, and if not, display the text within:

{% if name %}
  Hello {{ name }}!
{% endif %}

Liquid recognizes the elements between the curly braces as our instructions, executes them on the server, and returns the results as our rendered Portal content. This means that objects, tags, and filters are pretty important building blocks for the Liquid Template language!

Liquid Objects and Types

Objects mean data, but what kind of data can we expect? In the standard Liquid documentation, objects have a variety of Types

  • String
  • Number
  • Boolean
  • Nil
  • Array

An object in Liquid can be one of these Types. String, Number, and Boolean should be familiar. Nil is the equivalent of Null and can be important: null values mean something very different than empty strings in CDS. An Array allows working with collections or lists of values, such as a list of strings or numbers. You can iterate on Arrays or access individual items using their position or index.

Right away, we see extensions to the Liquid language by the Power Portals framework. Power Portals supports a few additional Types, as described in the online documentation:

  • Dictionary
  • DateTime

A Dictionary is similar to an Array as it holds collections of values, but items in a Dictionary can be accessed using a string key, not just the index. This will be important when retrieving CDS data. For example, if you have a list of Contacts, you can access a contact by their Id directly:

{{ contact[Id] }}

Also note that an Object can also be a comprised of several of these Types. If we think back to CDS objects, we have Entities with Attributes. For example, a Contact has Attributes like First Name, Last Name, and Created On. An object that represents a Contact and these attributes would then be a combination of String and DateTime types. We could represent these Contact Entity Attributes in Liquid with something similar to the following:

  • {{ contact.firstname }}
  • {{ contact.lastname }}
  • {{ contact.createdon }}

Here we can access a Contact Entity record and Attributes through a Liquid object named contact with String and DateTime properties in a fashion very similar to what we see in JavaScript.

This is another example of CDS extensions to the Liquid language, but standard Liquid objects work in a similar manner. For example, Shopify provides an object called page and from this object, you can access the title like so:

{{ page.title }}

With the basics of accessing data down, how can we work with this data in our Portal?

Operators and Conditional Tags!

Operators allow us to compare or evaluate objects, and the Power Portals support for operators includes a few additions to the Shopify foundation. The standard operators should be familiar to both developers and non-developers alike from math class: Equals (==), Not Equals (!=), Less Than (<), etc.

Operators are used when coupled with conditional logic tags, such as the if tag we have seen in the snippet above. We can update the snippet to use the Equals operator. For example, if we had a Boolean flag indicating whether someone is logged in, we can say hello to the user:

{% if logged_in == true %}
  Hello {{ user.fullname }}!
{% endif %}

More interesting operators worth note are Condition And (and), Condition Or (or). These two operators allow us to check for multiple conditions in one if statement. What if we want to be sure their full name has a value?

{% if logged_in == true and user.fullname != nil %}
  Hello {{ user.fullname }}!
{% endif %}

A few more conditionals that allow us to evaluate strings are contains, startswith, and endswith. We can also check for the page, only showing the Hello message on the Home page:

{% if logged_in == true and user.fullname != nil and page.title startswith 'Home' %}
  Hello {{ user.fullname }}!
{% endif %}

Another conditional tag is unlesswhich is the opposite of if. We could rework this last snippet using unless:

{% unless page.title startswith 'Profile' %} 
   {% if logged_in == true and user.fullname != nil %}
     Hello {{ user.fullname }}! 
   {% endif %}
{% endunless %}

Now, we will show the Hello for all pages except their Profile page. This also shows how we can nest these statements. Here, the if conditionals will never be evaluated until the unless conditional is met.

These are simple examples, but we can easily add some conditional logic inline that will be rendered on the server rather than building a lot of JavaScript that needs execute this logic the content on the client.

Filters for transformation

Now we know how to display data using objects and we can decide when to display data using conditional tags, we can transform or manipulate the data using Filters. We add filters to our object display syntax using a pipe “|” delimiter and Filters can be added by platforms extending liquid, but the default filters we get with the base Liquid implementation are pretty extensive.

Many filters offer simple formatting, such as converting the case of a string. Continuing our previous snippet, maybe we want to show how excited we are and make the name upper case by adding | upcase as a filter:

{% unless page.title startswith 'Profile' %} 
   {% if logged_in == true and user.fullname != nil %}
     Hello {{ user.fullname | upcase }}! 
   {% endif %}
{% endunless %}

Another common example is a format mask for dates. Lets tell the user what day and time it is when they logged into the site:

{% unless page.title startswith 'Profile' %} 
   {% if logged_in == true and user.fullname != nil %}
     Hello {{ user.fullname | upcase }}! Today is {{ "now" | date: "%Y-%m-%d %H:%M" }}.
   {% endif %}
{% endunless %}

Here we use a special object called now that will return the current date and time, and then we use the date filter that will format the date according to the mask provided. In this example the final result with the date would look something like:

Hello JIM! Today is 2019-08-06 23:24

Liquid provides many other filters for dealing with numbers and collections, such as adding numbers (| plus) or converting . These filters are more useful in more complex situations than just formatting dates or strings. You might be capturing variables while iterating on a loop, or you might be filtering items out of a collection for display on a custom page.

We will dive into some of these more complex filters when we cover collections and additional Portal specific extensions in more detail. We will use some more realistic examples to demonstrate how filters can be extremely useful!

Up next…

This was a fairly long post but we covered objects and data types, conditional tags, operators, and some powerful filters. Understanding these topics are critical because these are used throughout the more complex scenarios offered with Liquid Templates

Next we will take a look at Collections and their related control flow tags in more detail and how they related to more of the Power Portals specific extensions.

As always, comments, questions, and corrections are all welcome!

Liquid Templates in Dynamics Portals – Part 1

I’ve been working on Dynamics Portals projects over the last few years.  My first project involved heavy use of Web Forms and Web Form Steps and customizing forms using JavaScript. On my current project, I’ve had the opportunity to dive into Liquid Templates.

In this series of posts, I hope shed some light on one question that I had when first diving into Portals: What the heck are Liquid Templates and how can we use them in Portals?

Some Portals background

The documentation for Portals on docs.microsoft.com is pretty solid and includes a good definition Liquid Templates with regards to Portals:

Liquid is an open-source template language integrated into portals. It can be used to add dynamic content to pages, and to create a wide variety of custom templates.

Liquid Template support allows developers to customize their Portal past standard record based configuration. This includes customizing Portal page design and layout, adding custom data elements, or changing the logical flow of a Portals based application.

Liquid Template code executes on the server. This means that your template logic will run before the page is ever rendered, giving developers access to powerful capabilities not available via client side scripting.

Liquid Templates support has been available in Portals for some time. Adxstudios was acquired by Microsoft and the their Portals product has been re-branded into Dynamics Portals. While changes have been made to the platform in the few years since the acquisition, much of the functionality remains the same, including the Liquid Templates support.

When the support for Adxstudios Portals ended, we saw the release of Adoxio xRM Portals Community Edition:

Microsoft has released the Portals Source Code to the Microsoft Download Center under MIT license for developers to download

So now you can download a copy of the community version of xRM Portals and host in your own environment. This also allows developers a bit of insight into how the Portals solution works.

Open Source?

The Microsoft documentation does not provide much background on the Open Source template language being used by Portals. We could dive right in and start using Liquid Templates in our Portal, but I like to have some background as I learn something new. Specifically, I am looking for a better understanding on core features and what extensions have been added by Microsoft.

Now that we have the Community Edition code, we can take a peek under the hood. In the solution, we see a reference to the DotLiquid NuGet package. This project is a .NET port of the original Ruby based Liquid Template language created by Shopify. As their site notes, this templating language has been around since 2006 and it’s been proven by its use by many companies on a variety of products.

These additional sites provide some documentation as we start digging into Liquid Templates. While solid, the documentation provided by Microsoft is not complete, focusing on their Liquid Template extensions and Portals specific examples.

Portal building blocks!

Back to the note from the Microsoft documentation. What do they mean by “integrated into portals”? First, we can take a step back and look at the overall Dynamics Portals solution.

I like to view Portals as a content management system hosted on the Power Platform. Of course, Portals offers more than simple content management, such as exposing data from and collecting data for Power Apps like Dynamics 365 CE. When installing Portals in a new environment, we can install fully functioning solutions, such as a Customer Self-Service Portal that offers access to Knowledge base articles, trouble ticket submission and management, etc. You can see a quick breakdown of the pre-built Portal features at the Provision a Portal article. Pretty powerful stuff!

We can configure our Portal by creating or editing Entity records related to Portal components, such as Website, Web Pages, site navigation, Entity, Lists, and Entity forms.

This is all standard Portals work… How does this relate to Liquid Templates?

Well, the Microsoft Portals development team uses Liquid Templates when building the out of the box Portals. Looking at the Customer Self-Service Portal, we can see an example of Liquid Templates right in the Home page. Looking at the Web Page record, we can open the Page Template named Home, and see that it is configured to use a Web Template, also named Home.

Page Template – Home

All configuration records so far.

But once we open up the Home Web Template record and view the Source attribute, we can see Liquid Templates in action.

Web Template – Home, Source

In the source, the first bit we see are some curly braces with an ‘assign’ keyword, followed by some HTML and then a few more curly braces and keywords. So what are we looking at in the source?

Liquid Templates Markup

Liquid is a templating language which means that we provide an overall output template mixed with Liquid tags that the templating engine can process. For developers, should be somewhat familiar. It actually reminds me a bit of the Razor syntax used by ASP.NET MVC Views. If you are not a developer, the curly braces enclose our Liquid code allowing the Liquid Template engine to separate it from the rest of the static HTML.

With Liquid Templates, we can work with data as objects and object collections, conditionals such as if/then, and programmatic flow with for loops. In the Home page example, we can see the assignment of a value in the very first line:

{% assign forums_sm = sitemarkers["Forums"] %}

This one line is pulling a value out of the sitemarkers object collection and assigning it to a value to be used later in the template. Also in the template, we can see the following line:

{% include 'Search' %}

This simple statement is using another template named Search within the Home page. This seems pretty small but it’s a great example of code reuse offered up by the Liquid Template engine. Microsoft built their Search template code once and they can reuse it as needed. In this case, the Search Liquid Template is a 60+ line Liquid Template that can be reused with a single line of code!

Digging deeper

So this is a pretty high level overview with a simple example that only just scratches the surface of the Liquid Templates capabilities. For example, how can I loop on data returned from the Portal? What if I want to render something that is not HTML? What if I have some conditional logic that I need to check?

In the next few posts, we are going to dive into the core Liquid Template functionality and cover some of the Portals specific extensions available with Dynamics Portals. We are also going to look at some real world examples of using Liquid Templates in Portals.

In the meantime, questions and comments are all welcome!