SaaS Developer Guide - Part 3 - Achieving Economy of Scale

How did Cloud pick up an age-old term such as “economy of scale”? Well, it fits perfectly to the scenario Cloud service providers are trying to achieve. Achieving economy of scale is all about doing super optimized deployment where addition of users/organizations causes minimal or zero incremental cost to the SaaS provider. This blog of the SaaS Developer Guide blog series, introduces a list technologies that are available to realize economy of scale for SaaS.

Why is "economy of scale" so important?

Lets start small. As a Cloud Learner SaaS provider, imagine the initial deployment is done. What if it can only support one lecturer/institute? That is not going to be useful. What if a single deployment of 1000 nodes are put in-place to support everybody? That is not very useful either. The deployment should elastically scale during peak hours of use and we should be able to support maximum number of users with minimal amount of hardware resources. That is how we achieve economy of scale.

There is a list of technologies that are available for SaaS providers, to do a deployment to serve thousands of users based on different levels resource sharing. This is a quotation from IBM developerWorks that explains resource sharing and isolation.

"The more resources that are shared, the higher the density. Higher density lowers the provider's costs. At the same time, increased sharing reduces the level of isolation between tenants — the individual systems or services that are being delivered. Isolation is the degree to which one tenant can affect the activity and data of other tenants."[1]

This blog will concentrate on two of the most popular resource resource sharing mechanisms.

Containers

What is the difference between VM and containers? VMs work on virtualized hardware while containers share the operating system? During the recent years, virtualization technologies have evolved massively. Containerizing is such a technology that took a leap during the past couple of years. Containers work sharing the operating system as opposed to VMs that emulate hardware, making it possible to run five-to-six times more servers with containers as opposed to VMs. In container based SaaS deployment, each subscriber gets a single instance of the SaaS application, running in a container. This ensures SLA to the end user as we guarantee the CPU for each running instance. Docker is one such technology. Docker is the leading light weight container technology today due to the following reasons.

• Easy to use compared to other container technologies
• Docker’s libcontainer is the de-facto standard for Linux containers. It is open source.
• Supported by Google and Redhat
• Docker can be used to pack and ship software

Modern PaaSes such as Kubernetes, GCloud or Cloud Foundry can run docker images. The world is moving to the containers as we are passing the technology-hype curve for docker. Docker can be very effective as containers can be sprung up on first request for the tenant. Containers can killed after an idle time and the same infrastructure resources can be used over an over again to serve 1000s of tenants.

Multi-tenancy

The highest form of density would be using multi-tenancy. When it comes to multi-tenancy, the same application instance will be used by several tenants. In this mode, isolation is minimal and activity of one tenant will affect others. This is a good option for SaaS, if the user activity on the application is predictable and does not need to heavy processing. There are tenant models.

• Each tenant has it's own database
• Each tenant has a scheme in the database
• Each tenant is distinguished by an Id in the schema   

Some SaaS providers use multi-tenant mode to cater to the demo period with cut down set of features. With careful monitoring and distribution of tenants it can provide a large economy of scale, especially if tenants are sharing the same schema and the tenant ID distinctly identify which data belongs to which tenants.

Cloud Learner

Applications can support both multi-tenancy and container based deployment. The Cloud Learner application is implemented in the hybrid approach. This allows the application to be deployed in different modes depending on other aspects such as subsription layers. For example the Platinum subscription layer can have an instance of Cloud Learner application for itself deployed on containers while the demo mode will share the same running instance. 

 
In a future post we'll look at how the dockerized Cloud Learner application is deployed in GCloud.

[1] http://www.ibm.com/developerworks/library/j-multitenant-java/
[2] https://msdn.microsoft.com/en-us/library/aa479086.aspx 

WSO2 Identity Server - Supporting Binary Claims such as Windows SID and objectGUID

Configuring a binary attribute (such as Windows SID or objectGUID) as a claim in WSO2 Identity Server and sending it over XML token such as SAML needs an additional configuration.

Problem Identification

 [2016-03-17 16:48:48,203] @nextlabs.com [1] [IS]ERROR {org.wso2.carbon.identity.sso.saml.processors.SPInitSSOAuthnRequestProcessor} - Error processing the authentication request  
 org.wso2.carbon.identity.base.IdentityException: Error Serializing the SAML Response  
     at org.wso2.carbon.identity.base.IdentityException.error(IdentityException.java:162)  
     at org.wso2.carbon.identity.sso.saml.util.SAMLSSOUtil.marshall(SAMLSSOUtil.java:352)  
     at org.wso2.carbon.identity.sso.saml.processors.SPInitSSOAuthnRequestProcessor.process(SPInitSSOAuthnRequestProcessor.java:161)  
     at org.wso2.carbon.identity.sso.saml.SAMLSSOService.authenticate(SAMLSSOService.java:164)  
     at org.wso2.carbon.identity.sso.saml.servlet.SAMLSSOProviderServlet.handleAuthenticationReponseFromFramework(SAMLSSOProviderServlet.java:691)  
 .....  
 .....  
 .....  
 Caused by: org.w3c.dom.ls.LSException: The character '☼' is an invalid XML character  
     at org.apache.xml.serialize.DOMSerializerImpl.write(Unknown Source)  
     at org.wso2.carbon.identity.sso.saml.util.SAMLSSOUtil.marshall(SAMLSSOUtil.java:348)  
     ... 55 more  
 Caused by: java.io.IOException: The character '☼' is an invalid XML character  
     at org.apache.xml.serialize.BaseMarkupSerializer.fatalError(Unknown Source)  
     at org.apache.xml.serialize.BaseMarkupSerializer.surrogates(Unknown Source)  

Solution

Add the following parameter in <carbon_home>/repository/conf/usermgt.xml for each binary attribute that you wish to convert to XML.

 <Property name="java.naming.ldap.attributes.binary">objectGUID</Property>  

SaaS Developer Guide - Part 2 - Scalability and Resiliency

This blog post is divided into two sections,

• Background - Touches on definitions and calculations
• Current Technology and Execution - Available technology, their features and Cloud Learner deployment

Background 

Distribution Systems has been around in software world for quite a while - since 1970s. It has been evolving, getting new terms, and becoming a key ingredient in Cloud based systems. Formally distributed systems are defined as a system consisting of components communicating over a network. Today all highly SaaS systems are designed as a horizontally scaling "set of services". A service is a evolution of a distributed component. If a well defined interface is given to a component in a distributed system it becomes an interface.

Why Services?

• Services can be developed, deployed and upgraded independantly
• Services can be scaled and made highly available independently. More replication for important services. More nodes for services with higher load
• Allows to use different technologies
• Strong boundaries for modularity

In the Cloud Learner application, there is going to be several services,

• Class CRUD service
• UserMgt service
• Subscription service
• Course CRUD service
• Front end component

Load balanced Nodes

When the load goes high on the "Class CRUD service" we can have two instances of the "Class CRUD service" to serve the requests. The load balancer will distribute the requests between these nodes.

"Elasticity" or "auto-scaling", is the primary scalable ingredient of  a SaaS. It means the system scales up as the load goes high and it scales down as the load becomes low. This allows a SaaS to exploit the full potential of the underlying IaaS/PaaS, which has the ability to provision infrastructure on-demand. Elasticity leads to optimized resource consumption that results in economy of scale. We'll be discussing about "economy of scale" in length in a future blog.

Resiliency and scalability go hand-in-hand, in a large distributed system. For example one of the primary methods of providing resiliency is detecting faulty node and removing it from cluster while and creating another instance for it. If one instance of the component "A" goes down, the performance of that module degrades by (n-1) where n is number of nodes in the module. But the system will continue to function. This is a key feature of resilient architecture.

              Percentage performance hit on a cluster = (n-&)/n * 100%

       where,

              n is the total number of nodes

              & is the faulty number of nodes

When n is large, the impact of loosing a single node shows less performance hit. But what happens when n is small? For example if the number of nodes are 2 then, 1 node malfunctioning is a critical factor.

Everybody starts starts small. So as an initial SaaS provider with a cluster of 2 nodes that are fully utilized by the load, adding an additional node in active mode would give n+1 resilient factor. When it comes to Cloud Learner application only one node is enough to manage subscriptions, but for resilience another node is added.

Current Technology and Execution

Load is predicted and tested, calculations are done, it is important to glance at the current cloud landscape to understand the "execution" aspects of achieving scalability and resilience. The line between PaaS and IaaS are blurring as IaaS providers keep on adding PaaS level features. Following public cloud features can be leveraged for scalability and resiliency.

• Load balancing
• Health check
• Rule based auto scaling
• Rule based routing
• Availability zones 

For example look at the functionalities provided by GCloud for auto-scaling.

Now that is the difference of "going cloud all the way". The sample application can be deployed with scalability and resilience in on a PaaS as follows. There are two FE components. All nodes have n+1 resilient factor. Here the ClassCRUD services is deployed separately because it's load factor is higher than other services. ClassCRUD service the most important service, in the Cloud Learner application as it serves 90% of the user actions. Therefore it has 3 nodes.

SaaS Developer Guide - Sample Application - Cloud Learner

Using a sample application is going to spice up the SaaS Developer Guide blogposts. It is going to help in explaining deep technical concepts in an easy understandable way. A picture is worth thousand words. Similarly a sample app is worth ten thousand words. So this is the sample application that will be used by all SaaS Developer Guide blog posts.

Cloud Learner Business Architecture

Cloud Learner is a cloud based distant educational system. It allows lecturers to offer online courses. Students can sign up to classes. Each class has a forum and notification board, where online lecture times and other notifications are hosted. Cloud Learner is not going facilitate conducting online classes as it is a complete different application domain. The creator of Cloud Learner has made a deal with another SaaS provider that specializes in communication and unified communication solutions, namely UniCom. In short, UniCom offers online conferencing capabilities, that Cloud Learner lecturers are going to use to conduct online classes.

Cloud Learner User-story

Users can signup to Cloud Learner.

• Users can sign up as a lecturer - Lecturers can create courses and advertise them in Cloud Learner. Lectures start the class and each class would get,
• Notice board
• Forum
• Evaluation tests
• Online conferencing facility for conducting classes
• Users can sign up as students - Students can view the offered courses and subscribe to them after paying subscription fees. Students has access to forum and notification board once subscribed.

Cloud Learner Subscription Levels

Students can register and browse Cloud Learner free of charge. However lecturers have to pay subscriptions. There are three subscription categories available.

Demo

• One repetition of one course
• 25 student subscriptions

Silver

• One course
• 25 student subscriptions

Gold

• One course
• Unlimited number of student subscriptions

Platinum

• Four courses
• Unlimited number of student subscriptions

Each class can have  up-to maximum of 8 online conferences, and the the charges are included in the subscription.


High-level architecture for Cloud Learner

Cloud learner has several modules.

Class CRUD Service - Create, read, update and delete all class related functionalities such as forum, test and notices

Course CRUD Service - Create, read, update and delete all courses

Subscription Management Service - Manages subscription of students to services

User registration and Management Service - Handles all user related matters

Conferences are created for each online lecture in external communication provider upon request of the lecturer.

SaaS Developer Guide - Part 1 - The Basics from SaaS Providers POV

This is the first blog post of a series, that is going to explain concepts and guide lines on how to build a successful SaaS application to deliver your services over the cloud. It aims to present the concepts and then dig deep into technical nitty-gritty behind SaaS concepts such as auto-scaling, resilience, user-experience, economy of scale and etc... All blog posts can be found under the tag SaaS Developer Guide

Cloud providers have spoiled the current SaaS consumer market with cool features. There are lot techniques and technologies, that are used to deliver SaaS as it has become a very mature form of software delivery.  Each day we hear SaaS providers delivering awesome user experience, and the question is how would you meet their sky-rocketing expectations?  Firstly, as a SaaS provider, it is important to have a deep understanding of SaaS characteristics, because it leads to best architectural principles, design patterns, implementation and deployment practices. 

SaaS characteristics,

• The software is centrally hosted by the vendor

• Provides self-service mode of operation

• License is subscription based, and supports "pay-as-you-go" model

The software is centrally hosted by the vendor

SaaS software needs to be designed such that it can serve thousands of concurrent users. Use access it via browser over the Internet. It must have properties such as,

• Highly Scalable - Modern software depends on horizontal scalability, i.e. software is designed as a distributed system or as a set of distributed services. In a distributed system the services coordinate over a network by passing messages to achieve a common task. This brings the advantage of ability to maintain each service independently from other parts of the system, allowing continuous delivery and controlled upgrades. Each service is cohesive and stateless,  allowing the system to be rapidly scaled up and down as desired.

• Resilient - SaaS needs should have self-healing fault tolerant features as it is deployed on an IaaS layer most of the time. If an instance goes down, there must be measures to bring up another instance automatically. If an instance is malfunctioning, that needs to be identified quickly and removed from the distributed system.

• Optimized resource consumption - In order to optimize the usage of IaaS by the SaaS, it needs to be dynamically scalable. As the demand goes high the system should scale up and when the demand goes down it should scale down. Lazy loading of containers and tenants is another approach of minimizing the IaaS layer consumption. In short terms this is called the "economy of scale". This means how many users the application will serve using X number of resources. This is a broad area that starts with  the architectural level decisions such as "container-zing" vs "multi-tenancy" and goes all the way down to coding, caching data, CDN networks to browser caching.

• Security - Security is another major area that a SaaS provider needs to be concerned about. This includes security of the data as well as the perimeter security. Data needs to be backed-up regularly and only authorized parties must have access to the data. APIs are to be protected and proper perimeter security must be in-place

Provides self-service mode of operation

SaaS is a user-facing software and needs to be maximally UX engineered. User stories must be written and tested. User behavior can be recorded (by recording user clicks) and analyzed to identify gaps. UX all the way is very important. Some of the characteristics of self-service mode of operation are,

• Self-registration - Users register organizations and users

• Password resetting, user/profile management, authorizations

• Subscription management 

• Monitoring their usage and rates

• Ability to monitor business activity

• Provide certain degree of customization

• Configurable dashboards 

License is subscription based, and supports "pay-as-you-go" model

Monitoring usage and metering is the key to support subscription based model of payments. Users pay per use and different bands of usages are available. When it comes to "pay per use", usage needs to be monitored at granular level. Each metering attributed needs to be recorded so that it is counted in the billing process. Recording the actions of thousands of users is a challenging task because of the volumes. Further the recorded data needs to be summarized periodically to execute throttling on usage and to generate the monthly bill/overages. There are well established architectural and design patterns that allow SaaS applications to handle large data volumes and generate billing information.

Enterprise Identity Patterns with WSO2 IS: Complying to SOX with SaaS Applications

The Sarbanes-Oxley(SOX) act of 2002 is a legislation passed by the U.S. Congress to protect shareholders and the general public from accounting errors and fraudulent practices in the enterprise, as well as improve the accuracy of corporate disclosures[1]. The act require the identity and access management (IAM) program of an enterprise to meet the following to become compliant with the standards.

• Securely manage and assign and control user access rights
• Disseminate tasks and associated privileges among multiple users
• Adjust user access rights when responsibilities change
• Revoke user access upon termination
• Provide uniform access policy
• Manage access based on business roles
• Manage allocation of user credentials

In an enterprise, managing user credentials and access rights in a decentralized way to meet the above requirement is highly inefficient, costly, error prone and eventually will not meet the requirements. This is because today's enterprise has a large number of different systems commissioned over a time. The complexity becomes worse as some systems are cloud based SaaS applications while others are on-premise.

How do you establish a centralized IAM program to achieve the above requirements?

• Centrally manage credentials, access controls and profiles
• Connect all possible on-premise systems the centrally managed IAM system
• Provision users, profiles and access control to cloud and external domains
• Use single sign on protocols to validate credentials at central point and provide a token to authenticate to on-premise and cloud system

WSO2 Identity server is a central backbone that connects and manages multiple identities across applications, APIs, the cloud, mobile, and Internet of Things devices, regardless of the standards on which they are based built on. This is how one can use WSO2 Identity server to achieve the above requirements.

WSO2 IS connects to the enterprise LDAP, and the unified system meets above requirements. The principals of the design are,

• Configure WSO2 IS to maintain users and roles in the LDAP
• Design LDAP groups based on on business roles. Use required level of granularity provide segregation of duties. As users change business roles assign them roles at WSO2 IS admin console - the centralized IAM system
• Assign correct access controls to different business roles
• Configure SSO with on-premise and cloud services (SaaS and PaaS) based on SAML2, OpenID Connect and open standards
• For propagating access controls,
• Use SCIM or web APIs to provision/deprovision users into external system
• On-premise systems are in the same identity control domain. We can have a choice of provisioning, or providing roles along with the authentication toke, or LDAP synching.  Use one of the methods compatible with the system.
• Write connectors for automatic provisioning and de-provisioning of users

[1] https://en.wikipedia.org/wiki/Sarbanes%E2%80%93Oxley_Act
[2] http://wso2.com/products/identity-server/

Configure SAML SSO to PeopleHR using WSO2 Identity Server

I wanted PeopleHR to do SAML SSO with WSO2 Identity provider, but couldn't find any documentation that gave generic steps on how to configure SAML SSO in PeopleHR. I hope this blog will help you through. I will try to keep it as generic as possible and use WSO2 Identity Server as an example only.

1 - Create a metafile in the following format

<?xml version="1.0"?>
<EntityDescriptor xmlns="urn:oasis:names:tc:SAML:2.0:metadata" entityID="dimuthul.local">
  <IDPSSODescriptor xmlns:ds="http://www.w3.org/2000/09/xmldsig#" protocolSupportEnumeration="urn:oasis:names:tc:SAML:2.0:protocol">
    <KeyDescriptor use="signing">
      <ds:KeyInfo xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
        <ds:X509Data>
          <ds:X509Certificate>PEM-FORMAT-OF-THE-PUBLIC-KEY</ds:X509Certificate>
        </ds:X509Data>
      </ds:KeyInfo>
    </KeyDescriptor>
    <NameIDFormat>urn:oasis:names:tc:SAML:1.1:nameid-format:emailAddress</NameIDFormat>
    <SingleSignOnService Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-Redirect" Location="https://localhost:9443/samlsso"/>
    <SingleSignOnService Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST" Location="https://localhost:9443/samlsso"/>
    <SingleSignOnService Binding="urn:oasis:names:tc:SAML:2.0:bindings:SOAP" Location="https://localhost:9443/samlsso"/>
  </IDPSSODescriptor>
  <ContactPerson contactType="technical">
    <SurName>Support</SurName>
    <EmailAddress>support@dimuthu.com</EmailAddress>
  </ContactPerson>
</EntityDescriptor>

2 - Upload it to PeopleHR by Going to Settings-->Company

3 - Now at the SAML Provider side here are the parameters. These parameters are found by studying the AuthnRequest.

• EntityId/Issuer (relying party or service provider Id) is "test-app-peopleweb"
• ACS (Assertion Consumer) URL is https://{replace-with-your-domain-name}.peoplehr.net/Pages/Saml/Consume.aspx
• NameID format is urn:oasis:names:tc:SAML:1.1:nameid-format:emailAddress
• Enable assertion signing 
• Supports IdP initiated SSO
• Supports SP initiated SSO

Important points to know,

• According to the SAML specification the EntityId given in the metadata.xml file and the "Issuer" of the SAML response should be equal. But in PeopleHR implementation it need not checked. 
• The Issuer in AuthnRequest must be "test-app-peopleweb" and it is not changeable.

Problems/Limitations

• Not possible to enable Federation Only SSO. The system sends an invitation mail that will ask people to create passwords. And even when this is avoided, when an user sign into PeopleHR for the first time (via SSO) a username/password is requested
• If employees need the mobile app, then native logins must be used

Improvements

• The UI should not say "Upload one login meta-data file". The meta-data file comes from SAML specifications
• The EntityIds must be properly checked at the PeopleHR side
• The EntityId should be configurable and definitely not carry a name as "test-app-peopleweb" as this is a production system