SaaS Developer Guide - Part 3 - Achieving Economy of Scale

How did Cloud pick up an age-old term such as “economy of scale”? Well, it fits perfectly to the scenario Cloud service providers are trying to achieve. Achieving economy of scale is all about doing super optimized deployment where addition of users/organizations causes minimal or zero incremental cost to the SaaS provider. This blog of the SaaS Developer Guide blog series, introduces a list technologies that are available to realize economy of scale for SaaS.

Why is "economy of scale" so important?

Lets start small. As a Cloud Learner SaaS provider, imagine the initial deployment is done. What if it can only support one lecturer/institute? That is not going to be useful. What if a single deployment of 1000 nodes are put in-place to support everybody? That is not very useful either. The deployment should elastically scale during peak hours of use and we should be able to support maximum number of users with minimal amount of hardware resources. That is how we achieve economy of scale.

There is a list of technologies that are available for SaaS providers, to do a deployment to serve thousands of users based on different levels resource sharing. This is a quotation from IBM developerWorks that explains resource sharing and isolation.

"The more resources that are shared, the higher the density. Higher density lowers the provider's costs. At the same time, increased sharing reduces the level of isolation between tenants — the individual systems or services that are being delivered. Isolation is the degree to which one tenant can affect the activity and data of other tenants."[1]

This blog will concentrate on two of the most popular resource resource sharing mechanisms.

Containers

What is the difference between VM and containers? VMs work on virtualized hardware while containers share the operating system? During the recent years, virtualization technologies have evolved massively. Containerizing is such a technology that took a leap during the past couple of years. Containers work sharing the operating system as opposed to VMs that emulate hardware, making it possible to run five-to-six times more servers with containers as opposed to VMs. In container based SaaS deployment, each subscriber gets a single instance of the SaaS application, running in a container. This ensures SLA to the end user as we guarantee the CPU for each running instance. Docker is one such technology. Docker is the leading light weight container technology today due to the following reasons.

• Easy to use compared to other container technologies
• Docker’s libcontainer is the de-facto standard for Linux containers. It is open source.
• Supported by Google and Redhat
• Docker can be used to pack and ship software

Modern PaaSes such as Kubernetes, GCloud or Cloud Foundry can run docker images. The world is moving to the containers as we are passing the technology-hype curve for docker. Docker can be very effective as containers can be sprung up on first request for the tenant. Containers can killed after an idle time and the same infrastructure resources can be used over an over again to serve 1000s of tenants.

Multi-tenancy

The highest form of density would be using multi-tenancy. When it comes to multi-tenancy, the same application instance will be used by several tenants. In this mode, isolation is minimal and activity of one tenant will affect others. This is a good option for SaaS, if the user activity on the application is predictable and does not need to heavy processing. There are tenant models.

• Each tenant has it's own database
• Each tenant has a scheme in the database
• Each tenant is distinguished by an Id in the schema   

Some SaaS providers use multi-tenant mode to cater to the demo period with cut down set of features. With careful monitoring and distribution of tenants it can provide a large economy of scale, especially if tenants are sharing the same schema and the tenant ID distinctly identify which data belongs to which tenants.

Cloud Learner

Applications can support both multi-tenancy and container based deployment. The Cloud Learner application is implemented in the hybrid approach. This allows the application to be deployed in different modes depending on other aspects such as subsription layers. For example the Platinum subscription layer can have an instance of Cloud Learner application for itself deployed on containers while the demo mode will share the same running instance. 

 
In a future post we'll look at how the dockerized Cloud Learner application is deployed in GCloud.

[1] http://www.ibm.com/developerworks/library/j-multitenant-java/
[2] https://msdn.microsoft.com/en-us/library/aa479086.aspx 

WSO2 Identity Server - Supporting Binary Claims such as Windows SID and objectGUID

Configuring a binary attribute (such as Windows SID or objectGUID) as a claim in WSO2 Identity Server and sending it over XML token such as SAML needs an additional configuration.

Problem Identification

 [2016-03-17 16:48:48,203] @nextlabs.com [1] [IS]ERROR {org.wso2.carbon.identity.sso.saml.processors.SPInitSSOAuthnRequestProcessor} - Error processing the authentication request  
 org.wso2.carbon.identity.base.IdentityException: Error Serializing the SAML Response  
     at org.wso2.carbon.identity.base.IdentityException.error(IdentityException.java:162)  
     at org.wso2.carbon.identity.sso.saml.util.SAMLSSOUtil.marshall(SAMLSSOUtil.java:352)  
     at org.wso2.carbon.identity.sso.saml.processors.SPInitSSOAuthnRequestProcessor.process(SPInitSSOAuthnRequestProcessor.java:161)  
     at org.wso2.carbon.identity.sso.saml.SAMLSSOService.authenticate(SAMLSSOService.java:164)  
     at org.wso2.carbon.identity.sso.saml.servlet.SAMLSSOProviderServlet.handleAuthenticationReponseFromFramework(SAMLSSOProviderServlet.java:691)  
 .....  
 .....  
 .....  
 Caused by: org.w3c.dom.ls.LSException: The character '☼' is an invalid XML character  
     at org.apache.xml.serialize.DOMSerializerImpl.write(Unknown Source)  
     at org.wso2.carbon.identity.sso.saml.util.SAMLSSOUtil.marshall(SAMLSSOUtil.java:348)  
     ... 55 more  
 Caused by: java.io.IOException: The character '☼' is an invalid XML character  
     at org.apache.xml.serialize.BaseMarkupSerializer.fatalError(Unknown Source)  
     at org.apache.xml.serialize.BaseMarkupSerializer.surrogates(Unknown Source)  

Solution

Add the following parameter in <carbon_home>/repository/conf/usermgt.xml for each binary attribute that you wish to convert to XML.

 <Property name="java.naming.ldap.attributes.binary">objectGUID</Property>  

SaaS Developer Guide - Part 2 - Scalability and Resiliency

This blog post is divided into two sections,

• Background - Touches on definitions and calculations
• Current Technology and Execution - Available technology, their features and Cloud Learner deployment

Background 

Distribution Systems has been around in software world for quite a while - since 1970s. It has been evolving, getting new terms, and becoming a key ingredient in Cloud based systems. Formally distributed systems are defined as a system consisting of components communicating over a network. Today all highly SaaS systems are designed as a horizontally scaling "set of services". A service is a evolution of a distributed component. If a well defined interface is given to a component in a distributed system it becomes an interface.

Why Services?

• Services can be developed, deployed and upgraded independantly
• Services can be scaled and made highly available independently. More replication for important services. More nodes for services with higher load
• Allows to use different technologies
• Strong boundaries for modularity

In the Cloud Learner application, there is going to be several services,

• Class CRUD service
• UserMgt service
• Subscription service
• Course CRUD service
• Front end component

Load balanced Nodes

When the load goes high on the "Class CRUD service" we can have two instances of the "Class CRUD service" to serve the requests. The load balancer will distribute the requests between these nodes.

"Elasticity" or "auto-scaling", is the primary scalable ingredient of  a SaaS. It means the system scales up as the load goes high and it scales down as the load becomes low. This allows a SaaS to exploit the full potential of the underlying IaaS/PaaS, which has the ability to provision infrastructure on-demand. Elasticity leads to optimized resource consumption that results in economy of scale. We'll be discussing about "economy of scale" in length in a future blog.

Resiliency and scalability go hand-in-hand, in a large distributed system. For example one of the primary methods of providing resiliency is detecting faulty node and removing it from cluster while and creating another instance for it. If one instance of the component "A" goes down, the performance of that module degrades by (n-1) where n is number of nodes in the module. But the system will continue to function. This is a key feature of resilient architecture.

              Percentage performance hit on a cluster = (n-&)/n * 100%

       where,

              n is the total number of nodes

              & is the faulty number of nodes

When n is large, the impact of loosing a single node shows less performance hit. But what happens when n is small? For example if the number of nodes are 2 then, 1 node malfunctioning is a critical factor.

Everybody starts starts small. So as an initial SaaS provider with a cluster of 2 nodes that are fully utilized by the load, adding an additional node in active mode would give n+1 resilient factor. When it comes to Cloud Learner application only one node is enough to manage subscriptions, but for resilience another node is added.

Current Technology and Execution

Load is predicted and tested, calculations are done, it is important to glance at the current cloud landscape to understand the "execution" aspects of achieving scalability and resilience. The line between PaaS and IaaS are blurring as IaaS providers keep on adding PaaS level features. Following public cloud features can be leveraged for scalability and resiliency.

• Load balancing
• Health check
• Rule based auto scaling
• Rule based routing
• Availability zones 

For example look at the functionalities provided by GCloud for auto-scaling.

Now that is the difference of "going cloud all the way". The sample application can be deployed with scalability and resilience in on a PaaS as follows. There are two FE components. All nodes have n+1 resilient factor. Here the ClassCRUD services is deployed separately because it's load factor is higher than other services. ClassCRUD service the most important service, in the Cloud Learner application as it serves 90% of the user actions. Therefore it has 3 nodes.