The SaaS Aggregation Benefit Mirage

In this service oriented on-demand world I’ve been running into something again and again lately that I’ve found interesting and a bit annoying.

To start, imagine I’m going to build an application that uses two 3rd party services on-demand.  We’ll just call them service A and service B and say each have two features.  For this example it does not really matter what the services do.

Service A
  Feature A-1
  Feature A-2
Service B
   Feature B-1
   Feature B-2

So, I create my application and it first uses service A do something and it uses Feature A-1 and A-2.  Then, with the output of that it uses service B to do something else using feature B-2.

Now, a few months down the line when things are going great I get a call from my account manager at Service A telling me I can now get all the features of service B directly from them included.  So, what they are telling me is that my service structure now looks like this:

Service A
  Feature A-1
  Feature A-2
  Feature B-1
  Feature B-2
Service B
   Feature B-1
   Feature B-2

On the surface this looks really good.  It’s the same thing with less hassle right?  Maybe not.

This is where my annoyance surfaces.  Dig in and dig in well.  What I find again and again is that it’s simply not true because of what I’ll just call the filter effect.  What you really are getting with this new and improved service A is more like.

Service A
  Feature A-1
   Feature A-2
   Feature B-1

Notice that Feature B-2 is missing and that probably no body mentioned it.  Or, it’s more like:

Service A
   Feature A-1
   Feature A-2
   Feature C-1
   Feature C-2
   Feature C-3
   Feature C-n-OMG
Service B
   Feature B-1
   Feature B-2

And you don’t care because C isn’t B and all you need as A-1, A-2, and B-2.  While they say it’s equal is not and the app use feature B-2 if you’ll recall.  How much time did you just spend?

So, by the time you get through all this and figure out that the new improved Service A + B is pretty useless and all you really want is what you already have you will have wasted a lot of time.  There are less features, more complexity, less control, and likely much worse service and support for the aggregated services since you have no direct relationship to the end point provider.

So, rambling aside the point is that these service provider mashup aggregaters are not what they often seem on the surface and I’m frequently finding that the best deal is going right to the source and that any “savings” on the surface likely gets eaten up later in a variety of ways that are difficult to predict.  In most cases, it’s best to go to the source to get what you want.

Stop Staring at my Polyglot!

I received an interesting comment/question via my blog recently.  It went a bit like this... 

I’m developing a distributed cloud application but my developers are pushing back on me for having a polyglot database strategy.  What should I do?

I won’t get into exactly what it is they are doing since that would take several more pages.  This is something of a stream of thought post so apologies if it is a little rough around the edges.  The easiest way to answer is in the context of an application I’ve been working on for a while that has some similarities to what this person wants to build.  Everything I’m describing is part of an app I’ve been building with a client since earlier this year.

Typically you'll need a few layers of "data storage” for any distributed batch or real time application (cloud native application) which is what I understand that you are trying to build.

I consider anything that holds data that is for presentation, computation, or transformation part of the data storage architecture and I like to break it down by time in storage from least amount of time to most. 

Short or Very Short Term: Single node caches (like memcached) or volatile computer node memory
Mid Term: Queue's and IMDG's
Long Term  Durable Storage:  Dynamo and BigTable derivatives abound 

There are numerous database products that live in or even between those tiers these days; more than ever before.  By no means is what follows even close to an exhaustive list.  A quick list of the ones I have worked with in the last few months personally looks like: 

Short Term: Memcached, Redis, RabbitMQ, ZeroMQ, DRAM, APC, MongoDB
Mid-Term: Redis, GridGain, RabbitMQ, ZeroMQ, MongoDB
Long-Term: Riak, MongoDB, S3, Ceph, Swift, CloudFiles, EBS, HBase

Short-Term storage is ALL in memory, not persisted to disk, and not intended to be used for long periods of time.  Your application also has to be able to deal with the fact that this type of storage is essentially ephemeral.  If the node gets a KILL signal from some source or another your app needs to know how to deal with this gracefully.  In other words, storage here is not durable at all.

Mid-Term storage is used for longer running processes.  It benefits greatly from being distributed and having a higher degree of durability.  This is generally still where most of the work in done in main system memory (no disk I/O) but also where you might do complex calculations or data transformations on your way to your goal.  You do it here because it’s fast.  You do things here because they can be shared amongst lots of workers (like queue subscribers). 

Long-Term storage is used for exactly that, long term durable storage of important data that provides sufficient and reasonable interfaces from which to retrieve that data again when needed.  Preferably it’s possible to do things like map-reduce jobs so that you can iterate and retrieve what is necessary which you may then operate on at one of the higher levels up this stack.

You’ll see that I’ve put some of them in all or multiple categories which might seem odd until you understand how they work and match the technology to what ever you are trying to achieve from a business perspective.

I have a tendency to avoid things that require overly complex operational management issues for starting up projects because I like to try to get my TCO (Total Cost of Ownership) over time (3-5 years) as low as possible while achieving the project goals and SLA’s.  There are a couple of exceptions on the list above that do have more operational overhead (MongoDB and HBase) but they are good enough in the right context that you might want to learn and use them anyway.

Now, back to the question at hand.  Should I use one type of DB or many for the needs at hand.  In this case, I’ve told them that they should use as few as possible, possibly only one.  The reason for this was that in their case they will value speed, consistency, and lower cost of operations at this early stage of their project.  They are developing an interesting distributed system for cool reasons.  I recommended a choice to them that I think will help them get to their goals fast and cost effectively while allowing them down the line to break off pieces of the application later as and if needed.

Parting words are that it will, over time, be nearly unavoidable that this (and most) applications of a distributed nature end up being database polyglotoumous.  However, I do think it adds a lot of complexity and overhead and in the early stages of a project it's not usually necessary unless what you are doing is of great complexity in which case you might want to break that down anyway to something more manageable.

Cloud Operating Systems: Do They Exist Yet?

I was asked an interesting question by friend of mine a few days ago.  He says simply, “Are there any Cloud Operating Systems?”

I then proceeded to argue that no, there really were not any because all of the necessary pre-requisites for being both cloud native while providing the basic services an operating system should provide were not available in the current crop of software claiming to be a cloud operating system upon which I could build cloud native applications.

Let’s think about what an Operating System gives you.  An OS is essentially an abstraction layer over hardware that provides the necessary interfaces(drivers) to get the work done that is needed by computer programs.  At its most basic an operating system provides storage, networking, and compute and the management of those things in a fairly transparent way.   It also provides a tremendous amount of flexibility to it’s user.

Wikipedia defines an Operating System as, “software, consisting of programs and data, that runs on computers, manages computer hardware resources, and provides common services for execution of various application software. The operating system is the most important type of system software in a computer system. Without an operating system, a user cannot run an application program on their computer, unless the application program is self booting.”  -wikipedia

If I was to write the definition for wikipedia using the definition for an operation system as the bse then I would say, “A cloud operating system (COS) is software, consisting of programs and data, that runs on IaaS providers, manages IaaS resources, and provides common services for execution of various application software and SaaS. The cloud operating system is the most important type of system software in a cloud computing system. Without a cloud operating system, a user cannot run a cloud native application program on the cloud computer, unless the application program is self booting.

To me, this sounds most like what we currently often call Platform as a Service (PaaS).

The problem I have with current PaaS systems is that they dictate heavily and take away far too much from the programmer from the underlying services.  This is a compromise of course.  But,  a true operating system is far more flexible than the existing crop of PaaS services.  Just imagine if your newest operating system upgrade wouldn’t let you access ½ the CPU cores or the networking services for some reason and your manufacture just said, tough cookies because that’s just how it is.  You’d probably try to return it if you could.

Where this got interesting for me in thinking about it later was the concept of “self-booting” that was mentioned in the wikipedia defintion above.  If you build the necessary intelligence into your PaaS such that it can bootstrap an application, the storage, networking, and compute necessary to run and then manage that moving forward then you do, in fact, have something more resembling a cloud operating system as defined above.

Scale Planning and the AKF Scale Cube

There are a lot of ways to draw diagrams for availability and scalability.  I use different ones for different purposes all the time.  However, when I was reading the Art of Scalability by AKF partners I ran across a nice compound diagram they call the AKF Scale Cube which helps simplify the explanation of the multi-dimensional nature of scalability issues in complex web application scenarios.  

I’ve been using this visualization model to help me explain how things fit together in both technical and business discussions.  It comes in very handy I must say.

Most recent I’ve been using it to describe a gnarly distributed application I’m working on for a client.  What follows is a generalized version of a functional use and some discussion of a compound view of ones I have created for clients of mine.

 


Some base-line definitions are in order if you haven’t read the book mentioned above.

X-Axis - Horizontal Scalability, Clones, Scale Out.  These are terms often associated with the X-Axis.  In the case of this graph, day you build a data processor then make 10 copies of it.  Well, that’s scaling to 10 units on the x-axis in this graph.  Depending on your application, this can help you increase your capacity; but not always!

Y-Axis - I’ve called this axis functional decomposition for a long time.  It can be thought of as breaking the application down from a monolithic single instance into discreet stand-alone parts.  I have some examples that are a bit of mix from various projects I’ve worked on in the past here.

Z-Axis - This is the tricky one for most folks.  This is what people might call sharding, partitioning, etc.  Keep in mind, I’m not only talking about a database here.  I’m talking about an entire complex multi-faceted distributed highly-available web application.

0,0,0 - The intersection of all three axis or 0,0,0.  This is what some would call an all-in-one server.  It’s often used for proof of concept for for launching without a care in the world for future growth needs.  There’s nothing wrong with it as long as you understand the limitations and technical debt associated with the approach.

Z-Axis Item Explanations

Client - Assuming this is a multi-tentant application you may want to shard your application by client such that each client or group of clients is assigned somehow to a specific cluster of nodes.

Geography - Assuming you’d like to have built in DRBC and your applciation is capable of surviving being split up into many pieces then you could end up sharding your application by data center and broader geographies such as city, state, country.

External Cloud - Using IaaS and PaaS resources outside of your own data centers.  For a refresher on IaaS and PaaS see the article I wrote, “Cloud Computing:  Get Your Head in the Clouds,” in 2008 that was heavily read over the years.

Internal Cloud - Using your own infrastructure resources BEHIND your own firewall to do whatever it is that your application does.  This doesn’t always have to be a cloud.  If you want to know what I think it takes to be a cloud then read the several articles I wrote over the years related to that topic.  I do set the bar pretty high though I’ve learned.

Purpose - You might want to simply partition along the Z-Axis by any generic purpose for various reasons.  I think of this a little bit like saying I want to put all widgets in data node 1 and all waggles in data node 2.  They’ll both fit in a single node but maybe I want to spread my risk around.  This one is a little nebulous but it can engender fun conversations about why things need to exist at all.

Shard Key - We see this all the time in traditional RDBMS style deployments and even in some of the newer tools in the NOSQL world.  It’s basically just some index of what node you put things one somewhere.  For those of you that had to deal with libraries before the internet you’ll remember the lovely card catalog.  It’s was nicely set up to help you figure out with shard of the library your book was close to.  Then, when you got there, good old dewey decimal system kicked in to take you the rest of the way.

Y-Axis Item Explanations

Data Processing - this could be some application that transforms data from one state to another.  For example, it might simply remove all the spaces in a document and replace them with dashes.  That’s a bit of a silly example, but just to make the point.

Data Aggregator - I’ve had to build project after project that needed one form or another of data aggregation.  So, just think of this as something that might consume and RSS feed and stick it in a database of some kind.

Distributed Calculation - I’ve been doing work and research with Map-Reduce, Actor Models, the Bulk Synchronous Parallel Model and more exotic instruments from past, present, and future.  This is simply something that does some kind of math or calculation of some sort.  For example, counting all the uses of the word onomatopoeia in 50TB of English essays by high school students across 100’s of of compute nodes.

Processor App - This is just a generic discreet application that processes something, like an API request for example.

Web App - This is an application, in my case, written in a modern MVC framework that has the job of interacting with web users and getting things done in the back-ground in various ways with various services.

Base Installation - I think of this as just shared code.  One of the developers I have been working with recently suggested that we extract a number of commonly used components from various application pieces on the Y-Axis and build a library of sorts.  Great suggestion in this case, so I stuck in on my general diagram to remind me in the future.

What’s interesting about all these conceptual applications is that if you create them correctly and with the correct architectural models that each item that lives on the X Axis will also be able to scale on the X and Z axis.  For example, you could have your web application running 5 X-Axis copies in 4 Y-Axis partitions; say external cloud, by client, purpose, and by geography per client.  So, you’d end up use four AWS Availability zones in 2 AWS locations running 96 application nodes in total.  Of course, your application has to be built correctly to take advantage of all of this distribution at every level.  But, that’s a topic for a later date I suppose.

In summary, this post was just to share some of my thinking around the use of a very nice visualization tool by the fine folks at AKF Partners.  So, a shout out to them for the nice tool and hopefully this helps people a bit understand how it can be used / thought of in a variety of ways.

Just remember, it's not one-size-fits all.  Your use, labels, and needs for such things will vary greatly depending on what you are trying to architect, develop, and deploy.

Cloud Native Applications

I’ve always believed that cloud computing is really two things. One, it is a technology architecture. Two, it is a business operating paradigm that we often call on-demand. Your application must satisfy the on-demand business model requirements satisfy the technical architecture requirements to be a cloud native application. There are not very many cloud native applications running in the world today. This is changing quickly. A Cloud Native Application is architected and designed to run on what is commonly referred to as a cloud IaaS or PaaS. The words I used there are very important. It is architected and designed to run in the cloud from the beginning. Therefore, it has a number of important traits as part of it’s DNA. The traits that a Cloud Native Application must have are:

Click to read more ...

Being Cloud

What it takes to be cloud has and continues to change rapidly. Today, in my opinion, the following “features” are required for something to really be cloud. Each of these can be (and has been) written about at length all over the internet. But, it’s putting them together that creates the magic of the cloud.

Click to read more ...