PDA

View Full Version : Automate Administration of Applications


mgorman
05-30-2007, 02:26 AM
There is something that every ISV like ourself is going to have to solve. It's the one thing that anyone looking at your "multi-tenant" strategy is going to bring up. (Even our sales guy saw this right away!) That is this:

Once you start creating copies of your software for your various tenants, the problem is you've got these... well... copies. Even if you've taken care to template-itize your app or appliance-itize it, there are still things you're going to have to roll out across your tenants.

A simple example is a new WAR file. But you may say "You can fix that by locating the WAR file on a read only volume that's shared", etc., etc., to which I would say "Okay, well what if you had to do some kind
of db cleanup where you need to run some kind of UPDATE statement against each tenant's MySQL database" to which you would say "Okay, you got us there."

So the problem that every ISV like us will face is "How do I automate the administration of the environments of all these tenants?" AppLogic has made it ridiculously easy to create the tenants' environments, but doesn't really offer me anything to help automate the subsequent administration.

What we envision is some kind of web-based "Application Manager" that allows simple scripts to be uploaded and executed against one or more application instances. To do this, I imagine something like this:


AppLogic runs some kind of "Application Manager Agent" on each running application. This agent monitors a certain inbound FTP directory for item #2 below...
AppLogic defines some kind of "Maintenance Archive" file that we dub a "MAR File" to make marketing happy. This MAR file is just a zip file that contains a execute.sh shell script along with anything else we want. So in this MAR file we can embed a new WAR file, SQL scripts, ant scripts, other shell scripts, whatever we want.
Using the web-based Application Manager, we would be able to browse our hard-drive for the MAR file, multi-select the application instances we want to apply the MAR file to, and the application manager would ftp the file to each of the running instances.
The Application Manager Agent running on each instance would notice "Oh, a new MAR file has arrived", at which point it would unzip it and execute execute.sh which would do whatever we want it to do.


If you don't build something like that, I think we are going to have to... and we really don't want to spend our time building out IT infrastructure tools. I think this would silence the critics that say you haven't really solved the whole multi-tenancy problem, you've just shortened the IT cycle.

nobu
05-30-2007, 06:27 PM
I think something similar can be done via scp zip file to aplliacnes and by executing a script via ssh.
Command execution part can be replaced by cronjob constantly monitoring the incoming MAR ball, too, I guess.

PeterNic
05-31-2007, 04:25 PM
Well, at the what-can-be-done-with-today's version, Nobu is right and it can be arranged in one of 3 ways:
- common NAS box that holds all the WAR files, each app is caching it periodically
- develop a script that pushes the new WAR files to the NAS of each application (the script can be run on a client workstation or in a GSC or similar appliance on the grid). You can use the ssh forwarding that allows 'scp' to the controller to forward either to a volume (of an inactive appliance) or to forward as scp to a running appliance.
- if the WAR files and/or other code is on the class volumes of an appliance, a good way to upgrade apps is to replace the class and restart the apps (appliance instances are not copies, they are clones; if the class changes, the instances will refresh themselves and take the new class volumes and settings). This is a good way to do both upgrades as well as hotfixes.

Nevertheless, a more complete and user-friendly solution is going to be discussed here further. This is certainly interesting and we would want the feedback of everyone who considers himself a multi-tenant SaaS (or simply has an opinion on this).

barmijo
06-01-2007, 12:09 PM
I'm the non-engineer in the thread but I see a couple downsides to what you've described. First, it forces all instances to convert at the same time. This may "seem" desirable, but if there's any issue your support will be swamped. Second, and more important, it leaves you with no backup of each instance to revert to if anything goes wrong.

I'd lean towards doing this in a different way. I'd create a new version of the appliance that includes the new code and a script that recognizes the old dataset and converts it if needed. This script would only run once, the first time the new appliance is started.

To trigger the upgrade I'd create another script running on a workstation that forces a restart of each instance during pre-scheduled maintenance periods. Between the stop and start of each appliance it'd also make a backup copy. As another alternative, you could actually allow the user to trigger the upgrade through the UI or possible through a separate portal.