Minimize app restarts?

[jonesy]

Hi all,

For the record, I'm still not in production. However, the learning curve seems to be flattening out a bit for me :)

I now have my application basically in place, but am now trying to make sure I've taken advantage of every architectural tool available to me to minimize the number of operations requiring a restart. The main thing I can see me doing that would require a restart would be making new connections between servers internal to the application, and making/enabling external connections to the internet.

Right now, I have no switch or HLB device in my configuration. I just have a web server, db server, and the nas and monitor devices (and a couple of devices that aren't yet connected to anything, like an app server I'm writing). I'm going to add another db server, another web server, etc. sooner rather than later (the db server, at least, will be added before we go to production).

What I'm noticing in playing around with adding components, though, is that there are yet more instances where the applogic model is forcing me to architect things a bit differently and think at a much lower level in much greater detail *before* I launch, which to my brain translates into a bit less flexibility. Here's what I mean:

I have a db server. The only thing connected to it right now is the web server. The db server uses an application volume to store its database files. This is a mysql5 device I built myself from scratch before I was upgraded to 2.1.1 (which supplies a ready-made mysql5 device).

I want to add one of those shiny new mysql5 devices supplied by 2.1.1, copy over my dump file I used to test my own mysql5 appliance, and use it to test the new appliance. However, there is no shared volume between them, and there is no existing network connection between them.

In a normal (read: physical) server environment, I'd either share an NFS directory between the two machines (even if it's temporarily), and/or enable them to ssh/scp to/from each other. In either of those instances (and probably more I'm not thinking of), *nothing* gets restarted. However, to do *any* of these operations now requires me not only to restart an individual server, but to restart the entire world, incurring several minutes of application downtime.

Is there a way using some gateway or switch device to set up permanent connections between the servers and to the 'NET' device, enable/disable switch ports as the connections are needed/not needed, and just restart the *switch* to get things going instead of the entire application?

[PeterNic]

Yes, there is a way:

1. Probably one of the examples is the LampCluster template (http://doc.3tera.net/AppLogic2/RefAppLampCluster.html), where we built a LAMP cluster application that gives access to developers to each individual appliance, without requiring that each developer has access to the grid GUI. Even there, you can log into the PS8 port switch and do anything you want between the appliances that are connected to the port switch.

2. You can also look at the WEBx4 or WEBx8 classes (you will need to step inside the appliance, View Interior and open the Instance Settings Attribute sheet on srv1 and compare to srv2/3/4), to see the "standby" attribute. You can pre-design an appliance, have it built and verified and ready to run, but keep it on standby until you need it. While the appliance is on standby, it doesn't take any resources (other than an instance of the boot volume, so it is ready to run). This will allow you to test your database clustering and know it works solid; then just mark the db2 appliance as standby and don't turn it on until you really need it.

3. Access between appliances: you will want to branch and modify MySQL (which you already have done).

3.1. Option 1: Add to your MySQL appliance a "ssh" or "nfs" input terminal. Then hook up either a port switch in front (works for ssh, not so well for nfs); or add a LUX-based "management console" appliance that simply has outputs db1 and db2. Keep the management console on standby until you need it -- when you need it, start it, ssh into it, do your copying, exit and stop it. You can also give it a "net" output in case you would want to access the Internet during the management (e.g., to download the dump file or to save a dump file to external storage service like Dynavol or S3).

3.2. Option 2: In your MySQL appliance, create an nfs output. Connect the nfs outputs of the MySQL appliances to a NAS box (see the WEB5 appliance on the best way to mount the nfs share in an appliance class). Then, whenever you like, log into any of the MySQL appliances and do whatever you need to. If you don't need the NAS appliance all the time, you can mark it as standby.



4. You may also care about the failover groups -- instance attribute that allows you to force two or more appliances to reside on different physical servers, so if a physical server fails, it won't take down more than one of the appliances.

Note: forcing you to think ahead of time -- and to be able to make and test your changes outside of your production environment -- is intended. Yes, it probably irritates you; hopefully it will save you from a major problem down the line (there are no applications that don't have downtime; there are those applications that have planned downtime and those that have unplanned downtime. We like the first group). And not to be snide, dynamic updates to the application structure is something that has been requested by a few customers (see the New Features forum) and is strongly on our wishlist for new features for a future version of AppLogic.

Finally, there are a few more approaches to structuring your apps that may help you... you may want to set up a support session with Jessie or get the ASP plan (http://www.3tera.com/assured-success-plan.html) (we'll get you a discount on it in appreciation of the feedback you are providing).

Best regards,
-- Peter