LampCluster Application

[BeckyHester]

This thread is dedicated to questions and comments related to the LampCluster - Scalable Lamp Cluster sample application.
The data sheet for the LampCluster application can be found at:
AppLogic 2.4.x: http://doc.3tera.net/AppLogic24/RefAppLampCluster.html
AppLogic 2.7.x: http://doc.3tera.net/AppLogic27/RefAppLampCluster.html (page will be published concurrently with 2.7 release)

Please post any questions and/or comments here.

[GridNote]

Broken Link

http://doc.3tera.net/AppLogic2/RefAppsLampCluster.html

[PeterNic]

GridNote, thanks! The broken link is fixed.

[jonesy]

The documentation for this application says to assign a "web_ip", which I imagine is what our DNS servers will point to. This seems pretty intuitive. However, I'm unclear as to the distinction between "net_ip" and "gateway", and I'm supposed to assign *both* apparently.

gateway seems intuitive enough. The IP is right on the dashboard. It needs to be routable, etc. However, I guess I'm unclear as to what net_ip is. Conceptually, it would seem to be the internal nic on a virtual gateway router, and then the 'gateway' ip would be the external nic. However, I don't have a pool of non-routable IP addresses on my dashboard, and I don't know what IP addressing scheme will be used by the internal network (the appliances) to talk to each other. On the apps I've done thus far, the "NET" device has had its internal network IP autoassigned, and I've just had to supply a gateway address.

How do I populate the value for "net_ip" so I can provision this app?
Thanks.

[PeterNic]

Brian,

The LampCluster application requires two public, routable IP addresses: the web_ip and the net_ip. The web_ip is the "service point" where users will point their browsers to use the app. The net_ip is an IP address that the application will use to reach services that it may use that are outside of the app; it is needed only if your app initiates outgoing service requests (e.g., to google, pool.ntp.org, S3, etc.).

If you open the editor and see the property redirections, you will see that in_web's ip_addr property is redirected to the web_ip property of the application; net's ip_addr property is redirected to the net_ip property of the application. (Maybe we need a feature to show the reverse mapping, like Excel highlights the cells used in a formula when you click on the formula.)

The web_ip and the net_ip *must* be different.

Regards,
-- Peter

[avantassel]

Hi,

I need DOM enabled in the PHP lamp cluster, seems the default has --disable-dom set in the configure.

Where is the source that this was installed from? I'm guessing I need to download it and run configure again. I was hoping there was a less painful way since we have live sites running.

any ideas? Thanks,

Andrew

[Jsmart]

I installed php in this appliance the old fashion way :)

"yum -y install php"

frankly, this is the way I installed all of the packages in the srvX and dbase appliances so that they would be as close to a standard install as possible.

You should be able to download the package and reinstall it with any options you need; the base appliance should have all the build tools and libs.

--Jessie

[avantassel]

I'm all about the old fashion way but the problem is I have a live website running and can't really bring it down to reinstall PHP.

So I will have to reinstall PHP on each server, srv1-srv4. How can I do this without affecting traffic? Is there a way to have say, srv4 up and running but to appear to the load balancer that it is down or to tell the HLB to not send traffic to it?

[PeterNic]

It would be better not to do this on a live site (esp. if you need to back out the changes), but if you must, you can do it.

You can make any of the web servers disappear from the HLB pool by simply "down"-ing the network interface that is its "in" terminal. ("ping in" will tell you the IP address, "ifconfig" will show you which interface it is to do "ifdown" on.).

Once upgraded, a 'comp restart' from the grid shell will reboot the server and bring it back into operation -- it takes HLB up to a minute to decide to use the server again, so you may want to wait until traffic is being forwarded to the server prior to taking down another server for upgrade.

Finally, it may make sense to leave one of the servers (e.g., srv1 or srv4) not upgraded for a week or so (and either keep it downed or turned off and marked as standby), so that if you find a problem with the upgraded PHP, you will have an easy way to keep the site running.

Let me know if this helps.

Regards,
-- Peter

[avantassel]

I went ahead and tried 'yum -y install php' on one of the servers, this seemed to upgrade php and php -i now shows that dom is enabled as shared, however this did not seem to update the libphp5.so which is used by apache (httpd) so phpinfo() still shows dom being disabled. any ideas how to update this? Do i need to rebuild apache and php to get this to work?

[PeterNic]

avantassel - I don't know the details around php installation with dom. However, the problem is the same as it would be if you were on a physical server, so you may be able to find help on the php forums. If you find something -- or hit a different issue -- please post here.

One approach would be to install the right php on a physical server and move the files you need to the LampCluster servers.

Regards,
-- Peter

[dkblinux98]

I used the LampCluster sample app as the base for creating a similar app but with WEB64 so that the OS would be 64-bit instead of 32. I also replaced the MYSQL5 with MYSQL64. (I should mention that I'm using the Beta AppLogic instance).

I had to branch the appliances WEB64 and MYSQL64 to add the IN SSH terminal for the PS8 appliance.
I also added a second INSSL for a non-load balanced web server connection.

When I tried to start the application the first time, it started the net and nas appliances and then hung on the mon appliance. Logs reported insufficient network resources. I wound up putting all appliances on standby except for inssl, the second inssll, the net and the nas. They will all come up on start. But when I try to start the dbase, the hlb, the ps8 or any of the web servers, I get the insufficient network resources error.

Only the web and database servers are modified appliances that I put in the design. The hlb and ps8 and mon were already there. I'm not sure whether the complaint is due to the application boundary having insufficient resources or the individual appliances or the grid itself. It is a development grid with nothing else other than this application running on it at the moment. Any help would be appreciated.

[Chris]

dkblinux98,

If you do "log list n=100" from the controller shell does it give anymore detailed info?

You can also do "grid info" which will give you total and allocated bandwidth info on the entire grid.

"app info appname" will give you bandwidth info for the specific application.

Then one step further to get info on your dbase appliance you can do "comp list appname" This will give you a quick run down of the default bandwidth limits set.

From the control panel -
Right clicking an appliance and going to resources will allow you to change the bandwidth limits for an appliance. Options>App config>Resources will allow you to make changes to the global limits of an application.

[dkblinux98]

Thanks for the reply. Here is some of the output...

collegeconfidential college_confidential> grid info
Grid Name : collegeconfidential
Grid Description :
AppLogic Version : 2.3.9
HA State : unavailable
Total Servers : 1
Servers Running : 1
Servers Enabled : 1
Applications Running : 0
Total CPUs : 8.00
Free CPUs : 7.75
CPU Load : 0.00
Allocated CPUs : 0.00
Total Memory : 8.00 GB
Reserved Memory : 1.51 GB
Free Memory : 6.49 GB
Allocated Memory : 0.00 MB
Total Bandwidth : 2000.00 Mbps
Free Bandwidth : 2000.00 Mbps
Allocated Bandwidth : 0.00 Mbps
Total Disks : 1
Total Storage : 925.45 GB
Free Storage : 388.46 GB

collegeconfidential college_confidential> app info
Name : college_confidential
Description : Scalable LAMP Cluster Application (v1.2.3-1)
ID : 1
Documentation URL : http://doc.3tera.net/AppLogic23/RefAppLampCluster.html
Template : no
Locked : no
User1 :
User2 :
Destroy on Stop : no
State : stopped
CPU : 4.45
Memory : 3.12 GB
Bandwidth : 2350 Mbps
Default Login Appliance :

I saw where the allocated bandwidth exceeds the available bandwidth so I've reduced that allocation down to 1G total. I then started the data, and net components and then attempted to manually start the dbase component. After an unusually long wait, it eventually failed. Here is the log output (not very helpful)...

collegeconfidential college_confidential> log list n=1000
Aug 17 04:02:04 collegeconfidential syslogd 1.4.1: restart.
Aug 18 08:10:05 collegeconfidential applogic: [200:error s:47]: srv1: VM 'vm.srv1.college_confidential.main.hlb' failed to be started.
Aug 18 08:10:05 collegeconfidential applogic: [66603:error s:47]: college_confidential: vm.srv1.college_confidential.main.hlb: failed to start VM
Aug 18 08:10:05 collegeconfidential applogic: [66572:error s:0]: college_confidential:main.hlb: failed to start component
Aug 18 12:46:12 collegeconfidential 3tshell: interactive shell stop, netadmin@hobsons-us 0,1,0
Aug 18 13:15:26 collegeconfidential applogic: [66572:error s:0]: college_confidential:main.dbase: failed to start component
Aug 18 13:15:26 collegeconfidential applogic: [66570:error s:51]: college_confidential:main.dbase: failed tofree resource for component
Aug 18 13:15:26 collegeconfidential applogic: [233:error s:81]: srv1: VM 'vm.srv1.college_confidential.main.dbase' failed to boot within boot timeout period.

[PeterNic]

dkblinux98,

Can you provide the output of "app info college_confidential -v"... I suspect there may be insufficient resources allocated to some appliances. Make sure you set the defaults on each appliance and that you don't override the resource values on "app start" (there was an old bug which I would like to make sure we avoid).

The two most frequent reasons catalog appliances fail to start:
- their boot disk is full (sometimes the logs fill up the volume). Fix: either delete the cca logs (in /var/log/cca) or simply do an "app clean <appname>" and start the app again
- they have insufficient resources to start (usually too little memory); if in doubt, give 128M for small appliances (e.g., gateways) and 512M to big appliances (app/web/database servers) -- just to check they would start if they have sufficient resources (this is a troubleshooting technique).

The most frequent reason new custom appliances don't start is that you missed to install the APK or it didn't install correctly. The must have steps that APK does are start vmad as soon as the networking in up, and at some point, run vme with started_ok event, telling AppLogic that the appliance started OK (if you don't do this, AppLogic will eventually time out the start and decide the appliance has hung). If you think this may be happening -- or there is some other trouble in the custom appliances -- try starting the app with --debug option. If the start begins to wait too long on a given appliance, either wait for the timeout or open another grid shell and try logging in to the appliance (another troubleshooting tip). Once you fix it all up, try "app start" without the --debug option -- a well-behaved application should start OK without it.

Does this help?

Regards,
-- Peter

[dkblinux98]

Peter,
Thanks for the additional debugging information. I wound up deleting the version of LampCluster that I created and I just provisioned the LampCluster_1_1_1_1 as-is. It started fine so I'll working on making minor modifications with it.

I'm having trouble with one thing. The app creates a dbase store which is directly mounted within the dbase server as opposed to being on the NAS. How do I copy my existing database information to that store?

[dkblinux98]

Nevermind. I was able to use fireftp to sftp to the db using the custom PS8 port. Thanks for the quick replies and assistance with this wonderful technology.

Oh, one more thing. I'm trying to clean up the storage space I've used up testing this environment. I want to resize a volume. I can see the volume...

collegeconfidential dev-cc> vol list dev-cc/volcache
Name Size(MB) Server State Filesystem Mirrored Mount State
---------------------------------------------------------------------------------------------------------------
main.vps64_centos50.boot 2560 srv1 ok ext3 N available
main.vps64_centos50.root 128000 srv1 ok ext3 N available

But I can't seem to get the syntax right to resize the volume. Any help would be appreciated.

[Chris]

dkblinux98,

If your wanting to resize the boot volume you will need to branch the appliance if you haven't done so already. (In gui right click appliance and hit branch)

"vol list appname --all" should show a new volume for boot once it has been branched.

Then to resize "vol resize name size=10G" obviously changing the 10G to what size you want.

[PeterNic]

dkblinux98,

Class volumes of catalog appliances cannot be resized -- they come with the class. As Chris mentioned, if you need to resize them, you need to branch the appliance and create your own custom class. Most of the catalog appliances are created with very small volumes.

If you just want to free up some disk space on the grid, you can either delete unused dev backups of your app or at least do "app clean" on them -- the latter will remove all the volcache volumes (which will be reinstantiated if you ever want to start the app again).

Regards,
-- Peter