New TCP/UDP (layer 3) load balancer, L3LB

[PeterNic]

Hi all,

This thread is a fork off the new http load balancer discussion thread (http://forum.3tera.com/showthread.php?t=282). This one is focused on layer-3 load balancing (tcp/udp), as opposed to layer 7 (http).

In order to keep appliances simple and easy to use, we decided to split the two load balancers:

HALB remains a layer 7 load balancer, now released in AppLogic 2.4.5 beta
L3LB is the new layer 3 load balancer, still a prototype, available on AppLogic 2.4.5 and 2.1.1


This thread is for discussing L3LB until it is released officially. The preliminary documentation is at http://doc.3tera.com/AppLogic24/CatSwitchesL3LB.html -- please note this is a pre-release appliance, nothing more than a working prototype.

Anyone who wants to give it a try and provide feedback, please reply in this thread or PM me.

Happy balancing,
-- Peter

[PeterNic]

Karl was kind enough to review the new datasheet and give L3LB a try. He OK'd releasing his feedback to the forum for wider discussion. I have edited only to provide numbers.


Bit of early initial thoughts from just looking at it and the docs (just
waiting for app to rebuild).

1. What would be nice to see, would be some
options for the health check i.e. smtpchk and ssl-hello-chk

2. Also nice would be able to specify certain output terminals as backup
only i.e. Servers of last resort - I know we can achieve a similar kind
of functionality with the INSSLR gateway - but that's really for between
applications, not for within an application. Would be nice to have
local application servers of last resort (this also applies to HALB with
HTTP as well so we can put a small web appliance in to say, "Oops, we
broke something" for example - or, "We're doing maintenance", because
we've turned all other outputs off.).

3. More thoughts on this. You allow a list of ports to be specified that
should be used for incoming connections - but the appliance assumes that
all connected outputs can handle all ports. What would be useful, would
be to allow us to specify a global list of ports as now, OR specify a
list of ports for each output terminal.

The reason being, in an app we are developing at the moment, we've got
two outputs connected (port 25), but we could also do with load balacing
another two appliances, which run on different TCP ports - right now we
have to add another appliance, which needlesly uses resources and slows
development down (Editor takes longer to load, more config to maintain)
when we could just use 2 of the 6 spare outputs on the existing L3LB in
the app.


Here are my initial thoughts; will discuss with the engineers and add more if applicable.


1. What would be nice to see, would be some
options for the health check i.e. smtpchk and ssl-hello-chk

I think it will be great to find out if we can make a generic mechanism for extending the health check methods as we can never cover all cases with the standard. Maybe some configuration parameters similar to how old BBS logins were scripted: configure the following string(s):

expected "hello" string
response to provide on behalf of a client
expected response string
(more of the above)


Having said that, having some of the really frequently used protocols -- like SMTP (plain or SSL) -- makes sense to have as built-ins (and using the ones HAProxy already supports is a good indication of what has been judged useful).


2. Also nice would be able to specify certain output terminals as backup
only i.e. Servers of last resort

Yes, that makes sense, both for HALB and for L3LB. In HLB, it didn't quite make sense, I think, mostly as it had no programmatic control. With HALB and L3LB's programmatic control, it is possible to have a bug in the controller lead to disabling all outputs. And, with the better health check, it is possible to discover servers that even though respond to ping/basic TCP, have problems at the back-end (e.g., error 500 to the database). We will likely add an "aux", "slr" (server of last resort" or "bkp" output.


3. You allow a list of ports to be specified that
should be used for incoming connections - but the appliance assumes that
all connected outputs can handle all ports. What would be useful, would
be to allow us to specify a global list of ports as now, OR specify a
list of ports for each output terminal.

I'll probably disagree on this one, at least for the generic catalog appliances. One of the advantages of the AppLogic object model is that it keeps everything neatly visible and separated at runtime. Appliances are designed for a single function, which allows them to be simple to configure and use, fail less often, and scale independently. I understand that sometimes, especially in services that need to scale wildly while keeping costs down, it makes sense to optimize things -- the way they are done in chips for consumer appliances that are replicated in millions -- savings of even $0.02 per instance results in huge advantages far outweighing the increased complexity. My recommendation is to keep things simple during development, then as you get them in production and see where the overhead is (resource use in this case), optimize where needed -- e.g., make the more complex load balancing appliance (either as a completely custom "ASIC" or as a extensively programmable one, as you suggested by adding a volume or fs output to place a extended config file -- akin to the "FPGA" in the chip world). Let me know if I am missing something.


Thanks for the suggestions and we'll work on the next spin.

Regards,
-- Peter

[Karl]

I can see where you're coming from with #3, wanting to keep it following the ethos of being simple to configure, I just think for more advanced users it's a little wasteful - especially if we're already doubling up on load balancers to give full component redundancy anyway - as the chances are, if the LB for the Web frontend fails, the LB for the mySQL is going to be sat idle anyway, so it doesn't matter if both are the same appliance (or rather the same 2 appliances).

In an ideal world we'd use a pair for each part of the application that needs it - but that increases development time, increases build time, increases startup/shutdown/restart time - especially as we've got no individual component restart yet for simple things like a config tweak or a CPU/memory increase (although that's a complete separate discussion and something customers are picking up on).

[Karl]

Right, random out of the box thought, "logical" appliances - We drop a physical appliance on be it a load balancer, web server etc. then we can create logical appliances of it, with their own output configurations etc. kind of like how modern day routers do virtual routing domains etc. That way, we only have one appliance, but it's configured just like a normal appliance, laid out like it - So you get the benefits there, but with the benefits of less overall resource usage and faster startup etc.

Now obviously that isn't going to be suitable for everyone, but I think it'd be quite useful in a number of situations - especially with the web appliance, think of it as a poor mans virtual hosting - very useful if you've got a couple of domains in the same application with different content and a truly separate web instance is overkill and would again slow down restarts/reconfigures etc.

As I say, just an off-the-wall, out-of-the-box idea.

[PeterNic]

Thank you for the comments and suggestions.

The L3LB appliance is now part of the standard catalog in AppLogic 2.4.7. Please post questions and issues at http://forum.3tera.com/showthread.php?t=418.

Best regards,
-- Peter