Monitoring Appliance - Historical Data

[dclark@mindsource.com]

Our installed base needs a way to access historical trending data from the monitoring system.
We are accustomed to having log sizes of 2-10 TB and our customers expect this (with sane rotation, etc).

Tim suggests utilizing the monitoring appliance and being able to set two flags: First - turn on / off reporting
Second - time interval for more granular metrics. ie: so that we can look at hit counters on a one minute basis or five minute basis. Unlike syslog which allows facilities and priorities - this would provide /all/ of the data.

And we sell the storage for this ;-)

We are so close to this - you already can make the graphs. RedPeak is happy to assist. Can we set aside a day to do this?

[PeterNic]

Dave,


We are so close to this - you already can make the graphs.

I'd love to see it.


Can we set aside a day to do this?

What is the "this"?

Best regards,
-- Peter

[tmf-redpeak]

Actually, its not that "we" are almost there - its that the monitoring application is almost there in its current form! - if someone could point at some file locations for a deeper feel of the internals of the monitoring system - maybe "we" can turn on the right stuff to make things happen.

I am envisioning the ability to set a global interval for when the counters are recorded. And a new box to the monitoring system toolbar that allows us to set a timeframe to view.

To address granularity concerns I see a need to be able to apply time intervals to specific counters - this could be done either via a file edit or via the graph creation UI.

Tim

[tmf-redpeak]

I think that this can be done really easy - i just don't know where to put the code quite yet -

I believe the monitoring application is updating a mysql database with real time statics, but it is doing it in a static fashion - If a new table for historical counter data was created and an SQL insert was made whenever data was posted by connected appliances then there would be a place for the data - also the database location would need to be changed to the mount point /mnt/data/mysql -something like that.

Then in the UI - I think if a box could be added to the graph to specify a time frame of some sort - then in the code that does an AJAX retrieval of the realtime single data points we have it go to a different url that pulls data from the table above and places that data in the Browser cache instead of single points - then the graph can just behave as normal I believe - this may need a bit more tweaking but essentially its this "simple".

Tim

[tsuliman]

I was thinking adding to that post to add some thing like Nagios or any other opensource monitoring appliance to the applogic or may be integrate those application to your monitoring appliance there are a whole market for cloud solutions and dealing with one part of it doesnt really will likely make you compete with VMWARE and others who claimed to be a cloud provider I think expanding monitoring solutions conversion and migration would be really great would you be open to that ?

[PeterNic]

tsulliman,

I agree this would be useful and we're looking at several options.

One thing that has helped others is the web services API exposed by MON, through which it is quite easy to obtain data and feed it into your monitoring system of choice (and probably, in the process, consolidate the 1-second samples to something more suitable for long term storage).

I will post here with any further developments.

Best regards,
- Peter