One advantage of using Monitis to monitor your systems and applications is the flexibility to use either the native agent or custom monitor code written in virtually any language. For custom monitors, the REST API provides the basic foundation to interface programmatically with Monitis. For many popular languages, there are open source SDKs available to make the process of interfacing with Monitis even easier. You can find links to Java, Perl, PHP, Ruby, C#, PowerShell, and VisualBasic SDKs and example scripts at http://monitis.com/api/api.html#sdk.
The Monitis Sandbox
For this article, we’ll be using the Monitis sandbox, a free development environment available to aid in the creation of Monitis custom monitors. You can sign up for an account at http://sandbox.monitis.com/.
Netstat is a command available on unix systems to provide information about the state of the network interfaces. It has several modes, one of which can display statistics about the number of packets and datagrams sent and received. While there is no common API across unix systems to access this data, it is possible to extract this information from the text output of the program. Since the output varies slightly from system to system, it is necessary to take these variations into account when using that output.
Netstat examples using the Python Monitis SDK
The examples for this article are available at https://github.com/monitisexchange/Python-SDK/tree/master/examples. There are two examples. The first, monitis_netstat.py, can be used as is to create, list, update, and delete netstat monitors. The second, monitis_netstat_minimal.py, is a minimal implementation intended to illustrate the concepts for this article, but lacking the structure to handle command line arguments, different operating systems, etc.
Let’s take a look at monitis_netstat_minimal.py, line by line. It illustrates many of the core concepts in the Python SDK.
The following lines are imports from the Python standard library, used for calling netstat, searching its output, and comparing output from netstat with that retrieved via the SDK.
from subprocess import Popen, PIPE from re import findall from datetime import datetime
Next are the imports from the Monitis Python SDK itself, used for creating new monitors and fetching existing monitors.
from monitis.monitors.custom import CustomMonitor, get_monitors from monitis.monitors.params import ResultParams, DataType from monitis.api import Monitis
The Monitis.sandbox flag causes the SDK to use the Monitis developer’s sandbox, http://sandbox.monitis.com. By default, when this flag isn’t set, it uses the production environment, http://www.monitis.com.
Monitis.sandbox = True
The following block creates a new monitor. ResultParams encapsulates the format for the parameters that a monitor accepts. One or more of them can be passed in to the next line. CustomMonitor.addMonitor takes the parameters and creates a monitor based on them. It returns a new CustomMonitor object that represents the monitor in the Monitis API.
rp = ResultParams( 'tcp_packets_in','TCP Packets In','pkts/s', DataType('integer')) cm = CustomMonitor.add_monitor(rp, name='netstat monitor', tag='netstat') print "Created monitor: %s" % cm.monitor_id
This code extracts the interesting data from the netstat output. In this case, that data is the count of TCP packets received. This is the only block of code in the minimal sample program that really isn’t dependent on the API. A different monitor could get data in any number of ways. Note that this call to netstat is based on OS X style output, but only a small change to the command and matching string is required to adapt it to virtually any unix-like operating system.
subproc = Popen( 'netstat -s -p tcp | grep "packets received$"', shell=True, stdout=PIPE) (netstat_out, netstat_err) = subproc.communicate() count = findall('(\d+) packets received', netstat_out)
Next, add_result takes the new data, and posts it in the format defined earlier in ResultParams.
Here, we check that retrieving the results from the monitor gets back the same data that we put in.
today = datetime.utcnow() result = cm.get_monitor_results(today.year, today.month, today.day)
Finally, delete the monitor. This exists in this example so that it cleans up after itself and doesn’t cause unused custom monitors to pile up.
In typical usage, a monitor is only deleted when that data is no longer needed. This principle is explored further in monitis_netstat.py.
Under normal circumstances, a monitor is created once, data is updated on a periodic basis over time, and then that data is viewed in the Monitis web interface. To ilustrate this, monitis_netstat.py has command line flags to create, list, delete, and update a monitor.
Create a new netstat monitor:
$ examples/monitis_netstat.py -s -c netstat_mon Using name netstat_mon Created new netstat monitor
List the monitors to verify that ours was created:
$ examples/monitis_netstat.py -s -l 2267 netstat_mon
Run the monitor 10 times over five minutes to gather some data to display.
$ while (true) do examples/monitis_netstat.py -s -u 2267; sleep 30; done 1253222, 1616400, 381497, 551524 1253393, 1616514, 381505, 551536 1253436, 1616553, 381512, 551545 1253468, 1616580, 381514, 551549 1253485, 1616596, 381517, 551558 1253526, 1616633, 381522, 551609 1253564, 1616666, 381531, 551622 1253599, 1616695, 381536, 551629 1253623, 1616716, 381538, 551633 1253647, 1616740, 381544, 551638
$ examples/monitis_netstat.py -s -d 2267 Monitor 2267 deleted $ examples/monitis_netstat.py -s -l
To get the code and try this yourself, you can clone a copy from GitHub, install the Monitis SDK module, and run it on your own system.
git clone git://github.com/monitisexchange/Python-SDK.git cd Python-SDK sudo python setup.py install
Monitis API Keys
Before you can run the tools, you will also need to set up the Monitis API Key and Secret Key. By default, these are read from the following environment variables, which you can set on the command line, or add to your .bashrc or other login command file. The values for these keys can be retrieved via the web user interface under Tools -> API -> API Key.
# production keys export MONITIS_APIKEY='API KEY' export MONITIS_SECRETKEY='SECRET KEY' # sandbox keys export MONITIS_SANDBOX_APIKEY='SANDBOX API KEY' export MONITIS_SANDBOX_SECRETKEY='SANDBOX SECRET KEY'
While the Monitis REST API makes creating custom monitors simple in almost any language, it becomes truly easy with an SDK to handle the REST interactions. This article has illustrated just how effortless it can be to collect important information from your systems and record it in Monitis custom monitors.
Signup and use free monitis sandbox here – http://sandbox.monitis.com.
See also Python Performance Tips, Part 1