The Squid proxy server is one of the most famous proxy servers in the world. This software is a ‘must have’ in every network administrator’s tool bag. Squid is being used for web content caching, web access control, as a reverse proxy – anywhere the goal is productivity and easy control. It has very useful features, is welldocumented, has a powerful access control list mechanism, and, most important for us, supports various means of monitoring.
Before turning to monitoring, let’s discuss an example to understand how Squid can help us to organize a working day in the office and accelerate internet connection speed.
Let’s assume there is a Squid based transparent web cache which provides access to the internet for 2 offices.
Access to inappropriate sites is denied during working hours.
То prevent circumvention of this rule, access to some anonymous proxies will be blocked as well.
So, let start!
In case you have not installed Squid, install it:
apt-get install squid
Install the Squid client as well, as we will need it for monitoring and testing purposes:
apt-get install squidclient
Now configure Squid:
cd /etc/squid cp squid.conf squid.conf.original vi squid.conf # add the lines # # restricted url’s list such as some social networks, adult sites # file must contain one domain per line acl restricted_urls url_regex -i "/etc/squid/restricted_urls" # anonymous proxies, web anonymizers and so on # file must contain one domain per line (example: domain. ) acl anonymous_list url_regex -i “/etc/squid/anon_proxy.list” # working hours of Monitis’ office (just an example) acl WorkingHours_Monitis time D 09:00-18:00 # working hours of Netangels’ office (just an example) acl WorkingHours_Netangels time D 10:00-19:00 # Netangel’s local network range (just an example) acl Netangels_Net src 192.168.10.0/24 # Monitis’ local network range (just an example) acl Monitis_Net src 192.168.20.0/24 acl Sec_Perimeter src Netangels_Net Monitis_Net # deny access for some resources in working hours http_access allow WorkingHours_Monitis !restricted_urls http_access allow WorkingHours_Netangels !restricted_urls # deny access to anonymizers http_access deny anonymous_list # preventing cache hits for local resources acl NetangelsLocal dstdomain .netangels.net acl MonitisLocal dstdomain .monitis.com # deny caching for nearby servers, because they don't benefit too # much from cache hits and for saving storage no_cache deny NetangelsLocal no_cache deny MonitisLocal # # deny access for unwanted networks http_access allow Sec_Perimeter # http_access deny all # restart the squid /etc/init.d/squid restart
And now it’s time to manage the monitoring.
Usually, people use snmp based tools and/or a log analyzer/parser to track various Squid metrics, but that is not convenient for us.
We need a simpler but equally effective solution, and that is M3.
squidclient -h localhost cache_object://localhost/ mgr:utilization
Last 5 minutes:
sample_start_time = 1342080662.846829 (Thu, 12 Jul 2012 08:11:02 GMT) sample_end_time = 1342080962.851335 (Thu, 12 Jul 2012 08:16:02 GMT) client_http.requests = 0.006667/sec client_http.hits = 0.000000/sec client_http.errors = 0.000000/sec client_http.kbytes_in = 0.003333/sec client_http.kbytes_out = 0.016666/sec client_http.all_median_svc_time = 0.000000 seconds client_http.miss_median_svc_time = 0.000000 seconds client_http.nm_median_svc_time = 0.000000 seconds client_http.nh_median_svc_time = 0.000000 seconds client_http.hit_median_svc_time = 0.000000 seconds server.all.requests = 0.000000/sec server.all.errors = 0.000000/sec server.all.kbytes_in = 0.000000/sec
squidclient -h localhost cache_object://localhost/ mgr:info <Headers missed> Squid Object Cache: Version 2.7.STABLE9 Start Time: Thu, 12 Jul 2012 08:04:02 GMT Current Time: Thu, 12 Jul 2012 09:21:16 GMT Connection information for squid: Number of clients accessing cache: 2 Number of HTTP requests received: 31 Number of ICP messages received: 0 Number of ICP messages sent: 0 Number of queued ICP replies: 0 Number of HTCP messages received: 0 Number of HTCP messages sent: 0 Request failure ratio: 0.00 Average HTTP requests per minute since start: 0.4 Average ICP messages per minute since start: 0.0 Select loop called: 11729 times, 395.079 ms avg Cache information for squid: Request Hit Ratios: 5min: 0.0%, 60min: 0.0% Byte Hit Ratios: 5min: 0.2%, 60min: 0.4% Request Memory Hit Ratios: 5min: 0.0%, 60min: 0.0% Request Disk Hit Ratios: 5min: 0.0%, 60min: 0.0% Storage Swap size: 90344 KB Storage Mem size: 124 KB Mean Object Size: 55.80 KB Requests given to unlinkd: 0 Median Service Times (seconds) 5 min 60 min: HTTP Requests (All): 0.01556 0.01556 Cache Misses: 0.01556 0.01556 Cache Hits: 0.00000 0.00000 Near Hits: 0.00000 0.00000 Not-Modified Replies: 0.00000 0.00000 DNS Lookups: 0.00000 0.06364 ICP Queries: 0.00000 0.00000 Resource usage for squid: UP Time: 4633.878 seconds CPU Time: 2.532 seconds CPU Usage: 0.05% CPU Usage, 5 minute avg: 0.07% CPU Usage, 60 minute avg: 0.05% Process Data Segment Size via sbrk(): 2484 KB Maximum Resident Size: 21216 KB Page faults with physical i/o: 0 Memory usage for squid via mallinfo(): Total space in arena: 2484 KB Ordinary blocks: 2360 KB 3 blks Small blocks: 0 KB 0 blks Holding blocks: 280 KB 1 blks Free Small blocks: 0 KB ….......................................
As you can see, there is much to gather and/or analyze, and the fact that I use the command
squidclient -h localhost cache_object://localhost/counters
in my M3 template does not mean these metrics are more important or useful, it’s just a choice dictated by my needs.
It’s now time to configure access rules for the cache_manager protocol.
vi /etc/squid/squid.conf # add/modify the lines acl manager proto cache_object http_access allow manager Sec_Perimeter # just an example # restart the squid /etc/init.d/squid restart
ServerKbOut – The amount of traffic written to origin servers and/or neighbor caches for server-side.
ServerKbIn – The amount of traffic (in kilobytes) read from the server-side for all protocols.
ClientKbIn – The amount of traffic (in kilobytes) received from clients in their requests.
ClientKbOut – The amount of traffic (in kilobytes) sent to clients in responses. Also measured at the HTTP layer.
ServerRequests – The number of server-side requests to HTTP servers, including neighbor caches.
ServerErrors – The number of Gopher, WAIS, and SSL requests that resulted in an error.
ClientRequest – The number of HTTP requests received from clients.
Page Faults – The number of (major) page faults as reported by getrusage().
ClientErrors – The number of client transactions that resulted in an error.
AbortedRequests – The number of server-side HTTP requests aborted due to client-side aborts.
Now check your dashboard:
More data in tabular view below
Some metrics examples in graphical view below.
As you can see, monitoring with Monitis M3 is very simple and powerful. The described means of monitoring is already tested and is being used on my servers.