[clug] Measuring Bandwidth usage by Application Protocol

[Steve McInerney]

On Thu, 2009-11-05 at 09:07 +1100, jm wrote:
> > I've used it to get summary info and then pull that into R for detailed
> > statistical analysis. 
> >   
> 
> This is what I'm trying to do. Get raw numbers to drag into R where I 
> can manipulate/play with them. So far I'm only been able to do this with 
> numbers exported from cacti and was hoping for something a little more 
> detailed so as to investigate individual users, protocol, autonomous 
> systems, etc. Can you comment further on how your using R or can you 
> point to any useful information on networks traffic and analysis with R.


Sure. Sorry for the delay, life intruded as it so often does.

Keeping in mind it's been about 2 years since I've last done this in
anger, so I haven't got any notes on the detail unf.


The general gist is reduce the data down to something "interesting" and
fiddle from there. Export to a format that R can deal with; import;
display as a percentile graph - I've found for this stuff percentiles
FTW - emphatically over avg etc.

Use various filters to exclude noise. eg within argus, awk! or R itself.

http://www.stedee.id.au/2007/09-29/advanced_sysadmin_troubleshooting_slow_websites

is a very very high level "howto" of ... how to. :-)

http://www.stedee.id.au/Apache_WebSite_Responsiveness

is a more detailed one of futzing around with apache logs to extract
info.


Argus can enable you to dump raw columns of data; that can in turn be
imported into R and analysed in some way shape or form. But you'll, in
general, need to apply some level of filtering - ala tcpdump style
expressions - first.

Sorry this is a tad light on detail. But tbh, I just don't recall
them :-)


Cheers!
- Steve