AANTS - scraper, mactrapper, macwatch, and other NetWatch Foundation Tools

This document attempts to demystify the AANTS physaddrwatch, mactrapper and catwatch tools, which work behind the scenes to populate the AANTS NetWatch database.

Scraping of ARP and ND tables

Historically, we had a set of scripts that pulled the arp and neighbor discovery tables (and ipx, and appletalk, for that matter) off of campus routers via SNMP. Due to scaling issues with snmp, in particular that the protocol requires lexicographical ordering, as the campus network grew in became increasingly infeasible to use SNMP and an approach to use screen scraping of ssh was employed. For some period of time there were tools such as physaddrwatch, physaddrwatch_v6, banphysaddrwatch, and such. These tools have all now been migrated to using the scraper framework which shells out to each router to collect this data. Accordingly there are a set of "processor" scripts that read in this data from the file and perform the appropriate inserts into the database tables used by netwatch.

MacTrapper

Description:
mactrapper is a utility that handles MAC change notification traps from Cisco switches. It is the passive companion to the macwatch utility, which does occasional active polling of devices. The information contained in the header of the utility itself includes this message on active vs passive MAC learning:

Note that it is imperative that you use this script in combination with one that 
polls each switch to capture the mac-address-table periodically, such as 
catwatch: http://net.doit.wisc.edu/~plonka/catwatch/
catwatch's "-r" option is meant for use in combination with mactrapper.

If you use only mactrapper, you may miss the location of some LAN 
stations, such as those that send traffic continuously and therefore rarely 
need to be relearned by the switch. MAC learning traps are only sent when 
the switch (re)learns or removes a MAC from its mac-learning table.

Also, trap delivery is unreliable, so its good to combine it with periodic 
polling to increase your confidence that you're collecting all the MAC 
information available.

See the mactrapper utility file for more information on configuring ports to send/receive MAC traps.

mactrapper now lives at /usr/local/ns/bin/mactrapper. The daemon runs on Peleus.

/bin/ksh -c while (( 1 ));  
do  /bin/rm -f /var/local/watch/mactrapper/mactrapper.log.old;  
test -f /var/local/watch/mactrapper/mactrapper.log && 
/bin/mv /var/local/watch/mactrapper/mactrapper.log /var/local/watch/mactrapper/mactrapper.log.old;  
/usr/local/bin/locker -e/var/local/watch/mactrapper/mactrapper.log /usr/local/bin/mactrapper -D 
-c'itsnotpublic' -s'DBI:mysql:net\;mysql_read_default_file=/home/net/.my.cnf' >
/var/local/watch/mactrapper/mactrapper.log 2>&1

Meaning: "Every minute do the following steps - Remove the file 'mactrapper.log.old', ignoring if it does not exist. If the file 'mactrapper.log' exists and is a regular file, move it to 'mactrapper.log.old' and run the mactrapper utility using 'mactrapper.log' as the output file and using the -D option (use 'ifDescr' rather than 'ifName' when learning device names). Connect to the AANTS (net) database using the net user's MySQL config file.

To start/restart/stop the mactrapper daemon, treat it as a standard SYSV init script at /usr/local/ns/etc/init.d/mactrapper.

macwatch

Description: macwatch is a utility that does occasional active polling of s*-access and s*-mgmt devices using clogin. The utility is run from the net crontab as follows:

# Run Charles's new clogin-based version of xxicatwatch, macwatch
7,27,47 * * * *        /usr/local/ns/bin/locker -ne $HOME/.macwatch.lock /usr/local/ns/bin/macwatch_master.pl > /var/local/log/scripts/macwatch_master-cron.log

and lives in the /usr/local/ns/bin/ directory. There are a few important parts:

macwatch_master.pl creates multiple processes (15 at current writing), then runs the individual macwatch utilities in these processes and collects the results for logging.

macwatch.pl does the actual device polling using clogin and the command:

/usr/local/ns/bin/clogin2 -c'show mac-address-table' -t5 -f${login_file} $device_name

Entries in the database are updated with the results.

/var/local/log/scripts/macwatch.log is the log for the macwatch tool, and a good place to start troubleshooting.

If the macwatch tool takes longer than normal to run, which usually indicates a processor overload on Peleus), an email is sent to the AANTS admin by a tool called macwatch_monitor.pl.

XXICatWatch

Description:
The xxicatwatch utility has been retired and its functions were taken over by the macwatch tool. See above.

importAirwave

Description: importAirwave is a perl script that imports mac addresses from airwave. The utility is run from the net crontab as follows:

1,6,11,16,21,26,31,36,41,46,51,56 * * * * /usr/local/ns/bin/locker -ne $HOME/.airwave_import.lock /usr/local/ns/bin/importAirwave.pl >/dev/null

and lives in the /usr/local/ns/bin/ directory.

trimWirelessFromMacswitch retains the last 1 day of mac addresses and the most recent place a mac address was seen.

35 0 * * * /usr/local/ns/bin/locker -ne $HOME/.trimWirelessFromMacswitch.lock /usr/local/ns/bin/trimWirelessFromMacswitch.pl >/dev/null