Sudden changes in any of the major system resources, such as available memory, CPU, or storage for your Hadoop cluster may indicate a possible security issue. Hadoop provides Metrics for monitoring these resources and automated scripts can be developed to output specific metrics to files and add them as partitions for appropriate Hive tables. In this talk, I will demonstrate how this historical data (stored as Hive tables) can be used for generating security alerts by defining variation thresholds. These alerts can be configured to perform specific actions (such as emailing the sysadmin etc.).
