I recommend Splunk a lot. I’ve been using it since about 2007, in varying degrees of complexity.
Updated Nov 2013 to use Ansible to provision.
As a log consolidation and data mining tool Splunk is great. But I’ll go for long periods where I don’t get to use it, so my ‘demo’ VMs get killed off. With my discovery of Vagrant, I decided to take my default CentOS 6 VM and put some Config Management to work to do an install of Splunk.
This repository on Github is the result. You can simply update the RPM name in playbook.yaml as version numbers increase, so it doesn’t take rolling a whole new Vagrant box.
I can now very rapidly pop up a demo Splunk instance if I want to evangelise it some more to customers. I might even take this as the root to building a multi-machine demo.