Overview

docs
tests
package

the ultimate tailer

• Free software: BSD license

Installation

pip install tailchaser

Usage

::

$ tailchaser /wher/my/logs/*.log

$ tailchaser -h

usage: tailer [options] source_pattern

the ultimate tail chaser

positional arguments:

source_pattern source pattern is the glob path to a file to be tailed

plus its rotated versions

optional arguments:

-h, --help	show this help message and exit
--verbose	prints a lot crap, default is: False
--dryrun	prints a lot crap and no hand-off, default is: False
--dont_backfill
	don’t backfill with rolled logs, default is: False
--dont_follow	don’t follow when you reach the end of the file exit, default is: False
--clear_checkpoint
	clears the checkpoint and lets you start from the begining, default:False
--read_period READ_PERIOD
	time given to read, default: 1.0
--read_pause READ_PAUSE
	time to pause between reads, default: 0

To use tailchaser in a project:

#
# Example 1 - Tail to Elastic
#

import requests

import tailchaser

class TailToElastic(tailchaser.Tailer):
    def handoff(self, file_tailed, checkpoint, record):
        """ Expect a record like:

        20160204 10:28:15,525 INFO PropertiesLoaderSupport - Loading properties file from URL [file:C:/WaterWorks/Broken/BSE//config/lme-market.properties]
        20160204 10:28:15,541 INFO PropertiesLoaderSupport - Loading properties file from URL [file:C:/WaterWorks/Broken/BSE//config/default-database.properties]
        20160204 10:28:15,541 INFO PropertiesLoaderSupport - Loading properties file from URL [file:C:/WaterWorks/Broken/BSE//config/default-hibernate.properties]
        """

        date, time, level, source, _, message = record.split(5)
        requests.json("http://someelacticserver.com:9200/myindex/log", json={
                        'timestamp': '{}T{}'.format(date, time)
                        'level': level,
                        'source': source,
                        'message': message
                        })




#
# Example 2 - Tail to Kafka - shows how to add your own arguments and then send messahes to kafka.
#


import msgpack
import tailchaser
from kafka import KafkaProducer

class TailToKafka(tailchaser.Tailer):
    def add_arguments(cls, parser=None):
        parser = super(TailToKafka, cls).add_arguments(parser)

    HOSTS = 'localhost:1234'
    TOPIC = 'log'
    def startup(self):
        self.kafka_producer = KafkaProducer(bootstrap_servers=self.HOSTS,value_serializer=msgpack.dumps)


    def handoff(self, file_tailed, checkpoint, record):
        """ Expect a record like:

        20160204 10:28:15,525 INFO PropertiesLoaderSupport - Loading properties file from URL [file:C:/WaterWorks/Broken/BSE//config/lme-market.properties]
        20160204 10:28:15,541 INFO PropertiesLoaderSupport - Loading properties file from URL [file:C:/WaterWorks/Broken/BSE//config/default-database.properties]
        20160204 10:28:15,541 INFO PropertiesLoaderSupport - Loading properties file from URL [file:C:/WaterWorks/Broken/BSE//config/default-hibernate.properties]
        """
        self.kafka_producer.send(self.TOPIC, message)

Documentation

https://tailchaser.readthedocs.org/

Development

To run the all tests run:

tox

Note, to combine the coverage data from all the tox environments run:

Windows	set PYTEST_ADDOPTS=--cov-append tox
Other	PYTEST_ADDOPTS=--cov-append tox