Skip to content

Commit 937b45f

Browse files
author
Sudev Ambadi
committed
changes in readme
1 parent b30846e commit 937b45f

File tree

1 file changed

+20
-5
lines changed

1 file changed

+20
-5
lines changed

README.md

+20-5
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,29 @@
1-
# python-kafka-logger
1+
# pykafka-async-logger
22

3-
A simple handler for python logger to push logs into Kafka instead of writting them to disk.
3+
A simple handler for python logger to push logs into Kafka instead of writting them to disk.
44

5-
The logging handler will write to Kafka in async batch using [pykafka](https://github.com/Parsely/pykafka). Current implementation works well, but is not fault tolerant if kafka fails (Server not available, Leader not available ..) ==> no logs.
5+
The logging handler will write to Kafka in async batch using [pykafka](https://github.com/Parsely/pykafka). Current implementation works well, but is not fault tolerant if kafka fails (Server not available, Leader not available) results in error/application crash.
66

77
### Todo:
88

99
* Logger fails when Kafka is down, need to tweak code such that Handler can be configured in runtime (If runtime change of handler is available!) else tweak to emit logs into file instead of pushing into Kafka.
10-
* Write test case where Kafka client connection is already established and then fails(Leader not available for partition and other Kafka errors)
10+
* Write test cases and fix issues where Kafka client connection is already established and then fails(Leader not available for partition and other Kafka errors)
1111
* Introduce snappy compression of messages.
1212
* Configure pykafka to use [librdkafka](https://github.com/edenhill/librdkafka), this will effectively better performance a lot, especially while using compression.
1313

14-
Inspired from https://github.com/taykey/python-kafka-logging
14+
Inspired from https://github.com/taykey/python-kafka-logging, python-kafka async implementation is here in this [branch](https://github.com/sudev/pykafka-async-logger/tree/python-kafka)
15+
16+
### Get the project
17+
18+
1. Clone the git repository
19+
```shell
20+
git clone https://github.com/taykey/python-kafka-logging/
21+
```
22+
1. Create a new virtualenv
23+
```shell
24+
virtualenv python_kafka_logging
25+
```
26+
1. Install project's requirements
27+
```shell
28+
python_kafka_logging/bin/pip install -r requirements.txt
29+
```

0 commit comments

Comments
 (0)