{"id":1815,"date":"2015-01-21T23:15:57","date_gmt":"2015-01-21T15:15:57","guid":{"rendered":"http:\/\/ixyzero.com\/blog\/?p=1815"},"modified":"2015-01-21T23:15:57","modified_gmt":"2015-01-21T15:15:57","slug":"kafka%e4%bd%bf%e7%94%a8%e7%9a%84%e4%b8%80%e4%ba%9btips%e5%b0%8f%e7%bb%93","status":"publish","type":"post","link":"https:\/\/ixyzero.com\/blog\/archives\/1815.html","title":{"rendered":"Kafka\u4f7f\u7528\u7684\u4e00\u4e9btips\u5c0f\u7ed3"},"content":{"rendered":"<p>\u5728\u4f7f\u7528Kafka\u7684\u8fc7\u7a0b\u4e2d\uff0c\u6700\u9ebb\u70e6\u7684\u5730\u65b9\u83ab\u8fc7\u4e8eConsumer\u4e86\uff0c\u5982\u679c\u7528High Level API\u7684\u8bdd\uff0c\u5bf9partition\u3001offset\u7b49\u90fd\u65e0\u6cd5\u8fdb\u884c\u63a7\u5236\uff1b\u5982\u679c\u7528Low Level API\u7684\u8bdd\uff0c\u53c8\u8fc7\u4e8e\u590d\u6742\uff0c\u5934\u75bc\u2026\u2026<\/p>\n<p>\u8fd9\u91cc\u8bb0\u5f55\u4e00\u4e0b\u5728\u6d4b\u8bd5\u8fc7\u7a0b\u4e2d\uff08\u4e3b\u8981\u9488\u5bf9Consumer\u7684offset\uff09\u78b0\u5230\u7684\u5de5\u5177\u4ee5\u53ca\u4f7f\u7528\u65b9\u6cd5\uff1a<\/p>\n<h6>\u5229\u7528Kafka\u81ea\u8eab\u63d0\u4f9b\u7684\u7cfb\u7edf\u5de5\u5177<\/h6>\n<p><a href=\"https:\/\/cwiki.apache.org\/confluence\/display\/KAFKA\/System+Tools\" target=\"_blank\">https:\/\/cwiki.apache.org\/confluence\/display\/KAFKA\/System+Tools<\/a><\/p>\n<h6>\u6e90\u7801\u5730\u5740<\/h6>\n<p><a href=\"https:\/\/github.com\/apache\/kafka\/tree\/0.8\/core\/src\/main\/scala\/kafka\/tools\" target=\"_blank\">https:\/\/github.com\/apache\/kafka\/tree\/0.8\/core\/src\/main\/scala\/kafka\/tools<\/a><\/p>\n<hr \/>\n<h6>Consumer\u7684offset\u68c0\u67e5(\u6839\u636egroup\u540d)<\/h6>\n<pre class=\"lang:default decode:true\">&gt; bin\/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker --group consumer_group_name<\/pre>\n<h6>\u83b7\u53d6\u67d0\u4e00topic\u7684offset<\/h6>\n<pre class=\"lang:default decode:true\">&gt; bin\/kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list localhost:9092 --topic test --time -1\n\n&gt; bin\/kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list localhost:9092 --topic test --time -2<\/pre>\n<h6>\u67e5\u770bZooKeeper\u4e2d\u7684offset\u60c5\u51b5<\/h6>\n<pre class=\"lang:default decode:true\">&gt; bin\/kafka-run-class.sh kafka.tools.ExportZkOffsets --zkconnect localhost:2181 --output-file abc.txt<\/pre>\n<h6>\u91cd\u653e\u67d0\u4e00topic\u7684\u5185\u5bb9\u81f3\u53e6\u4e00topic<\/h6>\n<pre class=\"lang:default decode:true\">&gt; bin\/kafka-run-class.sh kafka.tools.ReplayLogProducer --broker-list localhost:9092 --zookeeper localhost:2181 --inputtopic test --outputtopic new_test<\/pre>\n<h6>\u5728\u63a7\u5236\u53f0\u6253\u5370\u5df2\u6d88\u8d39\u7684messages<\/h6>\n<pre class=\"lang:default decode:true\">&gt; bin\/kafka-run-class.sh kafka.tools.SimpleConsumerShell --broker-list localhost:9092 --topic test<\/pre>\n<h6>\u5728\u63a7\u5236\u53f0\u7ed9\u5bf9\u5e94\u7684topic\u751f\u6210messages<\/h6>\n<pre class=\"lang:default decode:true \">&gt; bin\/kafka-run-class.sh kafka.producer.ConsoleProducer --broker-list localhost:9092 --topic test<\/pre>\n<h6>\u5728\u63a7\u5236\u53f0\u6253\u5370[\u5df2]\u6d88\u8d39\u7684messages<\/h6>\n<pre class=\"lang:default decode:true \">&gt; bin\/kafka-run-class.sh kafka.consumer.ConsoleConsumer --zookeeper localhost:2181 --topic test\n\n&gt; bin\/kafka-run-class.sh kafka.consumer.ConsoleConsumer --zookeeper localhost:2181 --topic test --from-beginning<\/pre>\n<h6>\u67e5\u770bZooKeeper\u4e0a\u6709\u54ea\u4e9btopic\u4ee5\u53catopic\u7684\u8be6\u7ec6\u4fe1\u606f<\/h6>\n<pre class=\"lang:default decode:true\">&gt; bin\/kafka-run-class.sh kafka.admin.TopicCommand --zookeeper localhost:2181 --list\n\n&gt; bin\/kafka-run-class.sh kafka.admin.TopicCommand --zookeeper localhost:2181 --describe --topic test<\/pre>\n<h6>\u53c2\u8003\u94fe\u63a5\uff1a<\/h6>\n<ul>\n<li><a href=\"http:\/\/kafka.apache.org\/documentation.html#quickstart\" target=\"_blank\">http:\/\/kafka.apache.org\/documentation.html#quickstart<\/a><\/li>\n<li><a href=\"https:\/\/kafka.apache.org\/08\/api.html\" target=\"_blank\">https:\/\/kafka.apache.org\/08\/api.html<\/a><\/li>\n<li><a href=\"http:\/\/wenku.baidu.com\/link?url=MFIN4K8UCJPGHtpNFfqPyaGZhAMV9S682RkK3Eqakk4tw-oSwf7rEAccmAmXI899cbpryERBYXNIh3Y2jcZ6YbrMJK2GI0L8Dk4Boc2LXGy\" target=\"_blank\">Kafka\u96c6\u7fa4\u6587\u6863<\/a><\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<h5>Kafka\u7684offset\u76d1\u63a7<\/h5>\n<ul>\n<li><a href=\"http:\/\/blog.csdn.net\/lizhitao\/article\/details\/27199863\" target=\"_blank\">http:\/\/blog.csdn.net\/lizhitao\/article\/details\/27199863<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/quantifind\/KafkaOffsetMonitor\" target=\"_blank\">https:\/\/github.com\/quantifind\/KafkaOffsetMonitor<\/a><\/li>\n<\/ul>\n<p>\u56e0\u4e3aKafkaOffsetMonitor\u4e2d\u6709\u4e9b\u8d44\u6e90\u6587\u4ef6(css,js)\u662f\u8bbf\u95ee\u5916\u7f51\u7684\uff0c\u7279\u522b\u662f\u6709\u8bbf\u95eegoogle\u8d44\u6e90\uff0c\u7ecf\u5e38\u4e0d\u80fd\u8bbf\u95eeo(\u256f\u25a1\u2570)o<\/p>\n<p>&nbsp;<\/p>\n<h5>Kafka\u7684\u53ef\u9760\u6027<\/h5>\n<ul>\n<li><a href=\"http:\/\/www.cnblogs.com\/fxjwind\/p\/3810740.html\" target=\"_blank\">http:\/\/www.cnblogs.com\/fxjwind\/p\/3810740.html<\/a><\/li>\n<li><a href=\"http:\/\/stackoverflow.com\/questions\/24661533\/restarting-a-kafka-python-consumer-consumes-all-the-messages-in-the-queue-agai\" target=\"_blank\">http:\/\/stackoverflow.com\/questions\/24661533\/restarting-a-kafka-python-consumer-consumes-all-the-messages-in-the-queue-agai<\/a><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>\u5728\u4f7f\u7528Kafka\u7684\u8fc7\u7a0b\u4e2d\uff0c\u6700\u9ebb\u70e6\u7684\u5730\u65b9\u83ab\u8fc7\u4e8eConsumer\u4e86\uff0c\u5982\u679c\u7528High Level API\u7684\u8bdd\uff0c\u5bf9pa [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[23,6,12],"tags":[10,438],"class_list":["post-1815","post","type-post","status-publish","format-standard","hentry","category-knowledgebase-2","category-other","category-tools","tag-kafka","tag-offset"],"views":4002,"_links":{"self":[{"href":"https:\/\/ixyzero.com\/blog\/wp-json\/wp\/v2\/posts\/1815","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ixyzero.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ixyzero.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ixyzero.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/ixyzero.com\/blog\/wp-json\/wp\/v2\/comments?post=1815"}],"version-history":[{"count":0,"href":"https:\/\/ixyzero.com\/blog\/wp-json\/wp\/v2\/posts\/1815\/revisions"}],"wp:attachment":[{"href":"https:\/\/ixyzero.com\/blog\/wp-json\/wp\/v2\/media?parent=1815"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ixyzero.com\/blog\/wp-json\/wp\/v2\/categories?post=1815"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ixyzero.com\/blog\/wp-json\/wp\/v2\/tags?post=1815"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}