flume配置如下：
[demoe3base@kf-app1conf]$catflume-conf.conf
#Finally,nowthatwe'vedefinedallofourcomponents,tell
#agent1whichoneswewanttoactivate.
agent1.channels=ch1
agent1.sources=source1
agent1.sinks=hdfssink1
#Defineamemorychannelcalledch1onagent1
agent1.channels.ch1.type=memory
agent1.channels.ch1.capacity=100000
agent1.channels.ch1.transactionCapacity=100000
agent1.channels.ch1.keep-alive=30
#DefineanAvrosourcecalledavro-source1onagent1andtellit
#tobindto0.0.0.0:41414.Connectittochannelch1.
agent1.sources.source1.channels=ch1
agent1.sources.source1.type=avro
agent1.sources.source1.bind=172.21.3.60
agent1.sources.source1.port=44444
agent1.sources.source1.threads=5
#Definealoggersinkthatsimplylogsalleventsitreceives
#andconnectittotheotherendofthesamechannel.
agent1.sinks.hdfssink1.channel=ch1
agent1.sinks.hdfssink1.type=hdfs
agent1.sinks.hdfssink1.hdfs.path=hdfs://kf-app1:8020/flume
agent1.sinks.hdfssink1.hdfs.writeFormat=Text
agent1.sinks.hdfssink1.hdfs.fileType=DataStream
agent1.sinks.hdfssink1.hdfs.rollInterval=0
agent1.sinks.hdfssink1.hdfs.rollSize=60554432
agent1.sinks.hdfssink1.hdfs.rollCount=0
agent1.sinks.hdfssink1.hdfs.batchSize=1000
agent1.sinks.hdfssink1.hdfs.txnEventMax=1000
agent1.sinks.hdfssink1.hdfs.callTimeout=60000
agent1.sinks.hdfssink1.hdfs.appendTimeout=60000
用命令启动：bin/flume-ngagent--conf./conf/-fconf/flume-conf.conf-nagent1
一切正常，而且flume.log日志也正常。
用bin/flume-ngavro-client-Hkf-app1-p44444-F/chunk1/demo/flume/test2.txt发送文件，flume.log如下：
[demoe3base@kf-app1logs]$tail-fflume.log
08五月201314:34:31,370INFO[lifecycleSupervisor-1-3](org.apache.flume.instrumentation.MonitoredCounterGroup.start:82)-Componenttype:SOURCE,name:source1started
08五月201314:34:31,370INFO[lifecycleSupervisor-1-3](org.apache.flume.source.AvroSource.start:155)-Avrosourcesource1started.
08五月201314:34:45,932INFO[pool-6-thread-1](org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.handleUpstream:171)-[id:0x34bf1d3b,/172.21.3.61:39262=>/172.21.3.60:44444]OPEN
08五月201314:34:45,938INFO[pool-7-thread-1](org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.handleUpstream:171)-[id:0x34bf1d3b,/172.21.3.61:39262=>/172.21.3.60:44444]BOUND:/172.21.3.60:44444
08五月201314:34:45,938INFO[pool-7-thread-1](org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.handleUpstream:171)-[id:0x34bf1d3b,/172.21.3.61:39262=>/172.21.3.60:44444]CONNECTED:/172.21.3.61:39262
08五月201314:34:46,267INFO[pool-7-thread-1](org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.handleUpstream:171)-[id:0x34bf1d3b,/172.21.3.61:39262:>/172.21.3.60:44444]DISCONNECTED
08五月201314:34:46,267INFO[pool-7-thread-1](org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.handleUpstream:171)-[id:0x34bf1d3b,/172.21.3.61:39262:>/172.21.3.60:44444]UNBOUND
08五月201314:34:46,268INFO[pool-7-thread-1](org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.handleUpstream:171)-[id:0x34bf1d3b,/172.21.3.61:39262:>/172.21.3.60:44444]CLOSED
08五月201314:34:46,268INFO[pool-7-thread-1](org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.channelClosed:209)-Connectionto/172.21.3.61:39262disconnected.
08五月201314:34:46,922INFO[hdfs-hdfssink1-call-runner-0](org.apache.flume.sink.hdfs.BucketWriter.doOpen:189)-Creatinghdfs://kf-app1:8020//FlumeData.1367994886244.tmp
问题来了：1、为什么是“FlumeData.1367994886244.tmp”临时文件，而不能将文件关闭呢？当我把代理强行kill掉或者关掉后，日志才打印出“08五月201314:21:17,556INFO[hdfs-hdfssink1-call-runner-5](org.apache.flume.sink.hdfs.BucketWriter.renameBucket:379)-Renaminghdfs://kf-app1:8020/flume/FlumeData.1367993804350.tmptohdfs://kf-app1:8020/flume/FlumeData.1367993804350”，难道说代理不能够自动关闭？
2、而且在发送第二个文件后发现日志报错UNBOUND,难道说一个通道直接接收一个文件？
08五月201314:30:47,202INFO[pool-6-thread-1](org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.handleUpstream:171)-[id:0x5a9b8ff9,/172.21.3.61:38652=>/172.21.3.60:44444]OPEN
08五月201314:30:47,203INFO[pool-7-thread-1](org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.handleUpstream:171)-[id:0x5a9b8ff9,/172.21.3.61:38652=>/172.21.3.60:44444]BOUND:/172.21.3.60:44444
08五月201314:30:47,203INFO[pool-7-thread-1](org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.handleUpstream:171)-[id:0x5a9b8ff9,/172.21.3.61:38652=>/172.21.3.60:44444]CONNECTED:/172.21.3.61:38652
08五月201314:30:47,913INFO[pool-7-thread-1](org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.handleUpstream:171)-[id:0x5a9b8ff9,/172.21.3.61:38652:>/172.21.3.60:44444]DISCONNECTED
08五月201314:30:47,913INFO[pool-7-thread-1](org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.handleUpstream:171)-[id:0x5a9b8ff9,/172.21.3.61:38652:>/172.21.3.60:44444]UNBOUND
08五月201314:30:47,913INFO[pool-7-thread-1](org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.handleUpstream:171)-[id:0x5a9b8ff9,/172.21.3.61:38652:>/172.21.3.60:44444]CLOSED
08五月201314:30:47,914INFO[pool-7-thread-1](org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.channelClosed:209)-Connectionto/172.21.3.61:38652disconnected.
以上两个问题还请明白的或者遇到过的给予指点呀。
分 -->
|
