做个了JAVAclient链接hbase报dfs.nameservices绑定无法解析的问题
hosts:
134.96.169.138hdpnn01
134.96.169.139hdpnn02
134.96.169.140hdpdn01
134.96.169.142hdpdn03
134.96.169.143hdpdn04
134.96.169.144hdpdn05
134.96.169.145hdpdn06
其中:
namenodehdpnn01,hdpnn02
journalnodehdpnn01,hdpnn02,hdpdn01
datanode:hdpdn01hdpdn03hdpdn04hdpdn05hdpdn06
hdfs-site.xml:
dfs.nameservices
dpicluster
dfs.namenode.checkpoint.edits.dir
${dfs.namenode.checkpoint.dir}
dfs.datanode.data.dir.perm
750
dfs.client.read.shortcircuit
true
dfs.journalnode.http-address
0.0.0.0:8480
dfs.client.failover.proxy.provider.dpicluster
org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider
dfs.namenode.https-address
hdpnn01:50470
dfs.namenode.http-address.dpicluster.nn2
hdpnn02:50070
dfs.ha.namenodes.dpicluster
nn1,nn2
dfs.namenode.http-address.dpicluster.nn1
hdpnn01:50070
ha.zookeeper.quorum
hdpdn01:2181,hdpnn01:2181,hdpnn02:2181
fs.defaultFS
hdfs://dpicluster
hadoop.proxyuser.hcat.hosts
hdpnn02
hadoop.proxyuser.hive.hosts
hdpnn02
hbase-site:
hbase.zookeeper.property.clientPort
2181
hbase.zookeeper.useMulti
true
hbase.defaults.for.version.skip
true
hbase.regionserver.global.memstore.lowerLimit
0.38
hbase.hstore.compactionThreshold
3
hbase.security.authorization
false
hbase.regionserver.global.memstore.upperLimit
0.4
hbase.hstore.flush.retries.number
120
hbase.superuser
hbase
hbase.hregion.memstore.mslab.enabled
true
zookeeper.session.timeout
30000
hbase.tmp.dir
/opt/hadoop/hbase
hbase.zookeeper.quorum
hdpdn01,hdpnn01,hdpnn02
hbase.hregion.majorcompaction
604800000
hbase.rootdir
hdfs://dpicluster/apps/hbase/data
hbase.hregion.memstore.block.multiplier
8
hfile.block.cache.size
0.40
hbase.hstore.blockingStoreFiles
20
hbase.hregion.max.filesize
10737418240
zookeeper.znode.parent
/hbase-unsecure
hbase.ipc.client.tcpnodelay
true
hbase.regionserver.handler.count
60
hbase.client.keyvalue.maxsize
10485760
hbase.hregion.memstore.flush.size
134217728
hbase.security.authentication
simple
hbase.cluster.distributed
true
hbase.client.scanner.caching
100
这样执行执行报dpicluster识别不了:
后来我修改了hosts:
hosts:
134.96.169.138hdpnn01dpicluster
134.96.169.139hdpnn02dpicluster
134.96.169.140hdpdn01
134.96.169.142hdpdn03
134.96.169.143hdpdn04
134.96.169.144hdpdn05
134.96.169.145hdpdn06
就可以执行通过,以下是执行结果:
2016-03-2000:25:56,077WARN[main]util.NativeCodeLoader(NativeCodeLoader.java:(62))-Unabletoloadnative-hadooplibraryforyourplatform...usingbuiltin-javaclasseswhereapplicable
2016-03-2000:25:56,317INFO[main]zookeeper.ZooKeeper(Environment.java:logEnv(100))-Clientenvironment:zookeeper.version=3.4.5-76--1,builton10/18/201300:43GMT
2016-03-2000:25:56,318INFO[main]zookeeper.ZooKeeper(Environment.java:logEnv(100))-Clientenvironment:host.name=hdpetl02
2016-03-2000:25:56,318INFO[main]zookeeper.ZooKeeper(Environment.java:logEnv(100))-Clientenvironment:java.version=1.7.0_03
2016-03-2000:25:56,318INFO[main]zookeeper.ZooKeeper(Environment.java:logEnv(100))-Clientenvironment:java.vendor=OracleCorporation
2016-03-2000:25:56,318INFO[main]zookeeper.ZooKeeper(Environment.java:logEnv(100))-Clientenvironment:java.home=/usr/java/jdk1.7.0_03/jre
2016-03-2000:25:56,318INFO[main]zookeeper.ZooKeeper(Environment.java:logEnv(100))-Clientenvironment:java.class.path=Dpi_Gao.jar
2016-03-2000:25:56,318INFO[main]zookeeper.ZooKeeper(Environment.java:logEnv(100))-Clientenvironment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2016-03-2000:25:56,318INFO[main]zookeeper.ZooKeeper(Environment.java:logEnv(100))-Clientenvironment:java.io.tmpdir=/tmp
2016-03-2000:25:56,318INFO[main]zookeeper.ZooKeeper(Environment.java:logEnv(100))-Clientenvironment:java.compiler=
2016-03-2000:25:56,319INFO[main]zookeeper.ZooKeeper(Environment.java:logEnv(100))-Clientenvironment:os.name=Linux
2016-03-2000:25:56,319INFO[main]zookeeper.ZooKeeper(Environment.java:logEnv(100))-Clientenvironment:os.arch=amd64
2016-03-2000:25:56,319INFO[main]zookeeper.ZooKeeper(Environment.java:logEnv(100))-Clientenvironment:os.version=2.6.32-279.el6.x86_64
2016-03-2000:25:56,319INFO[main]zookeeper.ZooKeeper(Environment.java:logEnv(100))-Clientenvironment:user.name=dpi
2016-03-2000:25:56,319INFO[main]zookeeper.ZooKeeper(Environment.java:logEnv(100))-Clientenvironment:user.home=/home/dpi
2016-03-2000:25:56,319INFO[main]zookeeper.ZooKeeper(Environment.java:logEnv(100))-Clientenvironment:user.dir=/data/data2/zhaozj
2016-03-2000:25:56,322INFO[main]zookeeper.ZooKeeper(ZooKeeper.java:(438))-Initiatingclientconnection,connectString=hdpnn01:2181,hdpdn01:2181,hdpnn02:2181sessionTimeout=30000watcher=hconnection-0x32731b00
2016-03-2000:25:56,346INFO[main]zookeeper.RecoverableZooKeeper(RecoverableZooKeeper.java:(120))-Processidentifier=hconnection-0x32731b00connectingtoZooKeeperensemble=hdpnn01:2181,hdpdn01:2181,hdpnn02:2181
2016-03-2000:25:56,349INFO[main-SendThread(hdpdn01:2181)]zookeeper.ClientCnxn(ClientCnxn.java:logStartConnect(966))-Openingsocketconnectiontoserverhdpdn01/134.96.169.140:2181.WillnotattempttoauthenticateusingSASL(unknownerror)
2016-03-2000:25:56,357INFO[main-SendThread(hdpdn01:2181)]zookeeper.ClientCnxn(ClientCnxn.java:primeConnection(849))-Socketconnectionestablishedtohdpdn01/134.96.169.140:2181,initiatingsession
2016-03-2000:25:56,367INFO[main-SendThread(hdpdn01:2181)]zookeeper.ClientCnxn(ClientCnxn.java:onConnected(1211))-Sessionestablishmentcompleteonserverhdpdn01/134.96.169.140:2181,sessionid=0x14fbc96bc6c845b,negotiatedtimeout=30000
RowName:8615305711575460110195005574120000000018600000002016030218051220160302180517502300000000253000008400000053000000000000000000140180000000Timestamp:1456920085092columnFamily:cfcolumnName:dpiValue:460110195005574|8615305711575|8657070207123835|ctlte.mnc011.mcc460.gprs|202.101.172.37|53|100.69.252.152|14018|115.169.126.169|115.169.126.3|115.169.126.130||64F0114B03|64F011|6|12|186|20160302180512|20160302180517|5023|253|84|1|1|OTHER
2016-03-2000:25:57,649INFO[main]client.HConnectionManager$HConnectionImplementation(HConnectionManager.java:closeZooKeeperWatcher(1810))-Closingzookeepersessionid=0x14fbc96bc6c845b
2016-03-2000:25:57,652INFO[main]zookeeper.ZooKeeper(ZooKeeper.java:close(684))-Session:0x14fbc96bc6c845bclosed
2016-03-2000:25:57,652INFO[main-EventThread]zookeeper.ClientCnxn(ClientCnxn.java:run(509))-EventThreadshutdown
我是新手,不了解zookeeper和hadoopHA工作原理,大神帮忙讲解一下,多谢多谢;
hadoop版本2.2.hbase版本0.96
分 -->
|