ITeye博客

bupt04406

bupt04406 · 共 97 篇文章

hbase flush前提: 等待相关事务都完成

DefaultMemStore: @Override public void rollback(Cell cell) { // If the key is in the snapshot, delete it. We should not update // this.size, because that tracks the size of only the memstore and // not the snapshot. The flush of this snapshot to disk has not // yet started because Store.flush() wai…

2016-07-04 16:40 · hbase · 评论(0)

druid

默认端口: coordinator http://localhost:8081 overlord http://localhost:8090/console.html broker 8082 (1) druid datasource配置保存7天的数据 详见http://druid.io/docs/latest/operations/rule-configuration.html https://groups.google.com/forum/#!searchin/druid-user/rules/druid-user/ve9hb1K6RV4/qcPGRynH3F8J druid 默认保存所有…

2015-10-29 12:13 · druid · 评论(0)

git

git 回滚: (1)本地回滚 git reset --hard 版本号 (2)强行回滚到远程服务器: git push -f git diff commit_previous commit > mypatch.diff :最原始的diff信息,对于这里的commit_previous(commit之前一个commit),可以使用“commit^”来表示,这样比较方便,不易出错。 http://stackoverflow.com/questions/791959/download-a-specific-tag-with-git https://git-scm.com/book/en/v2/G…

2014-10-30 09:38 · linux · 评论(0)

hbase的一些语义

hbase里面对一行操作前会加锁。 http://hadoop-hbase.blogspot.com/2012/01/hbase-intra-row-transactions.html https://issues.apache.org/jira/browse/HBASE-3584 HBASE-3584的功能是对同一行的Put和Delete可以原子性的完成。 Delete: put 't1', 'r1', 'f1:c', 'aaa', 1 put 't1', 'r1', 'f1:c', 'bbb', 2 put 't1', 'r1', 'f1:c', 'ccc', 3 delete 't1'…

2014-06-04 16:39 · hbase · 评论(0)

如何高效使用SimpleDateFormat

在我们的业务代码中很多地方会用到SimpleDateFormat,把long型的时间转换成string类型,这里就会用到SimpleDateFormat,而这个类不是线程安全的,所以业务代码中往往如下的方式使用: 01 /** 02 * 将毫秒时间转换为 20120813154028 这种形式 03 * 04 * @param millisecond 05 * @return 06 */ 07 public static String milliSecondToString(long millisecond) { 08 SimpleDateFormat sd = new SimpleDate…

2014-03-07 16:08 · java · 评论(0)

性能查看

(1) top H 看线程,以及线程id 1看各个cpu top -Hp pid (2) pstack pid | less 看线程在做什么 (3) jstack pid | less 根据线程id转换成16进制看 线程栈 通过 printf "%x\n" 30834 首先转化成16进制 jstat -gcutil 28385 1000 jstat -gccause 28385 1000 http://docs.oracle.com/javase/1.5.0/docs/tooldocs/share/jstat.html jmap -heap 12752 for ((i=1; $i<100; …

2014-01-13 20:27 · java · linux · 评论(0)

Unexpected state导致HMaster abort

线上和开发集群都遇到过master abort的情况,而且错误都是Unexpected state导致的FATAL错误 2014-01-08 13:44:40,770 FATAL org.apache.hadoop.hbase.master.HMaster: Master server abort: loaded coprocessors are: [] 2014-01-08 13:44:40,771 FATAL org.apache.hadoop.hbase.master.HMaster: 69 Unexpected state : usertable2,,1389159324467.a3…

2014-01-09 11:44 · hbase · 评论(2)

hbase build Too many unapproved licenses

[ERROR] Failed to execute goal org.apache.rat:apache-rat-plugin:0.8:check (default) on project hbase: Too many unapproved licenses: 2856 -> [Help 1] 在target/rat.txt下面寻找哪些文件前面打了问号,去掉这些文件! cat files | xargs -I {} echo rm -rf {} cat files | xargs -I {} rm -rf {} 如果还不行的话,通过修改pom.xml来绕过: <plugin> <!-- e…

2013-12-30 16:02 · hbase · 评论(0)

hbase hbck 导致的multi assign

http://hbase.apache.org/book.html#d2907e14182 $ ./bin/hbase hbck -fixAssignments hbck -fixAssignments执行的时候会从regionserver读取online的region,如果有个region正在opening,那么它会不在online列表中,这个时候hbck会再assign这个region,导致region multi assign.

2013-12-25 18:03 · hbase · 评论(0)

HBase Log Split 配置相关

(1)HMaster SplitLogManager // get a list for previously failed RS which need log splitting work // we recover hbase:meta region servers inside master initialization and // handle other failed servers in SSH in order to start up master node ASAP Set<ServerName> previouslyFailedServers = this.fileSys…

2013-12-15 16:54 · hbase · 评论(0)

hbase jmx

conf/hbase-env.sh 里面配了 JMX后就可以用 jconsole远程连到regionserver上面去看进程的metrics。 # Uncomment and adjust to enable JMX exporting # See jmxremote.password and jmxremote.access in $JRE_HOME/lib/management to configure remote password access. # More details at: http://java.sun.com/javase/6/docs/technotes/guides…

2013-12-11 20:42 · hadoop · hbase · 评论(0)

hbase bulk多family,锁住region

HBase在bulk 多family的数据时,会获得整个region的写锁 // we need writeLock for multi-family bulk load startBulkRegionOperation(hasMultipleColumnFamilies(familyPaths)); multi-CF bulk load is not atomic across column families https://issues.apache.org/jira/browse/HBASE-4552 如果开启了 verifyBulkLoads 或者不同hadoop集群 没distcp…

2013-10-29 15:14 · hbase · 评论(0)

Too many fetch failures

http://lucene.472066.n3.nabble.com/Reg-Too-many-fetch-failures-Error-td4037975.html http://www.slideshare.net/cloudera/hadoop-troubleshooting-101-kate-ting-cloudera http://lucene.472066.n3.nabble.com/Reg-Too-many-fetch-failures-Error-td4037975.html As you may be aware this means the reduces are una…

2013-10-29 10:42 · hadoop · 评论(0)

Phoneix

https://github.com/forcedotcom/phoenix http://www.vogella.com/articles/MySQLJava/article.html HBase shell : create 't1', {NAME => 'f1', VERSIONS => 5} put 't1', "row1", 'f1:1', "aaa" scan 't1' Phoneix: !tables CREATE TABLE IF NOT EXISTS "t1" ( row VARCHAR NOT NULL, "f1"."1" VARCHAR CONSTRAINT PK PR…

2013-10-22 11:42 · hbase · 评论(0)

hbase RowCounter 使用

hbase org.apache.hadoop.hbase.mapreduce.RowCounter tablename cf:cq 有同学说这个运行出来的结果不对,结果只有几十,而实际上应该有几百万。差别这么大,原因在哪里? 查看代码发现 RowCounter的代码片段 scan.setFilter(new FirstKeyOnlyFilter()); if (sb.length() > 0) { for (String columnName : sb.toString().trim().split(" ")) { String [] fields = columnName.split("…

2013-10-15 16:48 · hbase · 评论(0)