sqoop 导入hive 报错

yangyufans HBase 数据的导入 最后由 ujnzhoubing 于2016年07月07日回复

  • 1 回答
  • 2.1k 浏览

[root@node28 ~]#  sqoop import --connect jdbc:mysql://10.211.55.38:3306/test --username root --password 123456 --table test -m 1 --hive-import --hive-table "TEST"

Warning: /usr/local/sqoop-1.4.6/bin/../../hcatalog does not exist! HCatalog jobs will fail.

Please set $HCAT_HOME to the root of your HCatalog installation.

Warning: /usr/local/sqoop-1.4.6/bin/../../accumulo does not exist! Accumulo imports will fail.

Please set $ACCUMULO_HOME to the root of your Accumulo installation.

Warning: /usr/local/sqoop-1.4.6/bin/../../zookeeper does not exist! Accumulo imports will fail.

Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.

16/06/29 22:33:04 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6

16/06/29 22:33:04 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.

16/06/29 22:33:04 INFO tool.BaseSqoopTool: Using Hive-specific delimiters for output. You can override

16/06/29 22:33:04 INFO tool.BaseSqoopTool: delimiters with --fields-terminated-by, etc.

16/06/29 22:33:04 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.

16/06/29 22:33:04 INFO tool.CodeGenTool: Beginning code generation

SLF4J: Class path contains multiple SLF4J bindings.

SLF4J: Found binding in [jar:file:/usr/local/hadoop-2.5.1/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: Found binding in [jar:file:/usr/local/hbase-0.9.8/lib/slf4j-log4j12-1.6.4.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.

SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]

16/06/29 22:33:15 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `test` AS t LIMIT 1

16/06/29 22:33:15 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `test` AS t LIMIT 1

16/06/29 22:33:15 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/local/hadoop-2.5.1/share/hadoop/mapreduce

Note: /tmp/sqoop-root/compile/7024c578469ab36a5535bae09e7656f7/test.java uses or overrides a deprecated API.

Note: Recompile with -Xlint:deprecation for details.

16/06/29 22:33:17 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/7024c578469ab36a5535bae09e7656f7/test.jar

16/06/29 22:33:17 WARN manager.MySQLManager: It looks like you are importing from mysql.

16/06/29 22:33:17 WARN manager.MySQLManager: This transfer can be faster! Use the --direct

16/06/29 22:33:17 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.

16/06/29 22:33:17 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)

16/06/29 22:33:17 INFO mapreduce.ImportJobBase: Beginning import of test

16/06/29 22:33:17 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar

16/06/29 22:33:18 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps

16/06/29 22:33:31 INFO db.DBInputFormat: Using read commited transaction isolation

16/06/29 22:33:32 INFO mapreduce.JobSubmitter: number of splits:1

16/06/29 22:33:32 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1467249797849_0009

16/06/29 22:33:32 INFO impl.YarnClientImpl: Submitted application application_1467249797849_0009

16/06/29 22:33:32 INFO mapreduce.JobSubmitter: Cleaning up the staging area /tmp/hadoop-yarn/staging/root/.staging/job_1467249797849_0009

Exception in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.http.HttpConfig.getSchemePrefix()Ljava/lang/String;

at org.apache.hadoop.mapred.ClientServiceDelegate.getJobStatus(ClientServiceDelegate.java:428)

at org.apache.hadoop.mapred.YARNRunner.submitJob(YARNRunner.java:302)

at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:430)

at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1268)

at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1265)

at java.security.AccessController.doPrivileged(Native Method)

at javax.security.auth.Subject.doAs(Subject.java:415)

at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)

at org.apache.hadoop.mapreduce.Job.submit(Job.java:1265)

at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1286)

at org.apache.sqoop.mapreduce.ImportJobBase.doSubmitJob(ImportJobBase.java:196)

at org.apache.sqoop.mapreduce.ImportJobBase.runJob(ImportJobBase.java:169)

at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:266)

at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:673)

at org.apache.sqoop.manager.MySQLManager

  • ujnzhoubing 2016年07月07日 回答 #1楼
  • 看错误好像是sqoop,hive,hadoop版本兼容性的问题,能否提供您所使用的各个软件的版本,以及相应的配置文件和jar包版本?

  • 0 评论