Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

drop table throws exception #9827

Closed
photogamerun opened this issue Jan 26, 2018 · 2 comments
Closed

drop table throws exception #9827

photogamerun opened this issue Jan 26, 2018 · 2 comments

Comments

@photogamerun
Copy link

photogamerun commented Jan 26, 2018

when I use presto-cli

kinit -kt /tmp/gjjddatateam.keytab gjjddatateam@DC.SH.CTRIPCORP.COM
[admin@SVR13358HW1288 tmp]$ ./presto-cli-0.190-executable.jar --server https://svr13358hw1288:7778 --enable-authentication --krb5-remote-service-name hive --keystore-path /tmp/keystore.jks --keystore-password presto --user gjjddatateam --source presto-cli --catalog hive --schema default;
presto:default> CREATE TABLE IF NOT EXISTS tmp_gjjd.orders_by_date As select * from tmp_gjjd.uid00012 limit 1;
CREATE TABLE: 1 row

Query 20180126_023041_00130_xjv5c, FINISHED, 14 nodes
Splits: 268 total, 258 done (96.27%)
0:01 [3.07K rows, 2.79MB] [2.42K rows/s, 2.2MB/s]

presto:default> select * from tmp_gjjd.orders_by_date;
    uid     |  orderid   
------------+------------
 2050712588 | 4573386752 
(1 row)

Query 20180126_023058_00131_xjv5c, FINISHED, 2 nodes
Splits: 17 total, 17 done (100.00%)
0:00 [1 rows, 313B] [5 rows/s, 1.57KB/s]

since now everything is ok I can create table with gjjddatateam account
I didn't exit terminal

presto:default>drop table if exists tmp_gjjd.orders_by_date;
Query 20180126_030443_00251_xjv5c failed: The transaction didn't commit cleanly. Failed to execute some metastore delete operations: drop table tmp_gjjd.orders_by_date

I got this exception:
com.facebook.presto.spi.PrestoException: The transaction didn't commit cleanly. Failed to execute some metastore delete operations: drop table tmp_gjjd.uid00012
	at com.facebook.presto.hive.metastore.SemiTransactionalHiveMetastore$Committer.executeIrreversibleMetastoreOperations(SemiTransactionalHiveMetastore.java:1219)
	at com.facebook.presto.hive.metastore.SemiTransactionalHiveMetastore$Committer.access$2100(SemiTransactionalHiveMetastore.java:887)
	at com.facebook.presto.hive.metastore.SemiTransactionalHiveMetastore.commitShared(SemiTransactionalHiveMetastore.java:865)
	at com.facebook.presto.hive.metastore.SemiTransactionalHiveMetastore.commit(SemiTransactionalHiveMetastore.java:739)
	at com.facebook.presto.hive.HiveMetadata.commit(HiveMetadata.java:1503)
	at com.facebook.presto.hive.HiveConnector.commit(HiveConnector.java:177)
	at com.facebook.presto.transaction.TransactionManager$TransactionMetadata$ConnectorTransactionMetadata.commit(TransactionManager.java:577)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:111)
	at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:58)
	at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:75)
	at io.airlift.concurrent.BoundedExecutor.drainQueue(BoundedExecutor.java:78)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
	Suppressed: com.facebook.presto.spi.PrestoException: Table metadata not deleted since hdfs://ns/user/hive/warehouse/tmp_gjjd.db is not writable by hive@DC.SH.CTRIPCORP.COM
		at com.facebook.presto.hive.metastore.thrift.ThriftHiveMetastore.dropTable(ThriftHiveMetastore.java:426)
		at com.facebook.presto.hive.metastore.thrift.BridgingHiveMetastore.dropTable(BridgingHiveMetastore.java:156)
		at com.facebook.presto.hive.metastore.CachingHiveMetastore.dropTable(CachingHiveMetastore.java:438)
		at com.facebook.presto.hive.metastore.CachingHiveMetastore.dropTable(CachingHiveMetastore.java:438)
		at com.facebook.presto.hive.metastore.SemiTransactionalHiveMetastore$Committer.lambda$prepareDropTable$0(SemiTransactionalHiveMetastore.java:909)
		at com.facebook.presto.hive.metastore.SemiTransactionalHiveMetastore$IrreversibleMetastoreOperation.run(SemiTransactionalHiveMetastore.java:2039)
		at com.facebook.presto.hive.metastore.SemiTransactionalHiveMetastore$Committer.executeIrreversibleMetastoreOperations(SemiTransactionalHiveMetastore.java:1206)
		... 14 more
	Caused by: org.apache.hadoop.hive.metastore.api.MetaException: Table metadata not deleted since hdfs://ns/user/hive/warehouse/tmp_gjjd.db is not writable by hive@DC.SH.CTRIPCORP.COM
		at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$drop_table_result$drop_table_resultStandardScheme.read(ThriftHiveMetastore.java:31054)
		at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$drop_table_result$drop_table_resultStandardScheme.read(ThriftHiveMetastore.java:31031)
		at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$drop_table_result.read(ThriftHiveMetastore.java:30973)
		at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
		at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_drop_table(ThriftHiveMetastore.java:1109)
		at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.drop_table(ThriftHiveMetastore.java:1094)
		at com.facebook.presto.hive.metastore.thrift.ThriftHiveMetastoreClient.dropTable(ThriftHiveMetastoreClient.java:123)
		at com.facebook.presto.hive.metastore.thrift.ThriftHiveMetastore.lambda$dropTable$14(ThriftHiveMetastore.java:417)
		at com.facebook.presto.hive.metastore.thrift.HiveMetastoreApiStats.lambda$wrap$0(HiveMetastoreApiStats.java:42)
		at com.facebook.presto.hive.RetryDriver.run(RetryDriver.java:137)
		at com.facebook.presto.hive.metastore.thrift.ThriftHiveMetastore.dropTable(ThriftHiveMetastore.java:415)
		... 20 more

from hdfs side.

[admin@VMS16158 ~]$ hadoop fs -ls /user/hive/warehouse/tmp_gjjd.db/orders_by_date
Found 1 items
-rw-rw-r--   3 gjjddatateam hadoop        313 2018-01-26 10:30 /user/hive/warehouse/tmp_gjjd.db/orders_by_date/20180126_023041_00130_xjv5c_d7329f2a-4afe-4e81-909f-ab3908a567ea

I thinks there must be a bug, since I can create table with gjjddatateam account in kerberos model why I can not drop table

by the way in hive.properties I have conf hive.hdfs.impersonation.enabled=true

@losipiuk
Copy link
Contributor

That happens because party responsible for deleting table files from HDFS on DROP TABLE is Hive Metastore (not Presto itself).
Currently Presto does not support impersonating of end user for Hive metastore connection.
And HDFS operations performed on HDFS by Hive metastore are done on behalf of service user used for connecting to Metastore by Presto (hive@DC.SH.CTRIPCORP.COM in your case). And that user does not have permissions to delete this table files.

@sopel39 sopel39 changed the title drop table throw exception drop table throws exception Jan 26, 2018
@findepi
Copy link
Contributor

findepi commented Jan 24, 2019

This will be fixed by trinodb/trino#43

@findepi findepi closed this as completed Jan 24, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants