How to config use minio iceberg warehouse, keep getting "Invalid S3 URI" error, can anyone help me? #6411

ld000 · 2025-02-08T13:18:38Z

ld000
Feb 8, 2025

This is my jdbc and warehouse config:

gravitino.iceberg-rest.catalog-backend = jdbc
gravitino.iceberg-rest.uri = jdbc:mysql://192.168.xx.xxx:3306/gravitino?useUnicode=true&characterEncoding=utf-8&useSSL=false&nullCatalogMeansCurrent=true&serverTimezone=Asia/Shanghai&allowMultiQueries=true
gravitino.iceberg-rest.jdbc.user = root
gravitino.iceberg-rest.jdbc.password = xxx
gravitino.iceberg-rest.jdbc-driver = com.mysql.jdbc.Driver

gravitino.iceberg-rest.warehouse = s3://iceberg-data
gravitino.iceberg-rest.io-impl = org.apache.iceberg.aws.s3.S3FileIO
gravitino.iceberg-rest.s3-endpoint = http://192.168.xxx.xxx:9100
gravitino.iceberg-rest.s3-region = china
gravitino.iceberg-rest.s3-access-key-id = xxx
gravitino.iceberg-rest.s3-secret-access-key = xxx

I replaced my server information with 'xxx'.

With this config, I get the error below, where did I go wrong?

2025-02-08 20:53:43.317 WARN [iceberg-rest-18] [org.apache.iceberg.util.Tasks$Builder.runTaskWithRetry(Tasks.java:456)] - Retrying task after failure: Invalid S3 URI, cannot determine scheme: /tmp/my_namespace/my_table/metadata/00001-66cbfccd-f4e6-4a0a-8511-a58a784ed990.metadata.json
org.apache.iceberg.exceptions.ValidationException: Invalid S3 URI, cannot determine scheme: /tmp/my_namespace/my_table/metadata/00001-66cbfccd-f4e6-4a0a-8511-a58a784ed990.metadata.json
	at org.apache.iceberg.exceptions.ValidationException.check(ValidationException.java:49) ~[iceberg-api-1.5.2.jar:?]
	at org.apache.iceberg.aws.s3.S3URI.<init>(S3URI.java:72) ~[iceberg-aws-1.5.2.jar:?]
	at org.apache.iceberg.aws.s3.S3InputFile.fromLocation(S3InputFile.java:39) ~[iceberg-aws-1.5.2.jar:?]
	at org.apache.iceberg.aws.s3.S3FileIO.newInputFile(S3FileIO.java:125) ~[iceberg-aws-1.5.2.jar:?]
	at org.apache.iceberg.TableMetadataParser.read(TableMetadataParser.java:273) ~[iceberg-core-1.5.2.jar:?]
	at org.apache.iceberg.BaseMetastoreTableOperations.lambda$refreshFromMetadataLocation$0(BaseMetastoreTableOperations.java:189) ~[iceberg-core-1.5.2.jar:?]
	at org.apache.iceberg.BaseMetastoreTableOperations.lambda$refreshFromMetadataLocation$1(BaseMetastoreTableOperations.java:208) ~[iceberg-core-1.5.2.jar:?]
	at org.apache.iceberg.util.Tasks$Builder.runTaskWithRetry(Tasks.java:413) ~[iceberg-core-1.5.2.jar:?]
	at org.apache.iceberg.util.Tasks$Builder.runSingleThreaded(Tasks.java:219) ~[iceberg-core-1.5.2.jar:?]
	at org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:203) ~[iceberg-core-1.5.2.jar:?]
	at org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:196) ~[iceberg-core-1.5.2.jar:?]
	at org.apache.iceberg.BaseMetastoreTableOperations.refreshFromMetadataLocation(BaseMetastoreTableOperations.java:208) ~[iceberg-core-1.5.2.jar:?]
	at org.apache.iceberg.BaseMetastoreTableOperations.refreshFromMetadataLocation(BaseMetastoreTableOperations.java:185) ~[iceberg-core-1.5.2.jar:?]
	at org.apache.iceberg.BaseMetastoreTableOperations.refreshFromMetadataLocation(BaseMetastoreTableOperations.java:176) ~[iceberg-core-1.5.2.jar:?]
	at org.apache.iceberg.jdbc.JdbcTableOperations.doRefresh(JdbcTableOperations.java:100) ~[iceberg-core-1.5.2.jar:?]
	at org.apache.iceberg.BaseMetastoreTableOperations.refresh(BaseMetastoreTableOperations.java:97) ~[iceberg-core-1.5.2.jar:?]
	at org.apache.iceberg.BaseMetastoreTableOperations.current(BaseMetastoreTableOperations.java:80) ~[iceberg-core-1.5.2.jar:?]
	at org.apache.iceberg.BaseMetastoreCatalog.loadTable(BaseMetastoreCatalog.java:49) ~[iceberg-core-1.5.2.jar:?]
	at org.apache.iceberg.rest.CatalogHandlers.loadTable(CatalogHandlers.java:269) ~[iceberg-core-1.5.2.jar:?]
	at org.apache.gravitino.iceberg.common.ops.IcebergCatalogWrapper.lambda$loadTable$3(IcebergCatalogWrapper.java:181) ~[gravitino-iceberg-common-0.8.0-incubating.jar:?]
	at org.apache.gravitino.iceberg.common.ops.IcebergCatalogWrapper.injectTableConfig(IcebergCatalogWrapper.java:298) ~[gravitino-iceberg-common-0.8.0-incubating.jar:?]
	at org.apache.gravitino.iceberg.common.ops.IcebergCatalogWrapper.loadTable(IcebergCatalogWrapper.java:181) ~[gravitino-iceberg-common-0.8.0-incubating.jar:?]
	at org.apache.gravitino.iceberg.service.CatalogWrapperForREST.loadTable(CatalogWrapperForREST.java:95) ~[gravitino-iceberg-rest-server-0.8.0-incubating.jar:?]
	at org.apache.gravitino.iceberg.service.dispatcher.IcebergTableOperationExecutor.loadTable(IcebergTableOperationExecutor.java:77) ~[gravitino-iceberg-rest-server-0.8.0-incubating.jar:?]
	at org.apache.gravitino.iceberg.service.dispatcher.IcebergTableEventDispatcher.loadTable(IcebergTableEventDispatcher.java:153) ~[gravitino-iceberg-rest-server-0.8.0-incubating.jar:?]
	at org.apache.gravitino.iceberg.service.rest.IcebergTableOperations.loadTable(IcebergTableOperations.java:205) ~[gravitino-iceberg-rest-server-0.8.0-incubating.jar:?]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_412]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_412]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_412]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_412]
	at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) ~[jersey-server-2.41.jar:?]
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:146) ~[jersey-server-2.41.jar:?]
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:189) ~[jersey-server-2.41.jar:?]
	at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176) ~[jersey-server-2.41.jar:?]
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:93) ~[jersey-server-2.41.jar:?]
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:478) ~[jersey-server-2.41.jar:?]
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:400) ~[jersey-server-2.41.jar:?]
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) ~[jersey-server-2.41.jar:?]
	at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:256) ~[jersey-server-2.41.jar:?]
	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) ~[jersey-common-2.41.jar:?]
	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) ~[jersey-common-2.41.jar:?]
	at org.glassfish.jersey.internal.Errors.process(Errors.java:292) ~[jersey-common-2.41.jar:?]
	at org.glassfish.jersey.internal.Errors.process(Errors.java:274) ~[jersey-common-2.41.jar:?]
	at org.glassfish.jersey.internal.Errors.process(Errors.java:244) ~[jersey-common-2.41.jar:?]
	at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) ~[jersey-common-2.41.jar:?]
	at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:235) ~[jersey-server-2.41.jar:?]
	at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:684) ~[jersey-server-2.41.jar:?]
	at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) ~[jersey-container-servlet-core-2.41.jar:?]
	at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) ~[jersey-container-servlet-core-2.41.jar:?]
	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:358) ~[jersey-container-servlet-core-2.41.jar:?]
	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:311) ~[jersey-container-servlet-core-2.41.jar:?]
	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) ~[jersey-container-servlet-core-2.41.jar:?]
	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) ~[jetty-servlet-9.4.51.v20230217.jar:9.4.51.v20230217]
	at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1656) ~[jetty-servlet-9.4.51.v20230217.jar:9.4.51.v20230217]
	at org.apache.gravitino.server.authentication.AuthenticationFilter.doFilter(AuthenticationFilter.java:86) ~[gravitino-server-common-0.8.0-incubating.jar:?]
	at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) ~[jetty-servlet-9.4.51.v20230217.jar:9.4.51.v20230217]
	at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626) ~[jetty-servlet-9.4.51.v20230217.jar:9.4.51.v20230217]
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:552) ~[jetty-servlet-9.4.51.v20230217.jar:9.4.51.v20230217]
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) ~[jetty-server-9.4.51.v20230217.jar:9.4.51.v20230217]
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) ~[jetty-server-9.4.51.v20230217.jar:9.4.51.v20230217]
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) ~[jetty-server-9.4.51.v20230217.jar:9.4.51.v20230217]
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:505) ~[jetty-servlet-9.4.51.v20230217.jar:9.4.51.v20230217]
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) ~[jetty-server-9.4.51.v20230217.jar:9.4.51.v20230217]
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) ~[jetty-server-9.4.51.v20230217.jar:9.4.51.v20230217]
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) ~[jetty-server-9.4.51.v20230217.jar:9.4.51.v20230217]
	at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) ~[jetty-server-9.4.51.v20230217.jar:9.4.51.v20230217]
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) ~[jetty-server-9.4.51.v20230217.jar:9.4.51.v20230217]
	at org.eclipse.jetty.server.Server.handle(Server.java:516) ~[jetty-server-9.4.51.v20230217.jar:9.4.51.v20230217]
	at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) ~[jetty-server-9.4.51.v20230217.jar:9.4.51.v20230217]
	at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) [jetty-server-9.4.51.v20230217.jar:9.4.51.v20230217]
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) [jetty-server-9.4.51.v20230217.jar:9.4.51.v20230217]
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) [jetty-server-9.4.51.v20230217.jar:9.4.51.v20230217]
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) [jetty-io-9.4.51.v20230217.jar:9.4.51.v20230217]
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) [jetty-io-9.4.51.v20230217.jar:9.4.51.v20230217]
	at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) [jetty-io-9.4.51.v20230217.jar:9.4.51.v20230217]
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) [jetty-util-9.4.51.v20230217.jar:9.4.51.v20230217]
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) [jetty-util-9.4.51.v20230217.jar:9.4.51.v20230217]
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) [jetty-util-9.4.51.v20230217.jar:9.4.51.v20230217]
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) [jetty-util-9.4.51.v20230217.jar:9.4.51.v20230217]
	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) [jetty-util-9.4.51.v20230217.jar:9.4.51.v20230217]
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) [jetty-util-9.4.51.v20230217.jar:9.4.51.v20230217]
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) [jetty-util-9.4.51.v20230217.jar:9.4.51.v20230217]
	at java.lang.Thread.run(Thread.java:750) [?:1.8.0_412]

FANNG1 · 2025-02-09T01:15:04Z

FANNG1
Feb 9, 2025
Collaborator

Could you provide more context? The configuration of gravitino.iceberg-rest.warehouse seems to be /tmp not s3://xx in the server.

The Gravitino Iceberg REST server version
The configuration file name
How do you start the server

2 replies

ld000 Feb 9, 2025
Author

The Gravitino Iceberg REST server version
0.8.0-incubating
The configuration file name
${gravitino_home}/conf/gravitino-iceberg-rest-server.conf
How do you start the server
${gravitino_home}/bin/gravitino-iceberg-rest-server.sh start

FANNG1 Feb 9, 2025
Collaborator

I setup an minio service by docker and could query the data successfully. Could you double check your configuration, is there multi gravitino.iceberg-rest.warehouse configuration items in your config file?

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

How to config use minio iceberg warehouse, keep getting "Invalid S3 URI" error, can anyone help me? #6411

Uh oh!

{{title}}

Uh oh!

Replies: 1 comment 2 replies

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

How to config use minio iceberg warehouse, keep getting "Invalid S3 URI" error, can anyone help me? #6411

Uh oh!

ld000 Feb 8, 2025

Replies: 1 comment · 2 replies

Uh oh!

FANNG1 Feb 9, 2025 Collaborator

Uh oh!

ld000 Feb 9, 2025 Author

Uh oh!

FANNG1 Feb 9, 2025 Collaborator

ld000
Feb 8, 2025

Replies: 1 comment 2 replies

FANNG1
Feb 9, 2025
Collaborator

ld000 Feb 9, 2025
Author

FANNG1 Feb 9, 2025
Collaborator