[Bug report] SparkIcebergCatalogHiveBackendIT35 > testIcebergSQLExtensions() FAILED (Cron Integration Test 20240912)

[danhuawang]

Version

main branch

Describe what's wrong

Run spark IT failed
./gradlew -PskipTests -PtestMode=embedded -PjdkVersion=8 -PscalaVersion=2.12 -PskipDockerTests=false :spark-connector:spark-3.5:test --tests "org.apache.gravitino.spark.connector.integration.test.**"

Error message and/or stacktrace

SparkIcebergCatalogHiveBackendIT35 > testIcebergSQLExtensions() FAILED
    org.apache.iceberg.exceptions.RuntimeIOException: Failed to write json to file: hdfs://172.17.0.3:9000/user/hive/default_db/test_iceberg_distribution_and_order/metadata/00001-6deec507-e192-4dc6-a211-2016b827057e.metadata.json
        at org.apache.iceberg.TableMetadataParser.internalWrite(TableMetadataParser.java:132)
        at org.apache.iceberg.TableMetadataParser.overwrite(TableMetadataParser.java:114)
        at org.apache.iceberg.BaseMetastoreTableOperations.writeNewMetadata(BaseMetastoreTableOperations.java:170)
        at org.apache.iceberg.BaseMetastoreTableOperations.writeNewMetadataIfRequired(BaseMetastoreTableOperations.java:160)
        at org.apache.iceberg.hive.HiveTableOperations.doCommit(HiveTableOperations.java:185)
        at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54)
        at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67)
        at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52)
        at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:114)
        at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:86)
        at org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.execute(DefaultLauncherSession.java:86)
        at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.processAllTestClasses(JUnitPlatformTestClassProcessor.java:110)
        at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.access$000(JUnitPlatformTestClassProcessor.java:90)
        at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor.stop(JUnitPlatformTestClassProcessor.java:85)
        at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:62)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
        at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
        at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
        at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
        at com.sun.proxy.$Proxy2.stop(Unknown Source)
        at org.gradle.api.internal.tasks.testing.worker.TestWorker$3.run(TestWorker.java:193)
        at org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:129)
        at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:100)
        at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:60)
        at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56)
        at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:113)
        at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:65)
        at worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69)
        at worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74)

        Caused by:
        java.io.IOException: The stream is closed
            at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:117)
            at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
            at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
            at java.io.DataOutputStream.flush(DataOutputStream.java:123)
            at java.io.FilterOutputStream.close(FilterOutputStream.java:158)
            at org.apache.hadoop.hdfs.DataStreamer.closeStream(DataStreamer.java:1051)
            at org.apache.hadoop.hdfs.DataStreamer.closeInternal(DataStreamer.java:884)
            at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:837)

127 tests completed, 1 failed, 5 skipped

:spark-connector:spark-3.5:test failure marker

> Task :spark-connector:spark-3.5:test FAILED

Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

For more on this, please refer to https://docs.gradle.org/8.2/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.
48 actionable tasks: 10 executed, 38 up-to-date
FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':spark-connector:spark-3.5:test'.
> There were failing tests. See the report at: file:///actions-runner/_work/gravitino-test/gravitino-test/gravitino/build/reports/index.html

* Try:
> Run with --scan to get full insights.

BUILD FAILED in 6m 29s

How to reproduce

Reproduce by cron integration test

Additional context

No response

[danhuawang]

@FANNG1 Can you help check this issue?

[FANNG1]

Seems caused by the environment, let's check the next test result.