1.yarn会话模式的管理#查看yarn应用的日志 yarn logs -applicationId application_1782739955886_0005 |less #查看的结果如下; Container: container_e02_1782739955886_0005_01_000002 on hadoop001_35005 LogAggregationType: AGGREGATED LogType:directory.info LogLastModifiedTime:Mon Jun 29 22:37:40 0800 2026 LogLength:6332 LogContents: ls -l: total 32 -rw-r--r-- 1 flink flink 7 Jun 29 22:37 container_tokens -rwx------ 1 flink flink 705 Jun 29 22:37 default_container_executor_session.sh -rwx------ 1 flink flink 760 Jun 29 22:37 default_container_executor.sh lrwxrwxrwx 1 flink flink 170 Jun 29 22:37 flink-conf.yaml - /oracle/hadoop/tmp/nm-local-dir/usercache/flink/appcache/application_1782739955886_0005/filecache/24/application_1782739955886_0005-flink-conf.yaml7389539160443170593.tmp lrwxrwxrwx 1 flink flink 122 Jun 29 22:37 flink-dist-1.17.0.jar - /oracle/hadoop/tmp/nm-local-dir/usercache/flink/appcache/application_1782739955886_0005/filecache/10/flink-dist-1.17.0.jar lrwxrwxrwx 1 flink flink 136 Jun 29 22:37 FlinkTuTorial-1.17-1.0-SNAPSHOT.jar - /oracle/hadoop/tmp/nm-local-dir/usercache/flink/appcache/application_1782739955886_0005/filecache/15/FlinkTuTorial-1.17-1.0-SNAPSHOT.jar -rwx------ 1 flink flink 12658 Jun 29 22:37 launch_container.sh drwxrwxr-x 2 flink flink 4096 Jun 29 22:37 lib lrwxrwxrwx 1 flink flink 117 Jun 29 22:37 log4j.properties - /oracle/hadoop/tmp/nm-local-dir/usercache/flink/appcache/application_1782739955886_0005/filecache/23/log4j.properties drwxrwxr-x 10 flink flink 210 Jun 29 22:37 plugins drwx--x--- 2 flink flink 6 Jun 29 22:37 tmp find -L . -maxdepth 5 -ls: 68675569 4 drwx--x--- 5 flink flink 4096 Jun 29 22:37 . 135775831 0 drwx--x--- 2 flink flink 6 Jun 29 22:37 ./tmp 68675570 4 -rw-r--r-- 1 flink flink 7 Jun 29 22:37 ./container_tokens 68675571 4 -rw-r--r-- 1 flink flink 12 Jun 29 22:37 ./.container_tokens.crc 68675572 16 -rwx------ 1 flink flink 12658 Jun 29 22:37 ./launch_container.sh 68675573 4 -rw-r--r-- 1 flink flink 108 Jun 29 22:37 ./.launch_container.sh.crc 68675574 4 -rwx------ 1 flink flink 705 Jun 29 22:37 ./default_container_executor_session.sh 68675575 4 -rw-r--r-- 1 flink flink 16 Jun 29 22:37 ./.default_container_executor_session.sh.crc 68675576 4 -rwx------ 1 flink flink 760 Jun 29 22:37 ./default_container_executor.sh#通过yarn-session命令行的方式也可以停止yarn应用。echo stop | yarn-session.sh -id application_1782739955886_0005#这个命令专门用于停止 Flink YARN Session 模式下的集群2.yarn 的单作业运行模式在 YARN 环境中由于有了外部平台做资源调度所以我们也可以直接向YARN 提交一个单独的作业从而启动一个 FLINK 集群。在客户端解析代码。[flinkhadoop001 flink-1.17.0]$ bin/flink run -d -t yarn-per-job -c com.ycl.WordCountStreamUnboundedDemo lib/FlinkTuTorial-1.17-1.0-SNAPSHOT.jar SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/opt/module/flink-1.17.0/lib/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/opt/module/hadoop-3.3.6/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] 2026-06-29 22:36:01,965 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli [] - Found Yarn properties file under /tmp/.yarn-properties-flink. 2026-06-29 22:36:01,965 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli [] - Found Yarn properties file under /tmp/.yarn-properties-flink. WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by org.apache.flink.api.java.ClosureCleaner (file:/opt/module/flink-1.17.0/lib/flink-dist-1.17.0.jar) to field java.lang.String.value WARNING: Please consider reporting this to the maintainers of org.apache.flink.api.java.ClosureCleaner WARNING: Use --illegal-accesswarn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release 2026-06-29 22:36:02,580 WARN org.apache.flink.yarn.configuration.YarnLogConfigUtil [] - The configuration directory (/opt/module/flink-1.17.0/conf)already contains a LOG4J config file.If you want to use logback, then please delete or rename the log configuration file. 2026-06-29 22:36:02,823 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - No path for the flink jar passed. Using the location of classorg.apache.flink.yarn.YarnClusterDescriptor to locate the jar 2026-06-29 22:36:02,836 WARN org.apache.flink.yarn.YarnClusterDescriptor [] - Job Clusters are deprecated since Flink 1.15. Please use an Application Cluster/Application Mode instead. 2026-06-29 22:36:02,907 INFO org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider [] - Failing over to yarnrm2 2026-06-29 22:36:02,990 INFO org.apache.hadoop.conf.Configuration [] - resource-types.xml not found 2026-06-29 22:36:02,990 INFO org.apache.hadoop.yarn.util.resource.ResourceUtils [] - Unable to find resource-types.xml. 2026-06-29 22:36:03,042 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - The configured JobManager memory is 1600 MB. YARN will allocate 2048 MB to make up an integer multiple of its minimum allocation memory (1024 MB, configured via yarn.scheduler.minimum-allocation-mb). The extra 448 MB may not be used by Flink. 2026-06-29 22:36:03,042 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - The configured TaskManager memory is 1728 MB. YARN will allocate 2048 MB to make up an integer multiple of its minimum allocation memory (1024 MB, configured via yarn.scheduler.minimum-allocation-mb). The extra 320 MBmay not be used by Flink. 2026-06-29 22:36:03,042 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - Cluster specification: ClusterSpecification{masterMemoryMB1600, taskManagerMemoryMB1728, slotsPerTaskManager1} 2026-06-29 22:36:11,759 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - Cannot use kerberos delegation token manager, no valid kerberos credentials provided. 2026-06-29 22:36:11,764 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - Submitting application master application_1782739955886_0005 2026-06-29 22:36:12,021 INFO org.apache.hadoop.yarn.client.api.impl.YarnClientImpl [] - Submitted application application_1782739955886_0005 2026-06-29 22:36:12,022 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - Waiting for the cluster to be allocated 2026-06-29 22:36:12,025 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - Deploying cluster, current state ACCEPTED 2026-06-29 22:37:12,066 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - Deployment took more than 60 seconds. Please check if the requested resources are available in the YARN cluster 2026-06-29 22:37:27,477 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - YARN application has been deployed successfully. 2026-06-29 22:37:27,478 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - The Flink YARN session cluster has been started in detached mo de. In order to stop Flink gracefully, use the following command: $ echo stop | ./bin/yarn-session.sh -id application_1782739955886_0005 If this should not be possible, then you can also kill Flink via YARNs web interface or via: $ yarn application -kill application_1782739955886_0005 Note that killing Flink might not clean up all job artifacts and temporary files. 2026-06-29 22:37:27,479 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - Found Web Interface hadoop003:15478 of application application_1782739955886_0005. Job has been submitted with JobID 7ba5e9a55e7854651d57f43e85b7d280 Exception in thread Thread-4 java.lang.IllegalStateException: Trying to access closed classloader. Please check if you store classloaders directly or indirectly in static fields. If the stacktrace suggests that the leak occurs in a third party library and cannot be fixed immediately, you can disable this check with the configuration classloader.check-leaked-classloader. at org.apache.flink.util.FlinkUserCodeClassLoaders$SafetyNetWrapperClassLoader.ensureInner(FlinkUserCodeClassLoaders.java:184) at org.apache.flink.util.FlinkUserCodeClassLoaders$SafetyNetWrapperClassLoader.getResource(FlinkUserCodeClassLoaders.java:208) at org.apache.hadoop.conf.Configuration.getResource(Configuration.java:2839) at org.apache.hadoop.conf.Configuration.getStreamReader(Configuration.java:3113) at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:3072) at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:3045) at org.apache.hadoop.conf.Configuration.loadProps(Configuration.java:2923) at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2905) at org.apache.hadoop.conf.Configuration.get(Configuration.java:1247) at org.apache.hadoop.conf.Configuration.getTimeDuration(Configuration.java:1864) at org.apache.hadoop.conf.Configuration.getTimeDuration(Configuration.java:1841) at org.apache.hadoop.util.ShutdownHookManager.getShutdownTimeout(ShutdownHookManager.java:183) at org.apache.hadoop.util.ShutdownHookManager.shutdownExecutor(ShutdownHookManager.java:145) at org.apache.hadoop.util.ShutdownHookManager.access$300(ShutdownHookManager.java:65) at org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:102)bin/flink run -d -t yarn-per-job -c com.ycl.WordCountStreamUnboundedDemo lib/FlinkTuTorial-1.17-1.0-SNAPSHOT.jar这个方式启动后客户端退出不会影响正在执行的作业。#分离模式启动;客户端启动后在后台运行不会一直在前台界面停留。bin/flink run -d -t yarn-per-job -d -c com.ycl.WordCountStreamUnboundedDemo lib/FlinkTuTorial-1.17-1.0-SNAPSHOT.jar查看或者取消作业;#查看作业bin/flink list -t yarn-per-job -Dyarn.application.idapplication_1782910361073_0001#取消作业bin/flink cancel -t yarn-per-job -Dyarn.application.idapplication_1782910361073_0001 jobIdapplication_1782910361073_0001 是当前应用的ID,JOBID 是作业的ID,如果取消作业整个flink集群也会停掉。单作业模式作业停止后集群关闭了同时作业状态变成:Finished; 资源会被回收。#在分离模式下的一个问题从flink1.13版本就有。you can disable this check with the configuration classloader.check-leaked-classloader.#可以通过配置关闭这个报错。vim flink-conf.yamlclassloader.check-leaked-classloader: false再次运行作业看是否报错。 可以看到已经没有报错了。 [flinkhadoop001 flink-1.17.0]$ bin/flink run -d -t yarn-per-job -c com.ycl.WordCountStreamUnboundedDemo lib/FlinkTuTorial-1.17-1.0-SNAPSHOT.jar SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/opt/module/flink-1.17.0/lib/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/opt/module/hadoop-3.3.6/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] 2026-07-01 21:06:17,392 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli [] - Found Yarn properties file under /tmp/.yarn-properties-flink. 2026-07-01 21:06:17,392 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli [] - Found Yarn properties file under /tmp/.yarn-properties-flink. WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by org.apache.flink.api.java.ClosureCleaner (file:/opt/module/flink-1.17.0/lib/flink-dist-1.17.0.jar) to field java.lang.String.value WARNING: Please consider reporting this to the maintainers of org.apache.flink.api.java.ClosureCleaner WARNING: Use --illegal-accesswarn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release 2026-07-01 21:06:17,812 WARN org.apache.flink.yarn.configuration.YarnLogConfigUtil [] - The configuration directory (/opt/module/flink-1.17.0/conf)already contains a LOG4J config file.If you want to use logback, then please delete or rename the log configuration file. 2026-07-01 21:06:18,073 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - No path for the flink jar passed. Using the location of classorg.apache.flink.yarn.YarnClusterDescriptor to locate the jar 2026-07-01 21:06:18,095 WARN org.apache.flink.yarn.YarnClusterDescriptor [] - Job Clusters are deprecated since Flink 1.15. Please use an Application Cluster/Application Mode instead. 2026-07-01 21:06:18,174 INFO org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider [] - Failing over to yarnrm2 2026-07-01 21:06:18,250 INFO org.apache.hadoop.conf.Configuration [] - resource-types.xml not found 2026-07-01 21:06:18,250 INFO org.apache.hadoop.yarn.util.resource.ResourceUtils [] - Unable to find resource-types.xml. 2026-07-01 21:06:18,332 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - The configured JobManager memory is 1600 MB. YARN will allocate 2048 MB to make up an integer multiple of its minimum allocation memory (1024 MB, configured via yarn.scheduler.minimum-allocation-mb). The extra 448 MB may not be used by Flink. 2026-07-01 21:06:18,332 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - The configured TaskManager memory is 1728 MB. YARN will allocate 2048 MB to make up an integer multiple of its minimum allocation memory (1024 MB, configured via yarn.scheduler.minimum-allocation-mb). The extra 320 MBmay not be used by Flink. 2026-07-01 21:06:18,332 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - Cluster specification: ClusterSpecification{masterMemoryMB1600, taskManagerMemoryMB1728, slotsPerTaskManager1} 2026-07-01 21:06:25,891 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - Cannot use kerberos delegation token manager, no valid kerberos credentials provided. 2026-07-01 21:06:25,896 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - Submitting application master application_1782910361073_0002 2026-07-01 21:06:26,146 INFO org.apache.hadoop.yarn.client.api.impl.YarnClientImpl [] - Submitted application application_1782910361073_0002 2026-07-01 21:06:26,146 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - Waiting for the cluster to be allocated 2026-07-01 21:06:26,150 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - Deploying cluster, current state ACCEPTED 2026-07-01 21:06:32,916 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - YARN application has been deployed successfully. 2026-07-01 21:06:32,917 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - The Flink YARN session cluster has been started in detached mode. In order to stop Flink gracefully, use the following command: $ echo stop | ./bin/yarn-session.sh -id application_1782910361073_0002 If this should not be possible, then you can also kill Flink via YARNs web interface or via: $ yarn application -kill application_1782910361073_0002 Note that killing Flink might not clean up all job artifacts and temporary files. 2026-07-01 21:06:32,917 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - Found Web Interface hadoop001:30895 of application application_1782910361073_0002. Job has been submitted with JobID 88f7bb56d52dc5493de32c90b0498ea0 #查看作业 bin/flink list -t yarn-per-job -Dyarn.application.idapplication_1782910361073_0002#日志页面出现一个报错。Failed while trying to construct the redirect url to the log server. Log Server url may not be configuredLocal Logs:java.lang.Exception: Unknown container. Container either has not started or has already completed or doesnt belong to this node at all.#解决方法;未配置 jobhistory 的访问路径。vim yarn-site.xmlpropertynameyarn.log.server.url/namevaluehttp://hadoop001:19888/jobhistory/logs/value/property#查看作业 bin/flink list -t yarn-per-job -Dyarn.application.idapplication_1782912326958_0004 [flinkhadoop001 flink-1.17.0]$ bin/flink list -t yarn-per-job -Dyarn.application.idapplication_1782912326958_0004 SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/opt/module/flink-1.17.0/lib/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/opt/module/hadoop-3.3.6/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] 2026-07-01 21:35:25,159 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli [] - Found Yarn properties file under /tmp/.yarn-properties-flink. 2026-07-01 21:35:25,159 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli [] - Found Yarn properties file under /tmp/.yarn-properties-flink. 2026-07-01 21:35:25,315 WARN org.apache.flink.yarn.configuration.YarnLogConfigUtil [] - The configuration directory (/opt/module/flink-1.17.0/conf)already contains a LOG4J config file.If you want to use logback, then please delete or rename the log configuration file. 2026-07-01 21:35:25,527 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - No path for the flink jar passed. Using the location of classorg.apache.flink.yarn.YarnClusterDescriptor to locate the jar 2026-07-01 21:35:25,580 INFO org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider [] - Failing over to yarnrm2 2026-07-01 21:35:25,626 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - Found Web Interface hadoop002:18524 of application application_1782912326958_0004. Waiting for response... ------------------ Running/Restarting Jobs ------------------- 01.07.2026 21:34:58 : 1c60c35310ebcfe2a4601f40c1d6a0b4 : Flink Streaming Job (RUNNING) -------------------------------------------------------------- No scheduled jobs.socke 7777 端口写入数据flink消费#取消作业 -t:指定模式-D指定应用名称; bin/flink cancel -t yarn-per-job -Dyarn.application.idapplication_1782912326958_0004 1c60c35310ebcfe2a4601f40c1d6a0b4 [flinkhadoop001 flink-1.17.0]$ bin/flink cancel -t yarn-per-job -Dyarn.application.idapplication_1782912326958_0004 1c60c35310ebcfe2a4601f40c1d6a0b4 SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/opt/module/flink-1.17.0/lib/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/opt/module/hadoop-3.3.6/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] 2026-07-01 21:43:18,593 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli [] - Found Yarn properties file under /tmp/.yarn-properties-flink. 2026-07-01 21:43:18,593 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli [] - Found Yarn properties file under /tmp/.yarn-properties-flink. Cancelling job 1c60c35310ebcfe2a4601f40c1d6a0b4. 2026-07-01 21:43:18,761 WARN org.apache.flink.yarn.configuration.YarnLogConfigUtil [] - The configuration directory (/opt/module/flink-1.17.0/conf)already contains a LOG4J config file.If you want to use logback, then please delete or rename the log configuration file. 2026-07-01 21:43:19,039 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - No path for the flink jar passed. Using the location of classorg.apache.flink.yarn.YarnClusterDescriptor to locate the jar 2026-07-01 21:43:19,103 INFO org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider [] - Failing over to yarnrm2 2026-07-01 21:43:19,172 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - Found Web Interface hadoop002:18524 of application application_1782912326958_0004. Cancelled job 1c60c35310ebcfe2a4601f40c1d6a0b4.