Class CoreOptions


  • @PublicEvolving
    public class CoreOptions
    extends Object
    The set of configuration options for core parameters.
    • Field Detail

      • PARENT_FIRST_LOGGING_PATTERNS

        @Internal
        public static final String[] PARENT_FIRST_LOGGING_PATTERNS
      • CLASSLOADER_RESOLVE_ORDER

        public static final ConfigOption<String> CLASSLOADER_RESOLVE_ORDER
        Defines the class resolution strategy when loading classes from user code, meaning whether to first check the user code jar ("child-first") or the application classpath ( "parent-first")

        The default settings indicate to load classes first from the user code jar, which means that user code jars can include and load different dependencies than Flink uses (transitively).

        Exceptions to the rules are defined via ALWAYS_PARENT_FIRST_LOADER_PATTERNS.

      • ALWAYS_PARENT_FIRST_LOADER_PATTERNS

        public static final ConfigOption<List<String>> ALWAYS_PARENT_FIRST_LOADER_PATTERNS
        The namespace patterns for classes that are loaded with a preference from the parent classloader, meaning the application class path, rather than any user code jar file. This option only has an effect when CLASSLOADER_RESOLVE_ORDER is set to "child-first".

        It is important that all classes whose objects move between Flink's runtime and any user code (including Flink connectors that run as part of the user code) are covered by these patterns here. Otherwise, it is possible that the Flink runtime and the user code load two different copies of a class through the different class loaders. That leads to errors like "X cannot be cast to X" exceptions, where both class names are equal, or "X cannot be assigned to Y", where X should be a subclass of Y.

        The following classes are loaded parent-first, to avoid any duplication:

        • All core Java classes (java.*), because they must never be duplicated.
        • All core Scala classes (scala.*). Currently Scala is used in the Flink runtime and in the user code, and some Scala classes cross the boundary, such as the FunctionX classes. That may change if Scala eventually lives purely as part of the user code.
        • All Flink classes (org.apache.flink.*). Note that this means that connectors and formats (flink-avro, etc) are loaded parent-first as well if they are in the core classpath.
        • Java annotations and loggers, defined by the following list: javax.annotation;org.slf4j;org.apache.log4j;org.apache.logging;org.apache.commons.logging;ch.qos.logback. This is done for convenience, to avoid duplication of annotations and multiple log bindings.
      • ALWAYS_PARENT_FIRST_LOADER_PATTERNS_ADDITIONAL

        public static final ConfigOption<List<String>> ALWAYS_PARENT_FIRST_LOADER_PATTERNS_ADDITIONAL
      • FAIL_ON_USER_CLASS_LOADING_METASPACE_OOM

        public static final ConfigOption<Boolean> FAIL_ON_USER_CLASS_LOADING_METASPACE_OOM
      • PLUGIN_ALWAYS_PARENT_FIRST_LOADER_PATTERNS_ADDITIONAL

        public static final ConfigOption<List<String>> PLUGIN_ALWAYS_PARENT_FIRST_LOADER_PATTERNS_ADDITIONAL
      • FLINK_SQL_GATEWAY_JVM_OPTIONS

        public static final ConfigOption<String> FLINK_SQL_GATEWAY_JVM_OPTIONS
      • FLINK_DEFAULT_JVM_OPTIONS

        public static final ConfigOption<String> FLINK_DEFAULT_JVM_OPTIONS
      • FLINK_DEFAULT_JM_JVM_OPTIONS

        public static final ConfigOption<String> FLINK_DEFAULT_JM_JVM_OPTIONS
      • FLINK_DEFAULT_TM_JVM_OPTIONS

        public static final ConfigOption<String> FLINK_DEFAULT_TM_JVM_OPTIONS
      • FLINK_LOG_DIR

        public static final ConfigOption<String> FLINK_LOG_DIR
        This option is here only for documentation generation, it is only evaluated in the shell scripts.
      • FLINK_PID_DIR

        public static final ConfigOption<String> FLINK_PID_DIR
        The config parameter defining the directory for Flink PID file. see: bin/config.sh#KEY_ENV_PID_DIR and bin/config.sh#DEFAULT_ENV_PID_DIR
      • FLINK_LOG_MAX

        public static final ConfigOption<Integer> FLINK_LOG_MAX
        This option is here only for documentation generation, it is only evaluated in the shell scripts.
      • FLINK_LOG_LEVEL

        public static final ConfigOption<String> FLINK_LOG_LEVEL
        This option is here only for documentation generation, it is only evaluated in the shell scripts.
      • FLINK_STD_REDIRECT_TO_FILE

        public static final ConfigOption<Boolean> FLINK_STD_REDIRECT_TO_FILE
        This option is here only for documentation generation, it is only evaluated in the shell scripts.
      • FLINK_SSH_OPTIONS

        public static final ConfigOption<String> FLINK_SSH_OPTIONS
        This option is here only for documentation generation, it is only evaluated in the shell scripts.
      • FLINK_HADOOP_CONF_DIR

        public static final ConfigOption<String> FLINK_HADOOP_CONF_DIR
        This option is here only for documentation generation, it is only evaluated in the shell scripts.
      • FLINK_YARN_CONF_DIR

        public static final ConfigOption<String> FLINK_YARN_CONF_DIR
        This option is here only for documentation generation, it is only evaluated in the shell scripts.
      • FLINK_HBASE_CONF_DIR

        public static final ConfigOption<String> FLINK_HBASE_CONF_DIR
        This option is here only for documentation generation, it is only evaluated in the shell scripts.
      • TMP_DIRS

        public static final ConfigOption<String> TMP_DIRS
        The config parameter defining the directories for temporary files, separated by ",", "|", or the system's File.pathSeparator.
      • DEFAULT_FILESYSTEM_SCHEME

        public static final ConfigOption<String> DEFAULT_FILESYSTEM_SCHEME
        The default filesystem scheme, used for paths that do not declare a scheme explicitly.
      • ALLOWED_FALLBACK_FILESYSTEMS

        public static final ConfigOption<String> ALLOWED_FALLBACK_FILESYSTEMS
      • FILESYTEM_DEFAULT_OVERRIDE

        public static final ConfigOption<Boolean> FILESYTEM_DEFAULT_OVERRIDE
        Specifies whether file output writers should overwrite existing files by default.
      • FILESYSTEM_OUTPUT_ALWAYS_CREATE_DIRECTORY

        public static final ConfigOption<Boolean> FILESYSTEM_OUTPUT_ALWAYS_CREATE_DIRECTORY
        Specifies whether the file systems should always create a directory for the output, even with a parallelism of one.
    • Constructor Detail

      • CoreOptions

        public CoreOptions()
    • Method Detail

      • getParentFirstLoaderPatterns

        public static String[] getParentFirstLoaderPatterns​(ReadableConfig config)
      • getPluginParentFirstLoaderPatterns

        public static String[] getPluginParentFirstLoaderPatterns​(Configuration config)
      • fileSystemConnectionLimit

        public static ConfigOption<Integer> fileSystemConnectionLimit​(String scheme)
        The total number of input plus output connections that a file system for the given scheme may open. Unlimited be default.
      • fileSystemConnectionLimitIn

        public static ConfigOption<Integer> fileSystemConnectionLimitIn​(String scheme)
        The total number of input connections that a file system for the given scheme may open. Unlimited be default.
      • fileSystemConnectionLimitOut

        public static ConfigOption<Integer> fileSystemConnectionLimitOut​(String scheme)
        The total number of output connections that a file system for the given scheme may open. Unlimited be default.
      • fileSystemConnectionLimitTimeout

        public static ConfigOption<Long> fileSystemConnectionLimitTimeout​(String scheme)
        If any connection limit is configured, this option can be optionally set to define after which time (in milliseconds) stream opening fails with a timeout exception, if no stream connection becomes available. Unlimited timeout be default.
      • fileSystemConnectionLimitStreamInactivityTimeout

        public static ConfigOption<Long> fileSystemConnectionLimitStreamInactivityTimeout​(String scheme)
        If any connection limit is configured, this option can be optionally set to define after which time (in milliseconds) inactive streams are reclaimed. This option can help to prevent that inactive streams make up the full pool of limited connections, and no further connections can be established. Unlimited timeout be default.