Class CoreOptions
- java.lang.Object
-
- org.apache.flink.configuration.CoreOptions
-
@PublicEvolving public class CoreOptions extends Object
The set of configuration options for core parameters.
-
-
Field Summary
Fields Modifier and Type Field Description static ConfigOption<String>ALLOWED_FALLBACK_FILESYSTEMSstatic ConfigOption<List<String>>ALWAYS_PARENT_FIRST_LOADER_PATTERNSThe namespace patterns for classes that are loaded with a preference from the parent classloader, meaning the application class path, rather than any user code jar file.static ConfigOption<List<String>>ALWAYS_PARENT_FIRST_LOADER_PATTERNS_ADDITIONALstatic ConfigOption<Boolean>CHECK_LEAKED_CLASSLOADERstatic ConfigOption<String>CLASSLOADER_RESOLVE_ORDERDefines the class resolution strategy when loading classes from user code, meaning whether to first check the user code jar ("child-first") or the application classpath ("parent-first")static ConfigOption<String>DEFAULT_FILESYSTEM_SCHEMEThe default filesystem scheme, used for paths that do not declare a scheme explicitly.static ConfigOption<Integer>DEFAULT_PARALLELISMstatic ConfigOption<Boolean>FAIL_ON_USER_CLASS_LOADING_METASPACE_OOMstatic ConfigOption<Boolean>FILESYSTEM_OUTPUT_ALWAYS_CREATE_DIRECTORYSpecifies whether the file systems should always create a directory for the output, even with a parallelism of one.static ConfigOption<Boolean>FILESYTEM_DEFAULT_OVERRIDESpecifies whether file output writers should overwrite existing files by default.static ConfigOption<String>FLINK_CLI_JVM_OPTIONSstatic ConfigOption<String>FLINK_DEFAULT_JM_JVM_OPTIONSstatic ConfigOption<String>FLINK_DEFAULT_JVM_OPTIONSstatic ConfigOption<String>FLINK_DEFAULT_TM_JVM_OPTIONSstatic ConfigOption<String>FLINK_HADOOP_CONF_DIRThis option is here only for documentation generation, it is only evaluated in the shell scripts.static ConfigOption<String>FLINK_HBASE_CONF_DIRThis option is here only for documentation generation, it is only evaluated in the shell scripts.static ConfigOption<String>FLINK_HS_JVM_OPTIONSstatic ConfigOption<String>FLINK_JAVA_HOMEstatic ConfigOption<String>FLINK_JM_JVM_OPTIONSstatic ConfigOption<String>FLINK_JVM_OPTIONSstatic ConfigOption<String>FLINK_LOG_DIRThis option is here only for documentation generation, it is only evaluated in the shell scripts.static ConfigOption<String>FLINK_LOG_LEVELThis option is here only for documentation generation, it is only evaluated in the shell scripts.static ConfigOption<Integer>FLINK_LOG_MAXThis option is here only for documentation generation, it is only evaluated in the shell scripts.static ConfigOption<String>FLINK_PID_DIRThe config parameter defining the directory for Flink PID file. see:bin/config.sh#KEY_ENV_PID_DIRandbin/config.sh#DEFAULT_ENV_PID_DIRstatic ConfigOption<String>FLINK_SQL_GATEWAY_JVM_OPTIONSstatic ConfigOption<String>FLINK_SSH_OPTIONSThis option is here only for documentation generation, it is only evaluated in the shell scripts.static ConfigOption<Boolean>FLINK_STD_REDIRECT_TO_FILEThis option is here only for documentation generation, it is only evaluated in the shell scripts.static ConfigOption<String>FLINK_TM_JVM_OPTIONSstatic ConfigOption<String>FLINK_YARN_CONF_DIRThis option is here only for documentation generation, it is only evaluated in the shell scripts.static String[]PARENT_FIRST_LOGGING_PATTERNSstatic ConfigOption<List<String>>PLUGIN_ALWAYS_PARENT_FIRST_LOADER_PATTERNSPlugin-specific option ofALWAYS_PARENT_FIRST_LOADER_PATTERNS.static ConfigOption<List<String>>PLUGIN_ALWAYS_PARENT_FIRST_LOADER_PATTERNS_ADDITIONALstatic ConfigOption<String>TMP_DIRSThe config parameter defining the directories for temporary files, separated by ",", "|", or the system'sFile.pathSeparator.
-
Constructor Summary
Constructors Constructor Description CoreOptions()
-
Method Summary
All Methods Static Methods Concrete Methods Modifier and Type Method Description static ConfigOption<Integer>fileSystemConnectionLimit(String scheme)The total number of input plus output connections that a file system for the given scheme may open.static ConfigOption<Integer>fileSystemConnectionLimitIn(String scheme)The total number of input connections that a file system for the given scheme may open.static ConfigOption<Integer>fileSystemConnectionLimitOut(String scheme)The total number of output connections that a file system for the given scheme may open.static ConfigOption<Long>fileSystemConnectionLimitStreamInactivityTimeout(String scheme)If any connection limit is configured, this option can be optionally set to define after which time (in milliseconds) inactive streams are reclaimed.static ConfigOption<Long>fileSystemConnectionLimitTimeout(String scheme)If any connection limit is configured, this option can be optionally set to define after which time (in milliseconds) stream opening fails with a timeout exception, if no stream connection becomes available.static String[]getParentFirstLoaderPatterns(ReadableConfig config)static String[]getPluginParentFirstLoaderPatterns(Configuration config)static String[]mergeListsToArray(List<String> base, List<String> append)
-
-
-
Field Detail
-
PARENT_FIRST_LOGGING_PATTERNS
@Internal public static final String[] PARENT_FIRST_LOGGING_PATTERNS
-
CLASSLOADER_RESOLVE_ORDER
public static final ConfigOption<String> CLASSLOADER_RESOLVE_ORDER
Defines the class resolution strategy when loading classes from user code, meaning whether to first check the user code jar ("child-first") or the application classpath ("parent-first")The default settings indicate to load classes first from the user code jar, which means that user code jars can include and load different dependencies than Flink uses (transitively).
Exceptions to the rules are defined via
ALWAYS_PARENT_FIRST_LOADER_PATTERNS.
-
ALWAYS_PARENT_FIRST_LOADER_PATTERNS
public static final ConfigOption<List<String>> ALWAYS_PARENT_FIRST_LOADER_PATTERNS
The namespace patterns for classes that are loaded with a preference from the parent classloader, meaning the application class path, rather than any user code jar file. This option only has an effect whenCLASSLOADER_RESOLVE_ORDERis set to"child-first".It is important that all classes whose objects move between Flink's runtime and any user code (including Flink connectors that run as part of the user code) are covered by these patterns here. Otherwise, it is possible that the Flink runtime and the user code load two different copies of a class through the different class loaders. That leads to errors like "X cannot be cast to X" exceptions, where both class names are equal, or "X cannot be assigned to Y", where X should be a subclass of Y.
The following classes are loaded parent-first, to avoid any duplication:
- All core Java classes (java.*), because they must never be duplicated.
- All core Scala classes (scala.*). Currently Scala is used in the Flink runtime and in the user code, and some Scala classes cross the boundary, such as the FunctionX classes. That may change if Scala eventually lives purely as part of the user code.
- All Flink classes (org.apache.flink.*). Note that this means that connectors and formats (flink-avro, etc) are loaded parent-first as well if they are in the core classpath.
- Java annotations and loggers, defined by the following list: javax.annotation;org.slf4j;org.apache.log4j;org.apache.logging;org.apache.commons.logging;ch.qos.logback. This is done for convenience, to avoid duplication of annotations and multiple log bindings.
-
ALWAYS_PARENT_FIRST_LOADER_PATTERNS_ADDITIONAL
public static final ConfigOption<List<String>> ALWAYS_PARENT_FIRST_LOADER_PATTERNS_ADDITIONAL
-
FAIL_ON_USER_CLASS_LOADING_METASPACE_OOM
public static final ConfigOption<Boolean> FAIL_ON_USER_CLASS_LOADING_METASPACE_OOM
-
CHECK_LEAKED_CLASSLOADER
public static final ConfigOption<Boolean> CHECK_LEAKED_CLASSLOADER
-
PLUGIN_ALWAYS_PARENT_FIRST_LOADER_PATTERNS
public static final ConfigOption<List<String>> PLUGIN_ALWAYS_PARENT_FIRST_LOADER_PATTERNS
Plugin-specific option ofALWAYS_PARENT_FIRST_LOADER_PATTERNS. Plugins use this parent first list instead of the global version.
-
PLUGIN_ALWAYS_PARENT_FIRST_LOADER_PATTERNS_ADDITIONAL
public static final ConfigOption<List<String>> PLUGIN_ALWAYS_PARENT_FIRST_LOADER_PATTERNS_ADDITIONAL
-
FLINK_JAVA_HOME
public static final ConfigOption<String> FLINK_JAVA_HOME
-
FLINK_JVM_OPTIONS
public static final ConfigOption<String> FLINK_JVM_OPTIONS
-
FLINK_JM_JVM_OPTIONS
public static final ConfigOption<String> FLINK_JM_JVM_OPTIONS
-
FLINK_TM_JVM_OPTIONS
public static final ConfigOption<String> FLINK_TM_JVM_OPTIONS
-
FLINK_HS_JVM_OPTIONS
public static final ConfigOption<String> FLINK_HS_JVM_OPTIONS
-
FLINK_CLI_JVM_OPTIONS
public static final ConfigOption<String> FLINK_CLI_JVM_OPTIONS
-
FLINK_SQL_GATEWAY_JVM_OPTIONS
public static final ConfigOption<String> FLINK_SQL_GATEWAY_JVM_OPTIONS
-
FLINK_DEFAULT_JVM_OPTIONS
public static final ConfigOption<String> FLINK_DEFAULT_JVM_OPTIONS
-
FLINK_DEFAULT_JM_JVM_OPTIONS
public static final ConfigOption<String> FLINK_DEFAULT_JM_JVM_OPTIONS
-
FLINK_DEFAULT_TM_JVM_OPTIONS
public static final ConfigOption<String> FLINK_DEFAULT_TM_JVM_OPTIONS
-
FLINK_LOG_DIR
public static final ConfigOption<String> FLINK_LOG_DIR
This option is here only for documentation generation, it is only evaluated in the shell scripts.
-
FLINK_PID_DIR
public static final ConfigOption<String> FLINK_PID_DIR
The config parameter defining the directory for Flink PID file. see:bin/config.sh#KEY_ENV_PID_DIRandbin/config.sh#DEFAULT_ENV_PID_DIR
-
FLINK_LOG_MAX
public static final ConfigOption<Integer> FLINK_LOG_MAX
This option is here only for documentation generation, it is only evaluated in the shell scripts.
-
FLINK_LOG_LEVEL
public static final ConfigOption<String> FLINK_LOG_LEVEL
This option is here only for documentation generation, it is only evaluated in the shell scripts.
-
FLINK_STD_REDIRECT_TO_FILE
public static final ConfigOption<Boolean> FLINK_STD_REDIRECT_TO_FILE
This option is here only for documentation generation, it is only evaluated in the shell scripts.
-
FLINK_SSH_OPTIONS
public static final ConfigOption<String> FLINK_SSH_OPTIONS
This option is here only for documentation generation, it is only evaluated in the shell scripts.
-
FLINK_HADOOP_CONF_DIR
public static final ConfigOption<String> FLINK_HADOOP_CONF_DIR
This option is here only for documentation generation, it is only evaluated in the shell scripts.
-
FLINK_YARN_CONF_DIR
public static final ConfigOption<String> FLINK_YARN_CONF_DIR
This option is here only for documentation generation, it is only evaluated in the shell scripts.
-
FLINK_HBASE_CONF_DIR
public static final ConfigOption<String> FLINK_HBASE_CONF_DIR
This option is here only for documentation generation, it is only evaluated in the shell scripts.
-
TMP_DIRS
public static final ConfigOption<String> TMP_DIRS
The config parameter defining the directories for temporary files, separated by ",", "|", or the system'sFile.pathSeparator.
-
DEFAULT_PARALLELISM
public static final ConfigOption<Integer> DEFAULT_PARALLELISM
-
DEFAULT_FILESYSTEM_SCHEME
public static final ConfigOption<String> DEFAULT_FILESYSTEM_SCHEME
The default filesystem scheme, used for paths that do not declare a scheme explicitly.
-
ALLOWED_FALLBACK_FILESYSTEMS
public static final ConfigOption<String> ALLOWED_FALLBACK_FILESYSTEMS
-
FILESYTEM_DEFAULT_OVERRIDE
public static final ConfigOption<Boolean> FILESYTEM_DEFAULT_OVERRIDE
Specifies whether file output writers should overwrite existing files by default.
-
FILESYSTEM_OUTPUT_ALWAYS_CREATE_DIRECTORY
public static final ConfigOption<Boolean> FILESYSTEM_OUTPUT_ALWAYS_CREATE_DIRECTORY
Specifies whether the file systems should always create a directory for the output, even with a parallelism of one.
-
-
Method Detail
-
getParentFirstLoaderPatterns
public static String[] getParentFirstLoaderPatterns(ReadableConfig config)
-
getPluginParentFirstLoaderPatterns
public static String[] getPluginParentFirstLoaderPatterns(Configuration config)
-
mergeListsToArray
@Internal public static String[] mergeListsToArray(List<String> base, List<String> append)
-
fileSystemConnectionLimit
public static ConfigOption<Integer> fileSystemConnectionLimit(String scheme)
The total number of input plus output connections that a file system for the given scheme may open. Unlimited be default.
-
fileSystemConnectionLimitIn
public static ConfigOption<Integer> fileSystemConnectionLimitIn(String scheme)
The total number of input connections that a file system for the given scheme may open. Unlimited be default.
-
fileSystemConnectionLimitOut
public static ConfigOption<Integer> fileSystemConnectionLimitOut(String scheme)
The total number of output connections that a file system for the given scheme may open. Unlimited be default.
-
fileSystemConnectionLimitTimeout
public static ConfigOption<Long> fileSystemConnectionLimitTimeout(String scheme)
If any connection limit is configured, this option can be optionally set to define after which time (in milliseconds) stream opening fails with a timeout exception, if no stream connection becomes available. Unlimited timeout be default.
-
fileSystemConnectionLimitStreamInactivityTimeout
public static ConfigOption<Long> fileSystemConnectionLimitStreamInactivityTimeout(String scheme)
If any connection limit is configured, this option can be optionally set to define after which time (in milliseconds) inactive streams are reclaimed. This option can help to prevent that inactive streams make up the full pool of limited connections, and no further connections can be established. Unlimited timeout be default.
-
-