Package org.apache.flink.streaming.api.functions.sink.legacy

Class TwoPhaseCommitSinkFunction<IN,​TXN,​CONTEXT>

  • Type Parameters:
    IN - Input type for SinkFunction.
    TXN - Transaction to store all of the information required to handle a transaction.
    CONTEXT - Context that will be shared across all invocations for the given TwoPhaseCommitSinkFunction instance. Context is created once
    All Implemented Interfaces:
    Serializable, org.apache.flink.api.common.functions.Function, org.apache.flink.api.common.functions.RichFunction, org.apache.flink.api.common.state.CheckpointListener, org.apache.flink.streaming.api.checkpoint.CheckpointedFunction, org.apache.flink.streaming.api.functions.sink.legacy.SinkFunction<IN>
    @Internal
public abstract class TwoPhaseCommitSinkFunction<IN,​TXN,​CONTEXT>
extends org.apache.flink.streaming.api.functions.sink.legacy.RichSinkFunction<IN>
implements org.apache.flink.streaming.api.checkpoint.CheckpointedFunction, org.apache.flink.api.common.state.CheckpointListener
    Deprecated.
    This interface will be removed in future versions. Use the new Sink interface instead.
    This is a recommended base class for all of the SinkFunction that intend to implement exactly-once semantic. It does that by implementing two phase commit algorithm on top of the CheckpointedFunction and CheckpointListener. User should provide custom TXN (transaction handle) and implement abstract methods handling this transaction handle.
    See Also:
    Serialized Form

    • Constructor Detail

      • TwoPhaseCommitSinkFunction

        public TwoPhaseCommitSinkFunction​(org.apache.flink.api.common.typeutils.TypeSerializer<TXN> transactionSerializer,
                                  org.apache.flink.api.common.typeutils.TypeSerializer<CONTEXT> contextSerializer)
        Deprecated.
        Use default ListStateDescriptor for internal state serialization. Helpful utilities for using this constructor are TypeInformation.of(Class), TypeHint and TypeInformation.of(TypeHint). Example: 
        
 TwoPhaseCommitSinkFunction(TypeInformation.of(new TypeHint<State<TXN, CONTEXT>>() {}));
        Parameters:
        transactionSerializer - TypeSerializer for the transaction type of this sink
        contextSerializer - TypeSerializer for the context type of this sink

    • Method Detail

      • initializeUserContext

        protected Optional<CONTEXT> initializeUserContext()
        Deprecated.

      • getUserContext

        protected Optional<CONTEXT> getUserContext()
        Deprecated.

      • currentTransaction

        @Nullable
protected TXN currentTransaction()
        Deprecated.

      • invoke

        protected abstract void invoke​(TXN transaction,
                               IN value,
                               org.apache.flink.streaming.api.functions.sink.legacy.SinkFunction.Context context)
                        throws Exception
        Deprecated.
        Write value within a transaction.
        Throws:
        Exception

      • beginTransaction

        protected abstract TXN beginTransaction()
                                 throws Exception
        Deprecated.
        Method that starts a new transaction.
        Returns:
        newly created transaction.
        Throws:
        Exception

      • preCommit

        protected abstract void preCommit​(TXN transaction)
                           throws Exception
        Deprecated.
        Pre commit previously created transaction. Pre commit must make all of the necessary steps to prepare the transaction for a commit that might happen in the future. After this point the transaction might still be aborted, but underlying implementation must ensure that commit calls on already pre committed transactions will always succeed.

        Usually implementation involves flushing the data.

        Throws:
        Exception

      • commit

        protected abstract void commit​(TXN transaction)
        Deprecated.
        Commit a pre-committed transaction. If this method fail, Flink application will be restarted and recoverAndCommit(Object) will be called again for the same transaction.

      • recoverAndCommit

        protected void recoverAndCommit​(TXN transaction)
        Deprecated.
        Invoked on recovered transactions after a failure. User implementation must ensure that this call will eventually succeed. If it fails, Flink application will be restarted and it will be invoked again. If it does not succeed eventually, a data loss will occur. Transactions will be recovered in an order in which they were created.

      • abort

        protected abstract void abort​(TXN transaction)
        Deprecated.
        Abort a transaction.

      • recoverAndAbort

        protected void recoverAndAbort​(TXN transaction)
        Deprecated.
        Abort a transaction that was rejected by a coordinator after a failure.

      • finishRecoveringContext

        protected void finishRecoveringContext​(Collection<TXN> handledTransactions)
        Deprecated.
        Callback for subclasses which is called after restoring (each) user context.
        Parameters:
        handledTransactions - transactions which were already committed or aborted and do not need further handling

      • finishProcessing

        protected void finishProcessing​(@Nullable
                                TXN transaction)
        Deprecated.
        This method is called at the end of data processing.

        The method is expected to flush all remaining buffered data. Exceptions will cause the pipeline to be recognized as failed, because the last data items are not processed properly. You may use this method to flush remaining buffered elements in the state into the current transaction which will be committed in the last checkpoint.

      • invoke

        public final void invoke​(IN value)
                  throws Exception
        Deprecated.
        This should not be implemented by subclasses.
        Specified by:
        invoke in interface org.apache.flink.streaming.api.functions.sink.legacy.SinkFunction<IN>
        Throws:
        Exception

      • invoke

        public final void invoke​(IN value,
                         org.apache.flink.streaming.api.functions.sink.legacy.SinkFunction.Context context)
                  throws Exception
        Deprecated.
        Specified by:
        invoke in interface org.apache.flink.streaming.api.functions.sink.legacy.SinkFunction<IN>
        Throws:
        Exception

      • finish

        public final void finish()
                  throws Exception
        Deprecated.
        Specified by:
        finish in interface org.apache.flink.streaming.api.functions.sink.legacy.SinkFunction<IN>
        Throws:
        Exception

      • notifyCheckpointComplete

        public final void notifyCheckpointComplete​(long checkpointId)
                                    throws Exception
        Deprecated.
        Specified by:
        notifyCheckpointComplete in interface org.apache.flink.api.common.state.CheckpointListener
        Throws:
        Exception

      • notifyCheckpointAborted

        public void notifyCheckpointAborted​(long checkpointId)
        Deprecated.
        Specified by:
        notifyCheckpointAborted in interface org.apache.flink.api.common.state.CheckpointListener

      • snapshotState

        public void snapshotState​(org.apache.flink.runtime.state.FunctionSnapshotContext context)
                   throws Exception
        Deprecated.
        Specified by:
        snapshotState in interface org.apache.flink.streaming.api.checkpoint.CheckpointedFunction
        Throws:
        Exception

      • initializeState

        public void initializeState​(org.apache.flink.runtime.state.FunctionInitializationContext context)
                     throws Exception
        Deprecated.
        Specified by:
        initializeState in interface org.apache.flink.streaming.api.checkpoint.CheckpointedFunction
        Throws:
        Exception

      • close

        public void close()
           throws Exception
        Deprecated.
        Specified by:
        close in interface org.apache.flink.api.common.functions.RichFunction
        Overrides:
        close in class org.apache.flink.api.common.functions.AbstractRichFunction
        Throws:
        Exception

      • enableTransactionTimeoutWarnings

        protected TwoPhaseCommitSinkFunction<IN,​TXN,​CONTEXT> enableTransactionTimeoutWarnings​(double warningRatio)
        Deprecated.
        Enables logging of warnings if a transaction's elapsed time reaches a specified ratio of the transactionTimeout. If warningRatio is 0, a warning will be always logged when committing the transaction.
        Parameters:
        warningRatio - A value in the range [0,1].
        Returns: