Apache Ignite C++

The apacheignite-cpp Developer Hub

Welcome to the apacheignite-cpp developer hub. You'll find comprehensive guides and documentation to help you start working with apacheignite-cpp as quickly as possible, as well as support if you get stuck. Let's jump right in!

Get Started    

Transactions

ACID compliant transactions ensuring guaranteed consistency.

Ignite supports 2 modes for cache operation, transactional and atomic. In transactional mode you are able to group multiple cache operations in a transaction, while atomic mode supports multiple atomic operations, one at a time. Atomic mode is more light-weight and generally has better performance over transactional caches.

However, regardless of which mode you use, as long as your cluster is alive, the data between different cluster nodes must remain consistent. This means that whichever node is being used to retrieve data, it will never get data that has been partially committed or that is inconsistent with other data.

Transactions

Transactions class contains functionality for starting and completing transactions, as well as subscribing listeners or getting metrics.

Cross-Cache Transactions

You can combine multiple operations from different caches into one transaction. Note that this allows to update caches of different types, like REPLICATED and PARTITIONED caches, in one transaction.

You can obtain an instance of Transactions as follows:

Ignite ignite = Ignition::Get();

Transactions transactions = ignite.GetTransactions();

Here is an example of how transactions can be performed in Ignite:

Transaction tx = transactions.TxStart();

int hello = cache.Get("Hello");

if (hello == 1)
  cache.Put("Hello", 11);

cache.Put("World", 22);

tx.Commit();

Two-Phase-Commit (2PC)

Ignite utilizes 2PC protocol for its transactions with many one-phase-commit optimizations whenever applicable. Whenever data is updated within a transaction, Ignite will keep transactional state in a local transaction map until commit() is called, at which point, if needed, the data is transferred to participating remote nodes.

For more information on how Ignite 2PC works, you can check out these blogs:

ACID Compliance

Ignite provides fully ACID (Atomicity, Consistency, Isolation, Durability) compliant transactions that ensure guaranteed consistency.

Concurrency Modes and Isolation Levels

Whenever TRANSACTIONAL atomicity mode is configured, Ignite supports TransactionConcurrency::OPTIMISTIC and TransactionConcurrency::PESSIMISTIC concurrency modes for transactions. Concurrency level determines when an entry-level transaction lock should be acquired - at the time of data access or during the prepare phase. Locking prevents concurrent access to an object. For example, when you attempt to update a ToDo list item with pessimistic locking, the server places a lock on the object until you either commit or rollback the transaction so that no other transaction or operation is allowed to update the same entry. Regardless of the concurrency level used in a transaction, there exists a moment in time when all entries enlisted in the transaction are locked before the commit.
Isolation level defines how concurrent transactions will 'see' and handle operations on the same keys. Ignite supports TransactionIsolation::READ_COMMITTED, TransactionIsolation::REPEATABLE_READ and TransactionIsolation::SERIALIZABLE isolation levels.
All combinations of concurrency modes and isolation levels can be used simultaneously. Below is the description of Ignite behavior and guarantees provided by each concurrency-isolation combination.

Pessimistic Transactions

In TransactionConcurrency::PESSIMISTIC transactions, locks are acquired during the first read or write access (depending on the isolation level) and held by the transaction until it is committed or rolled back. In this mode locks are acquired on primary nodes first and then promoted to backup nodes during the prepare stage. The following isolation levels can be configured with TransactionConcurrency::PESSIMISTIC concurrency mode:

  • TransactionIsolation::READ_COMMITTED - Data is read without a lock and is never cached in the transaction itself. The data may be read from a backup node if this is allowed in the cache configuration. In this isolation you can have the so-called Non-Repeatable Reads because a concurrent transaction can change the data when you are reading the data twice in your transaction. The lock is only acquired at the time of first write access. This means that an entry that have been read during the transaction may have a different value by the time the transaction is committed. No exception will be thrown in this case.

  • TransactionIsolation::REPEATABLE_READ - Entry lock is acquired and data is fetched from the primary node on the first read or write access and stored in the local transactional map. All consecutive access to the same data is local and will return the last read or updated transaction value. This means no other concurrent transactions can make changes to the locked data, and you are getting Repeatable Reads for your transaction.

  • TransactionIsolation::SERIALIZABLE - In TransactionConcurrency::PESSIMISTIC mode, this isolation level works the same way as TransactionIsolation::REPEATABLE_READ.

Note that in TransactionConcurrency::PESSIMISTIC mode, the order of locking is important. Moreover, Ignite will acquire locks sequentially and exactly in the order provided by a user.

Performance Considerations

Imagine that you have 3 nodes in your topology (A, B, C) and in your transaction you are doing a putAll for keys [1, 2, 3, 4, 5, 6]. Suppose that these keys are mapped to nodes in the following fashion: {A: 1, 4}, {B: 2, 5}, {C: 3, 6}. Since Ignite cannot re-arrange the lock acquisition order in TransactionConcurrency::PESSIMISTIC mode, it will have to make 6 sequential network round-trips: [A, B, C, A, B, C]. In a case when the key locking order is not important for the semantics of a transaction, it is advisable to group keys by partition and lock keys within the same partition together. This may significantly reduce the number of network messages in a large transaction. In this example, if keys were ordered for a putAll in the following way: [1, 4, 2, 5, 3, 6], then only 3 sequential round-trips would be required.

Topology Change Restrictions

Note that if at least one TransactionConcurrency::PESSIMISTIC transaction lock is acquired, it will be impossible to change the cache topology until the transaction is committed or rolled back. Therefore, it is not recommended to hold transaction locks for a long period of time.

Deadlock Detection in Pessimistic Transactions

One major rule that anyone has to follow when working with distributed pessimistic transactions is that locks for keys, participating in a transaction, must be acquired in a similar order. If this rule is violated at some point of time this may lead to a distributed deadlock.

Ignite doesn't avoid distributed deadlocks but rather has built-in functionality that makes it easier to debug and fix such situations.

As the code snippet below shows a pessimistic transaction has to be started with a timeout and if the timeout expires then the deadlock detection mechanism will try to find a possible deadlock that might have caused the timeout. When timeout expires TransactionTimeoutException is generated and propagated to application code as the cause of CacheException regardless of a deadlock. However, if a deadlock is detected then the cause of returned TransactionTimeoutException will be TransactionDeadlockException (at least for one transaction involved in the deadlock). In Ignite C++ these exceptions are going to be propagated as an IgniteError.

try {
	Transaction tx = ignite.GetTransactions().TxStart(
    TransactionConcurrency::PESSIMISTIC, TransactionIsolation::READ_COMMITTED, 300, 0);

	cache.Put(1, 1);
	cache.Put(2, 1);

	tx.Commit();
}
catch (IgniteError e) {
  std::cout << e.GetText() << std::endl;
}

The message that is a part of TransactionDeadlockException contains useful information that will help to find out a reason of a deadlock

Deadlock detected:

K1: TX1 holds lock, TX2 waits lock.
K2: TX2 holds lock, TX1 waits lock.

Transactions:

TX1 [txId=GridCacheVersion [topVer=74949328, time=1463469328421, order=1463469326211, nodeOrder=1], nodeId=ad68354d-07b8-4be5-85bb-f5f2362fbb88, threadId=73]
TX2 [txId=GridCacheVersion [topVer=74949328, time=1463469328421, order=1463469326210, nodeOrder=1], nodeId=ad68354d-07b8-4be5-85bb-f5f2362fbb88, threadId=74]

Keys:

K1 [key=1, cache=default]
K2 [key=2, cache=default]

If you want to avoid deadlocks at all refer to Optimistic Transactions and Deadlock-free Transactions sections below.

Optimistic Transactions

In TransactionConcurrency::OPTIMISTIC transactions, entry locks are acquired on primary nodes during the prepare step, then promoted to backup nodes and released once the transaction is committed. The locks are never acquired if the transaction is rolled back by user and no commit attempt was made. The following isolation levels can be configured with TransactionConcurrency::OPTIMISTIC concurrency mode:

  • TransactionIsolation::READ_COMMITTED - Changes that should be applied to the cache are collected on the originating node and applied upon the transaction commit. Transaction data is read without a lock and is never cached in the transaction. The data may be read from a backup node if this is allowed in the cache configuration. In this isolation you can have so-called Non-Repeatable Reads because a concurrent transaction can change the data when you are reading the data twice in your transaction. This mode combination does not check if the entry value has been modified since the first read or write access and never raises an optimistic exception.

  • TransactionIsolation::REPEATABLE_READ - Transactions at this isolation level work similar to TransactionConcurrency::OPTIMISTIC TransactionIsolation::READ_COMMITTED transactions with only one difference - read values are cached on the originating node and all subsequent reads are guaranteed to be local. This mode combination does not check if the entry value has been modified since the first read or write access and never raises an optimistic exception.

  • TransactionIsolation::SERIALIZABLE - Stores an entry version upon first read access. Ignite will fail a transaction at the commit stage if the Ignite engine detects that at least one of the entries used as part of the initiated transaction has been modified. This is achieved by internally checking the version of an entry remembered in a transaction to the one actually in the grid at the time of commit. In short, this means that if Ignite detects that there is a conflict at the commit stage of a transaction, we fail such a transaction throwing TransactionOptimisticException, which is propagated as an IgniteError in Ignite C++, and rolling back any changes made. User should handle this exception and retry the transaction.

IgniteTransactions txs = ignite.GetTransactions();

// Start transaction in optimistic mode with serializable isolation level.
while (true) {
    try {
      Transaction tx = txs.TxStart(TransactionConcurrency::OPTIMISTIC,
                                   TransactionIsolation::SERIALIZABLE);

  		// Modify cache entires as part of this transacation.
  		....
        
  		// Commit transaction.  
  		tx.Commit();

      // Transaction succeeded. Leave the while loop.
      break;
    }
    catch (IgniteError e) {
    	// Transaction has failed. Retry.
    }
}

Another important point to note here is that a transaction will still fail even if an entry that was simply read (with no modify, cache.Put(...)) since the value of the entry could be important to the logic within the initiated transaction.

Note that the key order is important for TransactionIsolation::READ_COMMITTED and TransactionIsolation::REPEATABLE_READ transactions since the locks are still acquired sequentially in these modes.

Deadlock-free Transactions

For TransactionConcurrency::OPTIMISTIC TransactionIsolation::SERIALIZABLE transactions locks are not acquired sequentially. In this mode keys can be accessed in any order because transaction locks are acquired in parallel with an additional check allowing Ignite to avoid deadlocks.
We need to introduce some concepts in order to describe how lock TransactionIsolation::SERIALIZABLE transactions work. Each transaction in Ignite is assigned a comparable version called XidVersion. Upon transaction commit each entry that is written in the transaction is assigned a new comparable version called EntryVersion. An TransactionConcurrency::OPTIMISTIC TransactionIsolation::SERIALIZABLE transaction with version XidVersionA will fail with a TransactionOptimisticException if:

  • There is an ongoing TransactionConcurrency::PESSIMISTIC or non-serializable TransactionConcurrency::OPTIMISTIC transaction holding a lock on an entry of the TransactionIsolation::SERIALIZABLE transaction.
  • There is another ongoing TransactionConcurrency::OPTIMISTIC TransactionIsolation::SERIALIZABLE transaction with version XidVersionB such that XidVersionB > XidVersionA and this transaction holds a lock on an entry of the TransactionIsolation::SERIALIZABLE transaction.
  • By the time the TransactionConcurrency::OPTIMISTIC TransactionIsolation::SERIALIZABLE transaction acquires all required locks there exists an entry with the current version different from the observed version before commit.

In a highly concurrent environment, optimistic locking might lead to a high transaction failure rate but pessimistic locking can lead to deadlocks if locks are acquired in a different order by transactions.
However, in a contention-free environment optimistic serializable locking may provide better performance for large transactions because the number of network trips depends only on the number of nodes that the transaction spans and does not depend on the number of keys in the transaction.

Atomicity Mode

Ignite supports 2 atomicity modes:

  • TRANSACTIONAL
  • ATOMIC

TRANSACTIONAL mode enables fully ACID-compliant transactions, however, when only atomic semantics are needed, it is recommended that ATOMIC mode is used for better performance.

ATOMIC mode provides better performance by avoiding transactional locks, while still providing data atomicity and consistency. Another difference in ATOMIC mode is that bulk writes, such as PutAll(...)and RemoveAll(...) methods are no longer executed in one transaction and can partially fail.

Performance

Note that transactions are disabled whenever ATOMIC mode is used, which allows to achieve much higher performance and throughput in cases when transactions are not needed.

Configuration

Atomicity mode can be configured via atomicityMode property of CacheConfiguration.

Default atomicity mode is ATOMIC.

<bean class="org.apache.ignite.configuration.IgniteConfiguration">
    ...
    <property name="cacheConfiguration">
        <bean class="org.apache.ignite.configuration.CacheConfiguration">
          <!-- Set a cache name. -->
   				<property name="name" value="myCache"/>

           <!-- Set atomicity mode, can be ATOMIC or TRANSACTIONAL. -->
    			 <property name="atomicityMode" value="TRANSACTIONAL"/>
            ... 
        </bean>
    </property>
          
    <!-- Optional transaction configuration. -->
    <property name="transactionConfiguration">
        <bean class="org.apache.ignite.configuration.TransactionConfiguration">
            <!-- Configure TM lookup here. -->
        </bean>
    </property>
</bean>

Transactions

ACID compliant transactions ensuring guaranteed consistency.