Skip to content

Fix cached ID reuse after commit failure in MySQLMaxValueIncrementer - #37322

Open
cookie-meringue wants to merge 1 commit into
spring-projects:mainfrom
cookie-meringue:fix-cache-invalidation-in-mysql-incrementer
Open

cookie-meringue wants to merge 1 commit into
spring-projects:mainfrom
cookie-meringue:fix-cache-invalidation-in-mysql-incrementer

Conversation

@cookie-meringue

Copy link
Copy Markdown
Contributor

Description

In MySQLMaxValueIncrementer, if commit fails with useNewConnection set to true and cacheSize greater than 1, an ID range that was not reserved in the database can remain in the cache.

Even if the sequence increment is rolled back, subsequent calls can still use the cached range. If the database allocates the same range again, duplicate IDs can result.

Normal ID allocation

The incrementer obtains IDs by increasing a value in a sequence table. With cacheSize greater than 1, it reserves a range at once and serves the remaining IDs without querying the database.

For example, with a sequence value of 98 and cacheSize set to 2, it updates the sequence to 100. Once that update is committed, the range 99–100 is reserved.

The first ID request returns 99, and the next returns cached ID 100. Since the database now stores 100, the next range obtained from the database starts at 101.

Failure flow

When replenishing the cache, the current implementation performs these steps:

  1. Increase the sequence value in the database.
  2. Read the increased sequence value with SELECT LAST_INSERT_ID() and assign it to maxId.
  3. Set nextId to maxId - getCacheSize() + 1.
  4. Commit the sequence update.
  5. Return nextId to the caller.

The problem: the cache fields are changed before commit succeeds.

If commit throws an exception, ID generation fails without returning an ID, but the nextId and maxId values assigned before commit remain unchanged.

The next ID request only checks whether maxId == nextId. If the values are equal, it increases the sequence value in the database to obtain more IDs.

If the values differ, it increments nextId and returns it without accessing the database.

For the example above, suppose the connection is terminated before commit and the sequence increment is rolled back. The stored sequence value returns to 98, while the fields remain nextId=99 and maxId=100.

ID request Processing Result
1 Increases the sequence from 98 to 100, reads 100, and sets nextId=99, maxId=100. The connection is then terminated, rolling back the update and causing commit to fail. The database value returns to 98, but the fields remain unchanged. Throws an exception; no ID is returned.
2 Since nextId=99 and maxId=100 differ, increments nextId to 100 without accessing the database. The database value is still 98. Returns 100. Inserting it as a primary key succeeds.
3 Since nextId and maxId are now both 100, increases the database value from 98 to 100 again, reads it, and sets nextId=99, maxId=100. This time, commit succeeds. Returns 99. Inserting it succeeds.
4 Since nextId=99 and maxId=100 differ, increments nextId to 100 without accessing the database. Returns 100 again. The INSERT fails because request 2 already inserted that ID.

The INSERT using ID 100 from request 2 succeeds. The duplicate-key error occurs when request 4 returns 100 again and attempts another INSERT.

A commit exception does not always mean that the sequence increment was rolled back. The example above describes a case where it was rolled back, as verified in the MySQL test below.

This issue does not occur with the default cacheSize of 1. Since nextId equals maxId when commit fails, the next ID request obtains IDs from the database again.

Fix

Set nextId to maxId in the existing exception handler for commit and auto-commit restoration:

this.nextId = this.maxId;

Setting the two fields to the same value makes the next ID request satisfy maxId == nextId. The incrementer then increases the sequence value in the database to obtain IDs again.

Reproduction against MySQL

Verified with MySQL 9.1.0 and cacheSize=2.

  1. Initialize the sequence value to 98.
  2. Let the incrementer increase the sequence value to 100 and read that value.
  3. Immediately before commit, use a separate connection to execute KILL CONNECTION for the test connection, then invoke the real JDBC commit().
  4. Observe the resulting CommunicationsException and confirm that the stored sequence value is 98 from a separate connection.
  5. Request three more IDs and insert each into a table with a primary key.
Source Subsequent IDs INSERT results
Before the fix 100, 99, 100 Third INSERT fails with Duplicate entry '100'
After the fix 99, 100, 101 All three INSERTs succeed
Exception stack traces
com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure

The last packet successfully received from the server was 9 milliseconds ago. The last packet sent successfully to the server was 10 milliseconds ago.
	at com.mysql.cj.jdbc.exceptions.SQLError.createCommunicationsException(SQLError.java:165)
	at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:55)
	at com.mysql.cj.jdbc.ConnectionImpl.commit(ConnectionImpl.java:811)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
	at java.base/java.lang.reflect.Method.invoke(Method.java:565)
	at MySqlCacheFailureRepro$1.lambda$getConnection$0(MySqlCacheFailureRepro.java:70)
	at jdk.proxy1/jdk.proxy1.$Proxy0.commit(Unknown Source)
	at org.springframework.jdbc.support.incrementer.MySQLMaxValueIncrementer.getNextKey(MySQLMaxValueIncrementer.java:177)
	at org.springframework.jdbc.support.incrementer.AbstractDataFieldMaxValueIncrementer.nextLongValue(AbstractDataFieldMaxValueIncrementer.java:142)
	at MySqlCacheFailureRepro.main(MySqlCacheFailureRepro.java:88)
Caused by: com.mysql.cj.exceptions.CJCommunicationsException: Communications link failure

The last packet successfully received from the server was 9 milliseconds ago. The last packet sent successfully to the server was 10 milliseconds ago.
	at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:62)
	at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:483)
	at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:52)
	at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:95)
	at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:140)
	at com.mysql.cj.exceptions.ExceptionFactory.createCommunicationsException(ExceptionFactory.java:156)
	at com.mysql.cj.protocol.a.NativeProtocol.readMessage(NativeProtocol.java:592)
	at com.mysql.cj.protocol.a.NativeProtocol.checkErrorMessage(NativeProtocol.java:769)
	at com.mysql.cj.protocol.a.NativeProtocol.sendCommand(NativeProtocol.java:708)
	at com.mysql.cj.protocol.a.NativeProtocol.sendQueryPacket(NativeProtocol.java:940)
	at com.mysql.cj.NativeSession.execSQL(NativeSession.java:817)
	at com.mysql.cj.jdbc.ConnectionImpl.commit(ConnectionImpl.java:791)
	... 7 more
Caused by: java.io.EOFException: Can not read response from server. Expected to read 4 bytes, read 0 bytes before connection was unexpectedly lost.
	at com.mysql.cj.protocol.FullReadInputStream.readFully(FullReadInputStream.java:58)
	at com.mysql.cj.protocol.a.SimplePacketReader.readHeaderLocal(SimplePacketReader.java:72)
	at com.mysql.cj.protocol.a.SimplePacketReader.readHeader(SimplePacketReader.java:54)
	at com.mysql.cj.protocol.a.SimplePacketReader.readHeader(SimplePacketReader.java:36)
	at com.mysql.cj.protocol.a.TimeTrackingPacketReader.readHeader(TimeTrackingPacketReader.java:43)
	at com.mysql.cj.protocol.a.TimeTrackingPacketReader.readHeader(TimeTrackingPacketReader.java:32)
	at com.mysql.cj.protocol.a.MultiPacketReader.readHeader(MultiPacketReader.java:45)
	at com.mysql.cj.protocol.a.MultiPacketReader.readHeader(MultiPacketReader.java:35)
	at com.mysql.cj.protocol.a.NativeProtocol.readMessage(NativeProtocol.java:586)
	... 12 more
org.springframework.dao.DataAccessResourceFailureException: Unable to commit new sequence value changes for incrementer_cache_27ff12edd8d04d2580f3631fd3d3b7d4.BATCH_JOB_EXECUTION_SEQ
	at org.springframework.jdbc.support.incrementer.MySQLMaxValueIncrementer.getNextKey(MySQLMaxValueIncrementer.java:184)
	at org.springframework.jdbc.support.incrementer.AbstractDataFieldMaxValueIncrementer.nextLongValue(AbstractDataFieldMaxValueIncrementer.java:142)
	at MySqlCacheFailureRepro.main(MySqlCacheFailureRepro.java:88)
Exception in thread "main" java.sql.SQLIntegrityConstraintViolationException: Duplicate entry '100' for key 'BATCH_JOB_EXECUTION.PRIMARY'
	at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:109)
	at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:114)
	at com.mysql.cj.jdbc.ClientPreparedStatement.executeInternal(ClientPreparedStatement.java:990)
	at com.mysql.cj.jdbc.ClientPreparedStatement.executeUpdateInternal(ClientPreparedStatement.java:1168)
	at com.mysql.cj.jdbc.ClientPreparedStatement.executeUpdateInternal(ClientPreparedStatement.java:1103)
	at com.mysql.cj.jdbc.ClientPreparedStatement.executeLargeUpdate(ClientPreparedStatement.java:1450)
	at com.mysql.cj.jdbc.ClientPreparedStatement.executeUpdate(ClientPreparedStatement.java:1086)
	at MySqlCacheFailureRepro.main(MySqlCacheFailureRepro.java:110)

Invalidate the cached ID range when commit or auto-commit restoration
fails so subsequent calls obtain a new range from the database.

Signed-off-by: cookie-meringue <daehyeon3351@gmail.com>
@spring-projects-issues spring-projects-issues added the status: waiting-for-triage An issue we've not yet triaged or decided on label Sep 22, 2026
@cookie-meringue

cookie-meringue commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor Author

This PR adds tests in the same section of DataFieldMaxValueIncrementerTests as #37321. Once #37321 is merged, this branch will need to be updated with the latest main, and any resulting conflicts will need to be resolved.

@sbrannen sbrannen added the in: data Issues in data modules (jdbc, orm, oxm, tx) label Sep 23, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

in: data Issues in data modules (jdbc, orm, oxm, tx) status: waiting-for-triage An issue we've not yet triaged or decided on

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants