Events scheduled with QuartzEventScheduler using JDBC store cannot be deserialized after upgrade to new version of Axonframework

After upgrade to Axonframework 3.2 there was an exception in log:

`2018-04-13 14:10:30.311 [ ] ERROR [service,] 18529 — [QuartzScheduler_EventScheduler-somehost1523600896804_MisfireHandler] o.s.s.quartz.LocalDataSourceJobStore : MisfireH andler: Error handling misfires: Couldn’t store trigger ‘DEFAULT.6da64b5bd2ee-304dbe6e-0b6f-4461-9164-618e1515521e’ for ‘App.event-80bd2ba5-8ac1-41d1-802a-6a6dee1eb74f’ job:Couldn’t retrieve job because the BLOB couldn’t be deserialized: org.axonframework.messaging.MessageDecorator; local class incompatible: stream classdesc serialVersionUID = 101886804922030717, local class serialVersionUID = 3969631713723578521

org.quartz.JobPersistenceException: Couldn’t store trigger ‘DEFAULT.6da64b5bd2ee-304dbe6e-0b6f-4461-9164-618e1515521e’ for ‘App.event-80bd2ba5-8ac1-41d1-802a-6a6dee1eb74f’ job:Couldn’t retrieve job because the BLOB couldn’t be deserialized: org.axonframework.messaging.MessageDecorator; local class incompatible: stream classdesc serialVersionUID = 101886804922030717, local class serialVersionUID = 3969631713723578521 at org.quartz.impl.jdbcjobstore.JobStoreSupport.storeTrigger(JobStoreSupport.java:1223) at org.quartz.impl.jdbcjobstore.JobStoreSupport.doUpdateOfMisfiredTrigger(JobStoreSupport.java:1037) at org.quartz.impl.jdbcjobstore.JobStoreSupport.recoverMisfiredJobs(JobStoreSupport.java:986) at org.quartz.impl.jdbcjobstore.JobStoreSupport.doRecoverMisfires(JobStoreSupport.java:3203) at org.quartz.impl.jdbcjobstore.JobStoreSupport$MisfireHandler.manage(JobStoreSupport.java:3951) at org.quartz.impl.jdbcjobstore.JobStoreSupport$MisfireHandler.run(JobStoreSupport.java:3972) Caused by: org.quartz.JobPersistenceException: Couldn’t retrieve job because the BLOB couldn’t be deserialized: org.axonframework.messaging.MessageDecorator; local class incompatible: stream classdesc serialVersionUID = 101886804922030717, local class serialVersionUID = 3969631713723578521 at org.quartz.impl.jdbcjobstore.JobStoreSupport.retrieveJob(JobStoreSupport.java:1392) at org.quartz.impl.jdbcjobstore.JobStoreSupport.storeTrigger(JobStoreSupport.java:1205) … 5 common frames omitted Caused by: java.io.InvalidClassException: org.axonframework.messaging.MessageDecorator; local class incompatible: stream classdesc serialVersionUID = 101886804922030717, local class serialVersionUID = 3969631713723578521 at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:616) at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1843) at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1713) at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1843) at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1713) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2000) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422) at java.util.HashMap.readObject(HashMap.java:1404) at sun.reflect.GeneratedMethodAccessor892.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1058) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2136) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422) at org.quartz.impl.jdbcjobstore.StdJDBCDelegate.getObjectFromBlob(StdJDBCDelegate.java:3201) at org.quartz.impl.jdbcjobstore.StdJDBCDelegate.selectJobDetail(StdJDBCDelegate.java:860) at org.quartz.impl.jdbcjobstore.JobStoreSupport.retrieveJob(JobStoreSupport.java:1385) … 6 common frames omitted`

It seems that scheduled event message wrapped by MessageDecorator, which doesn’t have serialVersionUID field and have been changed.

[martijnvanderwoud]

In case it helps anyone, I managed to work around this by creating an extension of the ObjectInputStream class to ignore the serialVersionUID like this:

package nl.pompkracht.iam.port.adapter.out.javaio;

import java.io.IOException;
import java.io.InputStream;
import java.io.ObjectInputStream;
import java.io.ObjectStreamClass;

public class ObjectInputStreamIgnoringSerialVersionUID extends ObjectInputStream {

	public ObjectInputStreamIgnoringSerialVersionUID(InputStream in) throws IOException {
		super(in);
	}

	@Override
	protected ObjectStreamClass readClassDescriptor() throws IOException, ClassNotFoundException {
		ObjectStreamClass resultClassDescriptor = super.readClassDescriptor();
		Class<?> localClass = Class.forName(resultClassDescriptor.getName());
		return ObjectStreamClass.lookup(localClass);
	}
}

I used the above extension to deserialize the job details in the QRTZ_JOB_DETAILS. Then I used the reqular ObjectOutputStream to re-serialize the job details, and wrote the resulting byte array back to the QRTZ_JOB_DETAILS table. I implemented this as a FlywayDB Java migration.

package db.migration;

import nl.pompkracht.iam.port.adapter.out.javaio.ObjectInputStreamIgnoringSerialVersionUID;
import org.flywaydb.core.api.migration.BaseJavaMigration;
import org.flywaydb.core.api.migration.Context;
import org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate;
import org.springframework.jdbc.datasource.SingleConnectionDataSource;

import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.ObjectOutputStream;
import java.sql.Connection;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

@SuppressWarnings("unused")
public class V2_0_0_1__repair_serialized_quartz_jobs extends BaseJavaMigration {

	@Override
	public void migrate(Context context) throws Exception {
		Connection connection = context.getConnection();
		NamedParameterJdbcTemplate jdbcTemplate =
				new NamedParameterJdbcTemplate(new SingleConnectionDataSource(connection, true));

		HashMap<String, Object> selectParams = new HashMap<>();
		selectParams.put("jobGroup", "AxonFramework-Events");
		List<Map<String, Object>> jobs = jdbcTemplate
				.queryForList(
						"select JOB_NAME, DESCRIPTION, JOB_DATA from QRTZ_JOB_DETAILS WHERE JOB_GROUP = :jobGroup",
						selectParams
				);

		for (Map<String, Object> job : jobs) {
			byte[] serializedJobData = (byte[]) job.get("JOB_DATA");
			ByteArrayInputStream serializedJobDataInputStream = new ByteArrayInputStream(serializedJobData);
			ObjectInputStreamIgnoringSerialVersionUID lenientInputStream =
					new ObjectInputStreamIgnoringSerialVersionUID(serializedJobDataInputStream);

			Object jobData = lenientInputStream.readObject();

			ByteArrayOutputStream serializedJobDataOutputStream = new ByteArrayOutputStream();
			ObjectOutputStream regularObjectOutputStream = new ObjectOutputStream(serializedJobDataOutputStream);
			regularObjectOutputStream.writeObject(jobData);

			Map<String, Object> updateParams = new HashMap<>();
			updateParams.put("jobName", job.get("JOB_NAME"));
			updateParams.put("jobData", serializedJobDataOutputStream.toByteArray());

			jdbcTemplate.update(
					"UPDATE QRTZ_JOB_DETAILS SET JOB_DATA = :jobData WHERE JOB_NAME = :jobName",
					updateParams
			);
		}
	}

	@Override
	public Integer getChecksum() {
		return 1544452428;
	}
}

[smcvb]

Great idea @martijnvanderwoud, thanks so much for sharing! Hopefully, not to many people run in to the issue (of course), but having a potential solution documented here is definitely worthwhile 👍