NPE Accessing Entity Property

Description Recently upgraded from Exposed v0.17.7 to v0.39.2. Initially everything looked great until we moved to production and started finding this exception occurring intermittently. Running this code millions of times a day produces the following exception less than 10 times a day but it seems to indicate there is some change in the Exposed Entity / EntityClass which is failing now whereas it was working consistently in v0.17.7. We have upgraded to v0.40.1 hoping that would fix the problem but the problem persists in v0.40.1.

Dependencies kotlinLibVersion=1.7.10 kotlinCoroutinesLibVersion=1.6.4 kotlinLanguageVersion=1.7 ktorLibVersion=2.1.0 kodeinLibVersion=7.14.0 shadowJarLibVersion=7.1.2 awsKotlinLibVersion=0.17.8-beta exposedLibVersion=0.40.1 junitLibVersion=5.9.1 mockkLibVersion=1.13.2

AffiliateData

data class AffiliateData(
    override val affiliateKey: String,
    override val name: String,
    override val affiliateId: UUID,
    override val dateCreated: DateTime,
    override val legacyPublisherId: Int?,
    override val everflowAffiliateId: Int,
    override val everflowAffiliateHash: String,
    override val isTest: Boolean,
    override val maskedName: String?,
    override val companyId: UUID?,
): AffiliateInterface {
    companion object {
        fun createInstance(affiliateEntity: AffiliateInterface): AffiliateInterface {
            return AffiliateData(
                affiliateKey = affiliateEntity.affiliateKey,
                name = affiliateEntity.name,
                affiliateId = affiliateEntity.affiliateId,
                dateCreated = affiliateEntity.dateCreated,
                legacyPublisherId = affiliateEntity.legacyPublisherId,
                everflowAffiliateId = affiliateEntity.everflowAffiliateId,
                everflowAffiliateHash = affiliateEntity.everflowAffiliateHash,
                isTest = affiliateEntity.isTest,
                maskedName = affiliateEntity.maskedName,
                companyId = affiliateEntity.companyId,
            )
        }
    }
}

Exception

java.lang.NullPointerException: null
at org.jetbrains.exposed.dao.Entity.lookup(Entity.kt:194)
at org.jetbrains.exposed.dao.Entity.getValue(Entity.kt:174)
at com.acquireinteractive.attribution.model.Affiliate.getEverflowAffiliateHash(Affiliate.kt:41)
at com.acquireinteractive.attribution.domain.AffiliateData$Companion.createInstance(AffiliateData.kt:29)
at com.acquireinteractive.attribution.model.repository.TrafficSourceRepository$getAffiliateData$1.invoke(TrafficSourceRepository.kt:298)
at com.acquireinteractive.attribution.model.repository.TrafficSourceRepository$getAffiliateData$1.invoke(TrafficSourceRepository.kt:297)
at org.jetbrains.exposed.sql.transactions.ThreadLocalTransactionManagerKt.inTopLevelTransaction$run(ThreadLocalTransactionManager.kt:214)
at org.jetbrains.exposed.sql.transactions.ThreadLocalTransactionManagerKt.access$inTopLevelTransaction$run(ThreadLocalTransactionManager.kt:1)
at org.jetbrains.exposed.sql.transactions.ThreadLocalTransactionManagerKt$inTopLevelTransaction$1.invoke(ThreadLocalTransactionManager.kt:240)
at org.jetbrains.exposed.sql.transactions.ThreadLocalTransactionManagerKt.keepAndRestoreTransactionRefAfterRun(ThreadLocalTransactionManager.kt:248)
at org.jetbrains.exposed.sql.transactions.ThreadLocalTransactionManagerKt.inTopLevelTransaction(ThreadLocalTransactionManager.kt:239)
at org.jetbrains.exposed.sql.transactions.ThreadLocalTransactionManagerKt$transaction$1.invoke(ThreadLocalTransactionManager.kt:189)
at org.jetbrains.exposed.sql.transactions.ThreadLocalTransactionManagerKt.keepAndRestoreTransactionRefAfterRun(ThreadLocalTransactionManager.kt:248)
at org.jetbrains.exposed.sql.transactions.ThreadLocalTransactionManagerKt.transaction(ThreadLocalTransactionManager.kt:159)
at org.jetbrains.exposed.sql.transactions.ThreadLocalTransactionManagerKt.transaction(ThreadLocalTransactionManager.kt:146)
at com.acquireinteractive.database.service.DatabaseService.acquireReader(DatabaseService.kt:17)
at com.acquireinteractive.database.repository.AbstractRepository.acquireReader(AbstractRepository.kt:27)
at com.acquireinteractive.attribution.model.repository.TrafficSourceRepository.getAffiliateData(TrafficSourceRepository.kt:297)
at com.acquireinteractive.attribution.service.AttributionService.getAffiliate(AttributionService.kt:174)
at com.acquireinteractive.loan.service.LoanService.createLoanRequest(LoanService.kt:377)
at com.acquireinteractive.loan.service.LoanService.createLoanRequestContext(LoanService.kt:597)
at com.acquireinteractive.loan.service.LoanService.access$createLoanRequestContext(LoanService.kt:47)
at com.acquireinteractive.loan.service.LoanService$queueLoanRequest$deferred$1.invokeSuspend(LoanService.kt:125)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)at java.lang.Thread.run(Thread.java:750)
-- | --

[robertatpw]

@AlexeySoshin There are a few other factors of which you should be aware that may not be obvious from my original posting.

1. The error is happening on other models, not just the Affiliate model. I just picked the Affiliate model to demonstrate an issue which is most pronounced since we’ve upgraded. If you would like to see other examples, I can provide additional stack traces although they are all being thrown from the same location in the Exposed Entity class.
2. The stack trace indicates in all counts that there is an exception being thrown from the function org.jetbrains.exposed.dao.Entity.lookup although I realize that the problem may not be in the Entity class itself but somewhere else in the Exposed framework. As I indicated in my last response, the lookup function seems to be completely synchronous but there seems to be something in the Exposed framework which is no longer synchronous in this case.
3. In all cases, the specific property of the various models which is throwing the exception varies.
4. If you would like more examples (i.e. additional stack traces, code snippets, etc) please let me know.

We really need to find a solution to this problem because in the near future we want to move to a pattern of accessing all properties on the Exposed Entities in order to pass around plain old data objects. In the particular case of the Affiliate model, I was attempting to head off this same exception happening further down in our code by getting all of the data up front but instead of resolving that issue it just made the problem more pronounced and moved the exception to this “AffiliateData::createInstance” function rather than elsewhere in the code.

[robertatpw]

@Tapac Following up on my last reply, here’s an example where the same problem happens just accessing a property on a newly created entity.

java.lang.NullPointerException: null
at org.jetbrains.exposed.dao.Entity.lookup(Entity.kt:194)
at org.jetbrains.exposed.dao.Entity.getValue(Entity.kt:174)
at com.acquireinteractive.loan.model.LoanRequest.getIdentityId(LoanRequest.kt:320)
at com.acquireinteractive.loan.service.LoanPhoneQualityScoreService.lookupPhoneQuality(LoanPhoneQualityScoreService.kt:34)
at com.acquireinteractive.loan.service.LoanPhoneQualityScoreService$lookupPhoneQualityAsync$1.invokeSuspend(LoanPhoneQualityScoreService.kt:23)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)

LoanRequestTable val identityId = uuid("identity_id").index()

LoanRequest override var identityId by LoanRequestTable.identityId

A new LoanRequest entity is created on every request and passed around the code base to be referenced by many subsystems. In this particular case it’s being used to get some phone quality information about the incoming request. The loan_request table schema is a very flat schema and you can see that the identity_id column is just a UUID. There’s nothing special about this particular property. Again, this problem didn’t occur in v0.17.7 of the Exposed library. It just started happening when we updated to v0.39.2 and is still happening now that we’ve upgraded to the latest available version.