Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Add DjangoJobExecution fail because sometimes EVENT_JOB_EXECUTED fire before EVENT_JOB_SUBMITTED

See original GitHub issue

Describe the bug Add DjangoJobExecution fail because sometimes EVENT_JOB_EXECUTED fire before EVENT_JOB_SUBMITTED

To Reproduce

def hello(request: HttpRequest):
    job = scheduler.add_job(h, trigger="date", id=str(timezone.now()), run_date=timezone.now() + timedelta(seconds=5))
    print(job)
    return JsonResponse({"success": True})

Expected behavior should ignore add DjangoJobExecution.

Screenshots h (trigger: date[2020-10-19 09:41:34 UTC], next run at: 2020-10-19 09:41:34 UTC) [19/Oct/2020 09:41:29] “POST /scheduler/hello HTTP/1.1” 200 17 hello //////////////TZH EVENT_JOB_EXECUTED /////////////HELLO 2020-10-19 09:41:34.862484 2020-10-19 09:41:29.846453+00:00 2020-10-19 09:41:34.846471+00:00 Executed 0.010432 1603100494.856903 None None //////////////TZH EVENT_JOB_SUBMITTED 2020-10-19 09:41:29.846453+00:00 2020-10-19 09:41:34.846471+00:00 /////////////HELLO 2020-10-19 09:41:34.939636 2020-10-19 09:41:29.846453+00:00 2020-10-19 09:41:34.846471+00:00 Started execution None None None None Error notifying listener Traceback (most recent call last): File “/home/anaconda2/envs/py3/lib/python3.8/site-packages/django_apscheduler/models.py”, line 156, in atomic_update_or_create job_execution = DjangoJobExecution.objects.select_for_update().get( File “/home/anaconda2/envs/py3/lib/python3.8/site-packages/django/db/models/query.py”, line 429, in get raise self.model.DoesNotExist( django_apscheduler.models.DjangoJobExecution.DoesNotExist: DjangoJobExecution matching query does not exist.

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File “/home/anaconda2/envs/py3/lib/python3.8/site-packages/django/db/backends/utils.py”, line 84, in _execute return self.cursor.execute(sql, params) File “/home/anaconda2/envs/py3/lib/python3.8/site-packages/django/db/backends/mysql/base.py”, line 73, in execute return self.cursor.execute(query, args) File “/home/anaconda2/envs/py3/lib/python3.8/site-packages/MySQLdb/cursors.py”, line 209, in execute res = self._query(query) File “/home/anaconda2/envs/py3/lib/python3.8/site-packages/MySQLdb/cursors.py”, line 315, in _query db.query(q) File “/home/anaconda2/envs/py3/lib/python3.8/site-packages/MySQLdb/connections.py”, line 240, in query _mysql.connection.query(self, query) MySQLdb._exceptions.IntegrityError: (1452, ‘Cannot add or update a child row: a foreign key constraint fails (wt.django_apscheduler_djangojobexecution, CONSTRAINT django_apscheduler_djangojobexecution_job_id_daf5090a_fk FOREIGN KEY (job_id) REFERENCES django_apscheduler_djangojob (id))’)

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File “/home/anaconda2/envs/py3/lib/python3.8/site-packages/apscheduler/schedulers/base.py”, line 831, in _dispatch_event cb(event) File “/home/anaconda2/envs/py3/lib/python3.8/site-packages/django_apscheduler/jobstores.py”, line 50, in handle_submission_event job_execution = DjangoJobExecution.atomic_update_or_create( File “/home/anaconda2/envs/py3/lib/python3.8/site-packages/django_apscheduler/models.py”, line 193, in atomic_update_or_create job_execution = DjangoJobExecution.objects.create( File “/home/anaconda2/envs/py3/lib/python3.8/site-packages/django/db/models/manager.py”, line 85, in manager_method return getattr(self.get_queryset(), name)(*args, **kwargs) File “/home/anaconda2/envs/py3/lib/python3.8/site-packages/django/db/models/query.py”, line 447, in create obj.save(force_insert=True, using=self.db) File “/home/anaconda2/envs/py3/lib/python3.8/site-packages/django/db/models/base.py”, line 753, in save self.save_base(using=using, force_insert=force_insert, File “/home/anaconda2/envs/py3/lib/python3.8/site-packages/django/db/models/base.py”, line 790, in save_base updated = self._save_table( File “/home/anaconda2/envs/py3/lib/python3.8/site-packages/django/db/models/base.py”, line 895, in _save_table results = self._do_insert(cls._base_manager, using, fields, returning_fields, raw) File “/home/anaconda2/envs/py3/lib/python3.8/site-packages/django/db/models/base.py”, line 933, in _do_insert return manager._insert( File “/home/anaconda2/envs/py3/lib/python3.8/site-packages/django/db/models/manager.py”, line 85, in manager_method return getattr(self.get_queryset(), name)(*args, **kwargs) File “/home/anaconda2/envs/py3/lib/python3.8/site-packages/django/db/models/query.py”, line 1249, in _insert return query.get_compiler(using=using).execute_sql(returning_fields) File “/home/anaconda2/envs/py3/lib/python3.8/site-packages/django/db/models/sql/compiler.py”, line 1397, in execute_sql cursor.execute(sql, params) File “/home/anaconda2/envs/py3/lib/python3.8/site-packages/django/db/backends/utils.py”, line 98, in execute return super().execute(sql, params) File “/home/anaconda2/envs/py3/lib/python3.8/site-packages/django/db/backends/utils.py”, line 66, in execute return self._execute_with_wrappers(sql, params, many=False, executor=self._execute) File “/home/anaconda2/envs/py3/lib/python3.8/site-packages/django/db/backends/utils.py”, line 75, in _execute_with_wrappers return executor(sql, params, many, context) File “/home/anaconda2/envs/py3/lib/python3.8/site-packages/django/db/backends/utils.py”, line 84, in _execute return self.cursor.execute(sql, params) File “/home/anaconda2/envs/py3/lib/python3.8/site-packages/django/db/utils.py”, line 90, in exit raise dj_exc_value.with_traceback(traceback) from exc_value File “/home/anaconda2/envs/py3/lib/python3.8/site-packages/django/db/backends/utils.py”, line 84, in _execute return self.cursor.execute(sql, params) File “/home/anaconda2/envs/py3/lib/python3.8/site-packages/django/db/backends/mysql/base.py”, line 73, in execute return self.cursor.execute(query, args) File “/home/anaconda2/envs/py3/lib/python3.8/site-packages/MySQLdb/cursors.py”, line 209, in execute res = self._query(query) File “/home/anaconda2/envs/py3/lib/python3.8/site-packages/MySQLdb/cursors.py”, line 315, in _query db.query(q) File “/home/anaconda2/envs/py3/lib/python3.8/site-packages/MySQLdb/connections.py”, line 240, in query _mysql.connection.query(self, query) django.db.utils.IntegrityError: (1452, ‘Cannot add or update a child row: a foreign key constraint fails (wt.django_apscheduler_djangojobexecution, CONSTRAINT django_apscheduler_djangojobexecution_job_id_daf5090a_fk FOREIGN KEY (job_id) REFERENCES django_apscheduler_djangojob (id))’)

Issue Analytics

State:
Created 3 years ago
Comments:8 (3 by maintainers)

Top GitHub Comments

1reaction

jcass77commented, Jun 8, 2021

@aprilahijriyan, I don’t think that the error that you are seeing is related to this issue.

Please log a new issue, and include details on (1) how you are starting the scheduler in your code and (2) how individual jobs are being scheduled.

It seems to me that DjangoJobStore.remove_job() is failing due to a foreign key constraint check.

But that does not make sense since we have the on_delete=models.CASCADE option defined on DjangoJobExecution.job (so Django is supposed to just delete all of the related job executions as well when removing a job from the job store). Do you have multiple threads running and / or do you make of a BackgroundScheduler perhaps?

0reactions

aprilahijriyancommented, Jun 8, 2021

full logs

WARNING:django_apscheduler.jobstores:Job 'post-21' no longer exists! Skipping logging of job execution...
DEBUG:apscheduler.scheduler:No jobs; waiting until a job is added
INFO:apscheduler.scheduler:Added job "_job_schedule_post" to job store "default"
DEBUG:apscheduler:Post published: string-Tt80tL
DEBUG:apscheduler.scheduler:Looking for jobs to run
INFO:apscheduler.executors.default:Job "_job_schedule_post (trigger: date[2021-06-08 06:15:00 UTC], next run at: 2021-06-08 06:15:00 UTC)" executed successfully
INFO:apscheduler.executors.default:Running job "_job_schedule_post (trigger: date[2021-06-08 06:15:00 UTC], next run at: 2021-06-08 06:15:00 UTC)" (scheduled at 2021-06-08 06:15:00+00:00)
DEBUG:apscheduler:post id: 21
Exception in thread APScheduler:
Traceback (most recent call last):
  File "/Users/people/.local/share/virtualenvs/projek_2-9laxC7T5/lib/python3.8/site-packages/django/db/backends/base/base.py", line 242, in _commit
    return self.connection.commit()
psycopg2.errors.ForeignKeyViolation: update or delete on table "django_apscheduler_djangojob" violates foreign key constraint "django_apscheduler_djangojobexecution_job_id_daf5090a_fk" on table "django_apscheduler_djangojobexecution"
DETAIL:  Key (id)=(post-21) is still referenced from table "django_apscheduler_djangojobexecution".


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/people/.local/share/virtualenvs/projek_2-9laxC7T5/lib/python3.8/site-packages/apscheduler/schedulers/blocking.py", line 32, in _main_loop
    wait_seconds = self._process_jobs()
  File "/Users/people/.local/share/virtualenvs/projek_2-9laxC7T5/lib/python3.8/site-packages/apscheduler/schedulers/base.py", line 1002, in _process_jobs
    self.remove_job(job.id, jobstore_alias)
  File "/Users/people/.local/share/virtualenvs/projek_2-9laxC7T5/lib/python3.8/site-packages/apscheduler/schedulers/base.py", line 619, in remove_job
    store.remove_job(job_id)
  File "/Users/people/.local/share/virtualenvs/projek_2-9laxC7T5/lib/python3.8/site-packages/django_apscheduler/jobstores.py", line 259, in remove_job
DEBUG:apscheduler:Post published: string-Tt80tL
INFO:apscheduler.executors.default:Job "_job_schedule_post (trigger: date[2021-06-08 06:15:00 UTC], next run at: 2021-06-08 06:15:00 UTC)" executed successfully
    DjangoJob.objects.get(id=job_id).delete()
  File "/Users/people/.local/share/virtualenvs/projek_2-9laxC7T5/lib/python3.8/site-packages/django/db/models/base.py", line 954, in delete
    return collector.delete()
  File "/Users/people/.local/share/virtualenvs/projek_2-9laxC7T5/lib/python3.8/site-packages/django/db/models/deletion.py", line 435, in delete
    signals.post_delete.send(
  File "/Users/people/.local/share/virtualenvs/projek_2-9laxC7T5/lib/python3.8/site-packages/django/db/transaction.py", line 246, in __exit__
    connection.commit()
  File "/Users/people/.local/share/virtualenvs/projek_2-9laxC7T5/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
    return func(*args, **kwargs)
  File "/Users/people/.local/share/virtualenvs/projek_2-9laxC7T5/lib/python3.8/site-packages/django/db/backends/base/base.py", line 266, in commit
    self._commit()
  File "/Users/people/.local/share/virtualenvs/projek_2-9laxC7T5/lib/python3.8/site-packages/django/db/backends/base/base.py", line 242, in _commit
    return self.connection.commit()
  File "/Users/people/.local/share/virtualenvs/projek_2-9laxC7T5/lib/python3.8/site-packages/django/db/utils.py", line 90, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/Users/people/.local/share/virtualenvs/projek_2-9laxC7T5/lib/python3.8/site-packages/django/db/backends/base/base.py", line 242, in _commit
    return self.connection.commit()
django.db.utils.IntegrityError: update or delete on table "django_apscheduler_djangojob" violates foreign key constraint "django_apscheduler_djangojobexecution_job_id_daf5090a_fk" on table "django_apscheduler_djangojobexecution"
DETAIL:  Key (id)=(post-21) is still referenced from table "django_apscheduler_djangojobexecution".