Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Generic recruiter: end-point customisation: 5pts

See original GitHub issue


Related to #2464 As part of the generic recruiter, we want to be able to do the following:

  • Have a customizable experiment method that generates exit_info about the participant. This exit_info is saved in the Participant object and might contain for example a randomly generated code that is used to validate their session with an external recruitment service.
  • Have a custom completion page (specified as an HTML template) where we might for example display this randomly generated code to the participant, or provide a link for the participant that will take them back to the external recruitment service.

Pre-implementation Notes

How Does this Work Now?

MTurk Recruiter

[N.B. The options described below that use the participant_task_completed() callback assume the completion of]

  1. The /worker_complete route will call the Experiment with the participant via exp.participant_task_completed(participant). This is probably the first opportunity to do something. Unlike the submission_successful() calls described below, this one is synchronous.
  2. When the MTurkRecruiter’s SNS notification route is called, the Experiment’s submission_successful() method will be called with the Participant: self.experiment.submission_successful(participant=self.participant)
  3. This self.experiment.submission_successful(participant=self.participant) will also occur if the heroku clock is running and maximum experiment duration has been exceeded and the Participant’s status is “Submitted”

Other (CLI) recruiter(s)

  1. The /worker_complete route will call the Experiment with the participant via exp.participant_task_completed(participant). This is probably the first opportunity to do something. Unlike the submission_successful() calls described below, this one is synchronous.
  2. The /worker_complete route will enqueue a task which triggers a call to the Experiment’s submission_successful() method with the Participant: self.experiment.submission_successful(participant=self.participant)

Regardless of recruiter, the participant_task_completed() call will happen synchronously when the experiment is completed, but as things are implemented currently, before Participant end_time is assigned, status is set to “Submitted”, and data checks are preformed. The async submission_successful() call happens after these updates are applied, but still before the Participant is redirected to the /ad page.

When should the Experiment assign exit_info?

It seems like either of these existing callback points would work. @pmcharrison Do you have a preference? I’m also not attached to where and how we orchestrate data checking, bonuses, participant status updates, etc, as there’s a lot of possibly needless chatter back and forth between the executing worker_function and various Experiment callbacks. That said, the easy path would probably still be to leave all of this as-is.

Since the submission_successful() call happens after data check, etc., I’m inclined to use that callback. It also has the fact that the default implementation is currently empty going for it:

    def submission_successful(self, participant):
        """Run when a participant submits successfully."""

We could either add the exit_info assignment directly to this method, or we could have it call a new, separate, special-purpose method as you originally suggested (I think this is probably clearer overall, despite the added call).

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:9

github_iconTop GitHub Comments

pmcharrisoncommented, Feb 11, 2021

Good question - I think it can be left empty by default.

On Thu, 11 Feb 2021 at 6:45 pm, Jesse Snyder wrote:

@pmcharrison Do you have in mind what the default value/behavior should be for exit_info? If and experiment does not customize anything, and deploys to heroku using MTurk recruitment, do you envision something specific winding up in that field?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe .

jacobyncommented, Mar 5, 2021

This branch has the same issue as in #2599 as a results the information in the entry point is not saved and is “empty” at the end-point.

Read more comments on GitHub >

github_iconTop Results From Across the Web

LinkedIn Recruiter features - Talent Solutions
Explore all of the LinkedIn Recruiter tool's features and updates to source, ... Save your custom filters to cut down manual work on...
Read more >
Endpoints - SmartRecruiters API
This section aims at presenting all the base endpoints that are part of the SmartRecruiters public Posting API. Available endpoints: /postings – method...
Read more >
10 Essential Recruiting Email Templates for 2022 | Built In
The top 10 recruiting email templates for every stage of recruiting and hiring to keep ... Personalization is key to compelling the individual...
Read more >
Overview Recruiter System Connect (RSC) - LinkedIn
Overview of Recruiter System Connect. ... which means you should manage job applications exclusively via the Middleware endpoints.
Read more >
The Recruitment Process: 10 Steps Necessary for Success
Few candidates like receiving messages from recruiters they don't know – especially when these messages are generic boilerplate templates. To get someone ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Post

No results found

github_iconTop Related Hashnode Post

No results found