Autograder with Java kernel: How to implement assertions?

Operating system

cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=18.04 DISTRIB_CODENAME=bionic DISTRIB_DESCRIPTION=“Ubuntu 18.04 LTS”

nbgrader --version

Python version 3.6.5 | packaged by conda-forge | (default, Apr 6 2018, 13:39:56) [GCC 4.8.2 20140120 (Red Hat 4.8.2-15)] nbgrader version 0.5.4

jupyterhub --version (if used with JupyterHub)

0.9.0rc1

jupyter notebook --version

5.5.0

Issue

I am using nbgrader on Jupyterhub with the scijava kernel. Code execution and importing packages works fine. The implementation of autograder tests with assert statements fail. A statement assert (1==1) is evaluated and nothing happens, which is fine. But a statement assert (1==2) produces an error inside the kernel, which is not recognized by nbgrader. Additionally, the autograder assigns full credit.

Is there a way to evaluate Java assertions? If not: How can errors be identified to not assign credits?

Expected behavior

• Assertion is evaluated as false.
• Autograder does not assign credits

Actual behavior

• Assertion produces error (see below)
• Autograder assigns full credit

[ERROR] null
Assertion failed: 

assert (1==2)
         |
         false

	at org.codehaus.groovy.runtime.InvokerHelper.assertFailed(InvokerHelper.java:404)
	at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.assertFailed(ScriptBytecodeAdapter.java:650)
	at Script5.run(Script5.groovy:1)
	at org.scijava.plugins.scripting.groovy.GroovyScriptEngine.eval(GroovyScriptEngine.java:303)
	at org.scijava.plugins.scripting.groovy.GroovyScriptEngine.eval(GroovyScriptEngine.java:122)
	at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:264)
	at org.scijava.script.ScriptModule.run(ScriptModule.java:160)
	at org.scijava.module.ModuleRunner.run(ModuleRunner.java:168)
	at org.scijava.jupyter.kernel.evaluator.Worker.run(Worker.java:108)
	at org.scijava.thread.DefaultThreadService$2.run(DefaultThreadService.java:221)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

Steps to reproduce the behavior

• Install Jupyterhub docker version (https://hub.docker.com/r/jupyterhub/jupyterhub/)
• Install nbgrader (http://nbgrader.readthedocs.io/en/stable/user_guide/installation.html)
• Install scijava-jupyter-kernel (https://github.com/scijava/scijava-jupyter-kernel)
• If the kernel does not appear in the frontend, take a look at these solutions: https://github.com/scijava/scijava-jupyter-kernel/issues/79
• Create a notebook with the assertion above

[jhamrick]

This sounds like a bug with the java kernel, would you mind filing an issue there instead? Specifically, the thing that needs to happen is for the java kernel to send an error message kernel reply, which will cause the notebook to create an “error” output: https://github.com/jupyter/nbformat/blob/master/nbformat/v4/nbbase.py#L55

nbgrader then looks for this error output. If it’s assigning full credit, that means that there is no error output on the cell which makes me think it’s a problem with the java kernel not sending the right type of response.

[jhamrick]

Closing this, as it seems there is a solution using the IJava kernel.