importcontent --fail-on-error does not capture all failures
See original GitHub issueObserved behavior
For #9258, a --fail-on-error
option was added to importcontent
so that it would fail the command if there were issues importing content. However, it currently only covers a missing network file.
Errors and logs
ERROR An error occurred during content import: File https://ourserver/content/storage/d/e/de89f92e4e6b4d79112596dde6eda776.mp4 is corrupted.
At this point, the command should fail when run with --fail-on-error
, but it carried on. While looking at this code again, I noticed that the --fail-on-error
was only taking effect for network 404s. It doesn’t handle files that fail checksum validation. It also doesn’t apply for importcontent disk
when the source file is missing.
Expected behavior
Any exception when importing content should exit importcontent
when run with --fail-on-error
.
User-facing consequences
They try to use importcontent
with --fail-on-error
to ensure fully provisioned devices and it ignores some errors.
Steps to reproduce
Setup a source kolibri instance and make a copy of one of the objects so it can be tested with.
channel="f393c30f95fb4bec87f873b2013ec9e3"
src=$(mktemp -d)
object="${src}/content/storage/2/d/2d254f0cda03a45f27df1f1336475077.epub"
KOLIBRI_HOME="$src" python -m kolibri manage importchannel network "$channel"
KOLIBRI_HOME="$src" python -m kolibri manage importcontent network "$channel"
cp "${object}" "${object}.orig"
KOLIBRI_HOME="$src" python -m kolibri start --port 8000
Remove the object and try to import from disk in a destination kolibri instance.
rm -f "${object}"
dest=$(mktemp -d)
KOLIBRI_HOME="$dest" python -m kolibri manage importchannel disk "$channel" "$src"
KOLIBRI_HOME="$dest" python -m kolibri manage importcontent --fail-on-error disk "$channel" "$src"
An error will be logged, but the command will succeed.
Replace the object with some random characters so it’s corrupted and try to import again.
echo "foo" > "${object}"
KOLIBRI_HOME="$dest" python -m kolibri manage importcontent --fail-on-error disk "$channel" "$src"
An error will be logged, but the command will succeed.
Repeat using the network.
KOLIBRI_HOME="$dest" python -m kolibri manage importcontent --fail-on-error network --baseurl http://127.0.0.1:8000 "$channel"
An error will be logged, but the command will succeed.
Context
We’re trying to populate bootable images with Kolibri pre-seeded and we want to ensure that our build process robustly populates the desired channels.
Issue Analytics
- State:
- Created a year ago
- Comments:5 (5 by maintainers)
Top GitHub Comments
#9591 fixes it.
Fixed in #9591