Unicode error when trying to run basic rnn training

On Ubuntu 16.04 no GPU, I followed the instructions to create MIDI note sequences with these MIDI files:dkc2.zip saved in /tmp/vgnotesequences2.tfrecord. Then ran the command

./bazel-bin/magenta/models/basic_rnn_train --experiment_run_dir=/tmp/basic_rnn/run1 --sequence_example_file=/tmp/vgnotesequences2.tfrecord --eval=false --hparams='{"rnn_layer_sizes":[50]}' --num_training_steps=20000

And ended up with this error:

INFO:tensorflow:Error reported to Coordinator: <type 'exceptions.UnicodeDecodeError'>, 'utf8' codec can't decode byte 0xed in position 143: invalid continuation byte Error reported to Coordinator: <type 'exceptions.UnicodeDecodeError'>, 'utf8' codec can't decode byte 0xed in position 143: invalid continuation byte Got error reported to coordinator: PaddingFIFOQueue '_1_rnn_model/padding_fifo_queue' is closed and has insufficient elements (requested 128, current size 0) [[Node: rnn_model/padding_fifo_queue_DequeueMany = QueueDequeueMany[_class=["loc:@rnn_model/padding_fifo_queue"], component_types=[DT_FLOAT, DT_INT64, DT_INT32], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](rnn_model/padding_fifo_queue, rnn_model/padding_fifo_queue_DequeueMany/n)]] Caused by op u'rnn_model/padding_fifo_queue_DequeueMany', defined at: File "/home/alex/magenta/bazel-bin/magenta/models/basic_rnn_train.runfiles/__main__/magenta/models/basic_rnn/basic_rnn_train.py", line 449, in <module> tf.app.run() File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 30, in run sys.exit(main(sys.argv)) File "/home/alex/magenta/bazel-bin/magenta/models/basic_rnn_train.runfiles/__main__/magenta/models/basic_rnn/basic_rnn_train.py", line 422, in main hparams_string=FLAGS.hparams, is_eval_mode=FLAGS.eval) File "/home/alex/magenta/bazel-bin/magenta/models/basic_rnn_train.runfiles/__main__/magenta/models/basic_rnn/basic_rnn_train.py", line 203, in make_graph lengths) = basic_rnn_ops.dynamic_rnn_batch(file_list, hparams) File "/home/alex/magenta/magenta/models/basic_rnn/basic_rnn_ops.py", line 125, in dynamic_rnn_batch return queue.dequeue_many(hparams.batch_size) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/data_flow_ops.py", line 434, in dequeue_many self._queue_ref, n=n, component_types=self._dtypes, name=name) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gen_data_flow_ops.py", line 465, in _queue_dequeue_many timeout_ms=timeout_ms, name=name) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/op_def_library.py", line 704, in apply_op op_def=op_def) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 2260, in create_op original_op=self._default_original_op, op_def=op_def) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1230, in __init__ self._traceback = _extract_stack()

I assume there is some sort of incorrect character in the midi data, but the note sequence parser did not have any issue with these midis so there may be something else going on. I have gotten similar errors with other sets of MIDI files.

[osetinsky]

@afuhrtrumpet I encountered something similar. I believe that you are confusing the output of convert_midi_dir_to_note_sequences.py. with that of basic_rnn_create_dataset.py

So first create a /tmp/notesequences.tfrecord with convert_midi_dir_to_note_sequences.py, and then create a /tmp/basic_rnn/sequence_examples/training_melodies.tfrecord with basic_rnn_create_dataset.py

It’s training_melodies.tfrecord that you want to pass to --sequence_example_file

[dbl001]

Also, something in ‘magenta’ wanted me to revert from numpy 0.20.3 to umpy 0.19.2

Error:

File “/Users/davidlaxer/anaconda/lib/python2.7/site-packages/dask/dataframe/core.py”, line 38, in <module> pd.computation.expressions.set_use_numexpr(False) AttributeError: ‘module’ object has no attribute ‘computation’

In [2]: print pandas.version 0.20.3

In [3]: quit David-Laxers-MacBook-Pro:magenta davidlaxer$ conda install pandas=0.19.2

On Jul 26, 2017, at 9:23 AM, David Laxer [e-mail] wrote:

Python indicates the bazel command is an ‘ordinary string’.

$ bazel run //magenta/models/melody_rnn:melody_rnn_train – --config=attention_rnn --run_dir=/tmp/melody_rnn/logdir/run1 --sequence_example_file=/Users/davidlaxer/magenta/magenta/testdata/notesequences.tfrecord --hparams=“batch_size=10,rnn_layer_sizes=[64,64]” –save_summaries_secs=10000 --save_interval_secs=10000 --num_training_steps=20000

Open a new issue?

<Screen Shot 2017-07-26 at 9.22.49 AM.png>

On Jul 25, 2017, at 5:24 PM, cghawthorne <[e-mail] mailto:[e-mail]> wrote:

@dbl001 https://github.com/dbl001 that looks like a different error caused by copy/pasting unicode characters into your command. If you continue to have the problem, please open a new bug.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/tensorflow/magenta/issues/83#issuecomment-317911015, or mute the thread https://github.com/notifications/unsubscribe-auth/AC9i28Vih4QzyyHt7PMkP3B4m4iCmkMNks5sRodIgaJpZM4JFpzp.