Onsets and Frames transcription issue

I’m very new to these type of technologies, and I could figure out how to use it from other source, but I’m struggling with Onsets and Frames. Here you can see the output I get after transcribing:

And here’s when I type in the commands:

python3 onsets_frames_transcription_transcribe.py \
>   --model_dir=d/train \
>   hr1.wav
WARNING:tensorflow:From /home/ulaili/miniconda3/envs/magenta/lib/python3.6/site-packages/tensorflow_core/python/compat/v2_compat.py:65: disable_resource_variables (from tensorflow.python.ops.variable_scope) is deprecated and will be removed in a future version.
Instructions for updating:
non-resource variables are not supported in the long term
WARNING:tensorflow:From /home/ulaili/miniconda3/envs/magenta/lib/python3.6/site-packages/tensorflow_core/python/autograph/impl/api.py:330: py_func (from tensorflow.python.ops.script_ops) is deprecated and will be removed in a future version.
Instructions for updating:
tf.py_func is deprecated in TF V2. Instead, there are two
    options available in V2.
    - tf.py_function takes a python function which manipulates tf eager
    tensors instead of numpy arrays. It's easy to convert a tf eager tensor to
    an ndarray (just call tensor.numpy()) but having access to eager tensors
    means `tf.py_function`s can use accelerators such as GPUs as well as
    being differentiable using a gradient tape.
    - tf.numpy_function maintains the semantics of the deprecated tf.py_func
    (it is not differentiable, and manipulates numpy arrays). It drops the
    stateful argument making all functions stateful.

W0127 01:04:03.258965 139814394660672 deprecation.py:323] From /home/ulaili/miniconda3/envs/magenta/lib/python3.6/site-packages/tensorflow_core/python/autograph/impl/api.py:330: py_func (from tensorflow.python.ops.script_ops) is deprecated and will be removed in a future version.
Instructions for updating:
tf.py_func is deprecated in TF V2. Instead, there are two
    options available in V2.
    - tf.py_function takes a python function which manipulates tf eager
    tensors instead of numpy arrays. It's easy to convert a tf eager tensor to
    an ndarray (just call tensor.numpy()) but having access to eager tensors
    means `tf.py_function`s can use accelerators such as GPUs as well as
    being differentiable using a gradient tape.
    - tf.numpy_function maintains the semantics of the deprecated tf.py_func
    (it is not differentiable, and manipulates numpy arrays). It drops the
    stateful argument making all functions stateful.

2021-01-27 01:04:05.711863: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory
2021-01-27 01:04:05.713240: E tensorflow/stream_executor/cuda/cuda_driver.cc:318] failed call to cuInit: UNKNOWN ERROR (303)
2021-01-27 01:04:05.714265: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (DESKTOP-UM451RI): /proc/driver/nvidia/version does not exist
WARNING:tensorflow:From /home/ulaili/miniconda3/envs/magenta/lib/python3.6/site-packages/magenta/models/onsets_frames_transcription/data.py:656: DatasetV1.output_shapes (from tensorflow.python.data.ops.dataset_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.compat.v1.data.get_output_shapes(dataset)`.
W0127 01:04:05.833350 139814394660672 deprecation.py:323] From /home/ulaili/miniconda3/envs/magenta/lib/python3.6/site-packages/magenta/models/onsets_frames_transcription/data.py:656: DatasetV1.output_shapes (from tensorflow.python.data.ops.dataset_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.compat.v1.data.get_output_shapes(dataset)`.
WARNING:tensorflow:From /home/ulaili/miniconda3/envs/magenta/lib/python3.6/site-packages/magenta/models/onsets_frames_transcription/train_util.py:87: The name tf.estimator.tpu.RunConfig is deprecated. Please use tf.compat.v1.estimator.tpu.RunConfig instead.

W0127 01:04:05.859240 139814394660672 module_wrapper.py:137] From /home/ulaili/miniconda3/envs/magenta/lib/python3.6/site-packages/magenta/models/onsets_frames_transcription/train_util.py:87: The name tf.estimator.tpu.RunConfig is deprecated. Please use tf.compat.v1.estimator.tpu.RunConfig instead.

INFO:tensorflow:Using config: {'_model_dir': 'd/train', '_tf_random_seed': None, '_save_summary_steps': 300, '_save_checkpoints_steps': 300, '_save_checkpoints_secs': None, '_session_config': allow_soft_placement: true
graph_options {
  rewrite_options {
    meta_optimizer_iterations: ONE
  }
}
, '_keep_checkpoint_max': None, '_keep_checkpoint_every_n_hours': 1, '_log_step_count_steps': None, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_service': None, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7f2885d32be0>, '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1, '_tpu_config': TPUConfig(iterations_per_loop=300, num_shards=None, num_cores_per_replica=None, per_host_input_for_training=2, tpu_job_name=None, initial_infeed_sleep_secs=None, input_partition_dims=None, eval_training_input_configuration=2, experimental_host_call_every_n_steps=1), '_cluster': None}
I0127 01:04:05.864387 139814394660672 estimator.py:212] Using config: {'_model_dir': 'd/train', '_tf_random_seed': None, '_save_summary_steps': 300, '_save_checkpoints_steps': 300, '_save_checkpoints_secs': None, '_session_config': allow_soft_placement: true
graph_options {
  rewrite_options {
    meta_optimizer_iterations: ONE
  }
}
, '_keep_checkpoint_max': None, '_keep_checkpoint_every_n_hours': 1, '_log_step_count_steps': None, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_service': None, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7f2885d32be0>, '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1, '_tpu_config': TPUConfig(iterations_per_loop=300, num_shards=None, num_cores_per_replica=None, per_host_input_for_training=2, tpu_job_name=None, initial_infeed_sleep_secs=None, input_partition_dims=None, eval_training_input_configuration=2, experimental_host_call_every_n_steps=1), '_cluster': None}
INFO:tensorflow:_TPUContext: eval_on_tpu False
I0127 01:04:05.874145 139814394660672 tpu_context.py:221] _TPUContext: eval_on_tpu False
2021-01-27 01:04:05.894072: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 3201000000 Hz
2021-01-27 01:04:05.895696: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fffbd23adb0 executing computations on platform Host. Devices:
2021-01-27 01:04:05.896358: I tensorflow/compiler/xla/service/service.cc:175]   StreamExecutor device (0): Host, Default Version
INFO:tensorflow:Starting transcription for hr1.wav...
I0127 01:04:05.919807 139814394660672 onsets_frames_transcription_transcribe.py:112] Starting transcription for hr1.wav...
INFO:tensorflow:Processing file...
I0127 01:04:05.921452 139814394660672 onsets_frames_transcription_transcribe.py:118] Processing file...
INFO:tensorflow:Running inference...
I0127 01:04:38.318580 139814394660672 onsets_frames_transcription_transcribe.py:129] Running inference...
INFO:tensorflow:Could not find trained model in model_dir: d/train, running initialization to predict.
I0127 01:04:38.320430 139814394660672 estimator.py:615] Could not find trained model in model_dir: d/train, running initialization to predict.
WARNING:tensorflow:From /home/ulaili/miniconda3/envs/magenta/lib/python3.6/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
W0127 01:04:38.326467 139814394660672 deprecation.py:506] From /home/ulaili/miniconda3/envs/magenta/lib/python3.6/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
INFO:tensorflow:Calling model_fn.
I0127 01:04:44.236499 139814394660672 estimator.py:1147] Calling model_fn.
INFO:tensorflow:Running infer on CPU
I0127 01:04:44.237373 139814394660672 tpu_estimator.py:3124] Running infer on CPU
WARNING:tensorflow:From /home/ulaili/miniconda3/envs/magenta/lib/python3.6/site-packages/tf_slim/layers/layers.py:1089: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.
Instructions for updating:
Please use `layer.__call__` method instead.
W0127 01:04:44.242000 139814394660672 deprecation.py:323] From /home/ulaili/miniconda3/envs/magenta/lib/python3.6/site-packages/tf_slim/layers/layers.py:1089: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.
Instructions for updating:
Please use `layer.__call__` method instead.
WARNING:tensorflow:From /home/ulaili/miniconda3/envs/magenta/lib/python3.6/site-packages/magenta/contrib/rnn.py:474: bidirectional_dynamic_rnn (from tensorflow.python.ops.rnn) is deprecated and will be removed in a future version.
Instructions for updating:
Please use `keras.layers.Bidirectional(keras.layers.RNN(cell))`, which is equivalent to this API
W0127 01:04:44.354001 139814394660672 deprecation.py:323] From /home/ulaili/miniconda3/envs/magenta/lib/python3.6/site-packages/magenta/contrib/rnn.py:474: bidirectional_dynamic_rnn (from tensorflow.python.ops.rnn) is deprecated and will be removed in a future version.
Instructions for updating:
Please use `keras.layers.Bidirectional(keras.layers.RNN(cell))`, which is equivalent to this API
WARNING:tensorflow:From /home/ulaili/miniconda3/envs/magenta/lib/python3.6/site-packages/tensorflow_core/python/ops/rnn.py:464: dynamic_rnn (from tensorflow.python.ops.rnn) is deprecated and will be removed in a future version.
Instructions for updating:
Please use `keras.layers.RNN(cell)`, which is equivalent to this API
W0127 01:04:44.358820 139814394660672 deprecation.py:323] From /home/ulaili/miniconda3/envs/magenta/lib/python3.6/site-packages/tensorflow_core/python/ops/rnn.py:464: dynamic_rnn (from tensorflow.python.ops.rnn) is deprecated and will be removed in a future version.
Instructions for updating:
Please use `keras.layers.RNN(cell)`, which is equivalent to this API
WARNING:tensorflow:From /home/ulaili/miniconda3/envs/magenta/lib/python3.6/site-packages/magenta/contrib/rnn.py:751: Layer.add_variable (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.
Instructions for updating:
Please use `layer.add_weight` method instead.
W0127 01:04:44.411645 139814394660672 deprecation.py:323] From /home/ulaili/miniconda3/envs/magenta/lib/python3.6/site-packages/magenta/contrib/rnn.py:751: Layer.add_variable (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.
Instructions for updating:
Please use `layer.add_weight` method instead.
WARNING:tensorflow:From /home/ulaili/miniconda3/envs/magenta/lib/python3.6/site-packages/magenta/models/onsets_frames_transcription/metrics.py:105: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.cast` instead.
W0127 01:04:45.947215 139814394660672 deprecation.py:323] From /home/ulaili/miniconda3/envs/magenta/lib/python3.6/site-packages/magenta/models/onsets_frames_transcription/metrics.py:105: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.cast` instead.
WARNING:tensorflow:From /home/ulaili/miniconda3/envs/magenta/lib/python3.6/site-packages/magenta/models/onsets_frames_transcription/metrics.py:120: div (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Deprecated in favor of operator or tf.math.divide.
W0127 01:04:45.974914 139814394660672 deprecation.py:323] From /home/ulaili/miniconda3/envs/magenta/lib/python3.6/site-packages/magenta/models/onsets_frames_transcription/metrics.py:120: div (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Deprecated in favor of operator or tf.math.divide.
INFO:tensorflow:Done calling model_fn.
I0127 01:04:46.274543 139814394660672 estimator.py:1149] Done calling model_fn.
INFO:tensorflow:Graph was finalized.
I0127 01:04:46.501704 139814394660672 monitored_session.py:240] Graph was finalized.
INFO:tensorflow:Running local_init_op.
I0127 01:04:46.927279 139814394660672 session_manager.py:500] Running local_init_op.
INFO:tensorflow:Done running local_init_op.
I0127 01:04:46.999941 139814394660672 session_manager.py:502] Done running local_init_op.
2021-01-27 01:04:47.949551: W tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 1133670912 exceeds 10% of system memory.
2021-01-27 01:04:47.949530: W tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 1133670912 exceeds 10% of system memory.
2021-01-27 01:04:47.953850: W tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 1133670912 exceeds 10% of system memory.
2021-01-27 01:04:50.696411: W tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 1133670912 exceeds 10% of system memory.
2021-01-27 01:04:50.712027: W tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 1133670912 exceeds 10% of system memory.
INFO:tensorflow:Calculating metrics for b'hr1.wav' with length 25784
I0127 01:06:39.989615 139810974861056 metrics.py:160] Calculating metrics for b'hr1.wav' with length 25784
INFO:tensorflow:Reference pitches were length 0, returning empty metrics for b'hr1.wav':
I0127 01:06:44.393559 139810974861056 metrics.py:215] Reference pitches were length 0, returning empty metrics for b'hr1.wav':
INFO:tensorflow:prediction_loop marked as finished
I0127 01:06:44.930123 139814394660672 error_handling.py:101] prediction_loop marked as finished
INFO:tensorflow:prediction_loop marked as finished
I0127 01:06:44.932056 139814394660672 error_handling.py:101] prediction_loop marked as finished
INFO:tensorflow:Transcription written to hr1.wav.midi.
I0127 01:06:50.247773 139814394660672 onsets_frames_transcription_transcribe.py:146] Transcription written to hr1.wav.midi.

I don’t know what’s not working here, I’d appreciate some help, I could give more specific information if this is not enough to help me. Thanks so much

[LordofDorknes]

I had this problem and to isolate it I switched to a clean venv via pycharm and installed only magenta (without conda). I dug through the output and found that the common issue between my failure to get good output was magenta not finding my model directory. It was then that I realized the directory I was pointing it to had a typo, so it was failing to find a trained model, and understandably outputting gibberish because it was using some default model ill-suited to the task.

Looking through @xMcouro’s output, I also see buried in there the line:

INFO:tensorflow:Could not find trained model in model_dir: d/train, running initialization to predict.

So it looks like that may be the problem here too. Once I fixed the directory I was pointing to when running the command, it started working perfectly.

[2017224400tdxf]

I had this problem and to isolate it I switched to a clean venv via pycharm and installed only magenta (without conda). I dug through the output and found that the common issue between my failure to get good output was magenta not finding my model directory. It was then that I realized the directory I was pointing it to had a typo, so it was failing to find a trained model, and understandably outputting gibberish because it was using some default model ill-suited to the task.

Looking through @xMcouro’s output, I also see buried in there the line:

INFO:tensorflow:Could not find trained model in model_dir: d/train, running initialization to predict.

So it looks like that may be the problem here too. Once I fixed the directory I was pointing to when running the command, it started working perfectly.

Thank you!