[ERROR]: Training different data -> IndexError: index 3 is out of bounds for axis 2 with size 3

Hello, first of all thank you for developing solaris. I’ve been working on object detection for a long time. But I’m new to Github. So I’m sorry for my faults!

I tried to train with my own data but I got an error. I received an error: IndexError: index 3 is out of bounds for axis 2 with size 3

As you mentioned in the document, I divided the satellite image (in tif format) into tiles. Then I divided geojson files in the same way. I did the mask creation process. Again I created my mask(footprint mask) in tif format. Then I created the training and test csv files as you specified. I have edited the configuration file of the pre-trained model xdxd_spacenet4.

Error Message

solaris_run_ml -c xdxd_spacenet4.yml

When I run this command, I get the error like above.

(solaris) deposerver@ubuntu:/mnt/depo1tb/yz/solaris/solaris/nets/configs$ solaris_run_ml -c xdxd_spacenet4.yml
/home/deposerver/anaconda3/envs/solaris/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:526: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/home/deposerver/anaconda3/envs/solaris/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:527: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/home/deposerver/anaconda3/envs/solaris/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:528: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/home/deposerver/anaconda3/envs/solaris/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:529: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/home/deposerver/anaconda3/envs/solaris/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:530: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/home/deposerver/anaconda3/envs/solaris/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:535: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
/home/deposerver/.local/lib/python3.6/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters
Traceback (most recent call last):
  File "/home/deposerver/anaconda3/envs/solaris/bin/solaris_run_ml", line 10, in <module>
    sys.exit(main())
  File "/home/deposerver/anaconda3/envs/solaris/lib/python3.6/site-packages/solaris/bin/solaris_run_ml.py", line 34, in main
    inferer(inf_df)
  File "/home/deposerver/anaconda3/envs/solaris/lib/python3.6/site-packages/solaris/nets/infer.py", line 64, in __call__
    src_im_height, src_im_width) = inf_tiler(im_path)
  File "/home/deposerver/anaconda3/envs/solaris/lib/python3.6/site-packages/solaris/nets/datagen.py", line 294, in __call__
    subarr = self.aug(image=subarr)['image']
  File "/home/deposerver/anaconda3/envs/solaris/lib/python3.6/site-packages/albumentations/core/composition.py", line 176, in __call__
    data = t(force_apply=force_apply, **data)
  File "/home/deposerver/anaconda3/envs/solaris/lib/python3.6/site-packages/albumentations/core/transforms_interface.py", line 87, in __call__
    return self.apply_with_params(params, **kwargs)
  File "/home/deposerver/anaconda3/envs/solaris/lib/python3.6/site-packages/albumentations/core/transforms_interface.py", line 100, in apply_with_params
    res[key] = target_function(arg, **dict(params, **target_dependencies))
  File "/home/deposerver/anaconda3/envs/solaris/lib/python3.6/site-packages/solaris/nets/transform.py", line 101, in apply
    return np.delete(im_arr, self.idx, self.axis)
  File "<__array_function__ internals>", line 6, in delete
  File "/home/deposerver/.local/lib/python3.6/site-packages/numpy/lib/function_base.py", line 4382, in delete
    "size %i" % (obj, axis, N))
IndexError: index 3 is out of bounds for axis 2 with size 3

How can I solve this problem? Anybody have any ideas? Thanks in advance.

What should I do?

I have some questions. It would be very helpful if you could help.

1. What should be the image format? (tif, png, jpeg etc.)
2. RBG or BGR ? (Assuming it is tif)
3. Does the bit information of the image matter? 24, 16, 8 etc. (Assuming it is tif)
4. Should the mask be in the form of images? Which file format should I use exactly?
5. Is there a service that automatically generates training and test csv? I could not see. It can be very helpful.

Environment information

• OS: Ubuntu 18.04
• solaris version: 0.1.3
• python version: 3.6.7 and 2.7.15 (I’ve just installed your env)

[nrweir]

Do you mean pre-processing in terms of tiling or image augmentation before it’s fed into the model?

Either way, the present answer is no but an enterprising user would be welcome to make a PR.

If you want to encourage that, I’d recommend creating a new issue here for it - I’m going to close this one since we’ve moved fairly far afield from the original question.

[williamobrein]

Can we run solaris preprocessing on the GPU? Because it takes too long on the CPU.