Too large name issue

Hi @vsoch ,

I’ve not too much time to dive into this issue. An user is trying to upload a new image to the sregistry server. The image was uploaded and I can see it in the server side, but nothing is shown in the webUI.

This is the error:

uwsgi_1   | Internal Server Error: /api/uploads/complete/
uwsgi_1   | Traceback (most recent call last):
uwsgi_1   |   File "/usr/local/lib/python3.5/site-packages/django/db/backends/utils.py", line 65, in execute
uwsgi_1   |     return self.cursor.execute(sql, params)
uwsgi_1   | psycopg2.DataError: value too long for type character varying(100)
uwsgi_1   | 
uwsgi_1   | 
uwsgi_1   | The above exception was the direct cause of the following exception:
uwsgi_1   | 
uwsgi_1   | Traceback (most recent call last):
uwsgi_1   |   File "/usr/local/lib/python3.5/site-packages/django/core/handlers/exception.py", line 41, in inner
uwsgi_1   |     response = get_response(request)
uwsgi_1   |   File "/usr/local/lib/python3.5/site-packages/django/core/handlers/base.py", line 249, in _legacy_get_response
uwsgi_1   |     response = self._get_response(request)
uwsgi_1   |   File "/usr/local/lib/python3.5/site-packages/django/core/handlers/base.py", line 187, in _get_response
uwsgi_1   |     response = self.process_exception_by_middleware(e, request)
uwsgi_1   |   File "/usr/local/lib/python3.5/site-packages/django/core/handlers/base.py", line 185, in _get_response
uwsgi_1   |     response = wrapped_callback(request, *callback_args, **callback_kwargs)
uwsgi_1   |   File "/usr/local/lib/python3.5/site-packages/django/views/decorators/csrf.py", line 58, in wrapped_view
uwsgi_1   |     return view_func(*args, **kwargs)
uwsgi_1   |   File "./shub/apps/api/actions/upload.py", line 110, in upload_complete
uwsgi_1   |     size = size)
uwsgi_1   |   File "./shub/apps/api/actions/create.py", line 121, in upload_container
uwsgi_1   |     datafile=instance.file)
uwsgi_1   |   File "/usr/local/lib/python3.5/site-packages/django/db/models/manager.py", line 85, in manager_method
uwsgi_1   |     return getattr(self.get_queryset(), name)(*args, **kwargs)
uwsgi_1   |   File "/usr/local/lib/python3.5/site-packages/django/db/models/query.py", line 394, in create
uwsgi_1   |     obj.save(force_insert=True, using=self.db)
uwsgi_1   |   File "/usr/local/lib/python3.5/site-packages/django/db/models/base.py", line 806, in save
uwsgi_1   |     force_update=force_update, update_fields=update_fields)
uwsgi_1   |   File "/usr/local/lib/python3.5/site-packages/django/db/models/base.py", line 836, in save_base
uwsgi_1   |     updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
uwsgi_1   |   File "/usr/local/lib/python3.5/site-packages/django/db/models/base.py", line 922, in _save_table
uwsgi_1   |     result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
uwsgi_1   |   File "/usr/local/lib/python3.5/site-packages/django/db/models/base.py", line 961, in _do_insert
uwsgi_1   |     using=using, raw=raw)
uwsgi_1   |   File "/usr/local/lib/python3.5/site-packages/django/db/models/manager.py", line 85, in manager_method
uwsgi_1   |     return getattr(self.get_queryset(), name)(*args, **kwargs)
uwsgi_1   |   File "/usr/local/lib/python3.5/site-packages/django/db/models/query.py", line 1063, in _insert
uwsgi_1   |     return query.get_compiler(using=using).execute_sql(return_id)
uwsgi_1   |   File "/usr/local/lib/python3.5/site-packages/django/db/models/sql/compiler.py", line 1099, in execute_sql
uwsgi_1   |     cursor.execute(sql, params)
uwsgi_1   |   File "/usr/local/lib/python3.5/site-packages/django/db/backends/utils.py", line 65, in execute
uwsgi_1   |     return self.cursor.execute(sql, params)
uwsgi_1   |   File "/usr/local/lib/python3.5/site-packages/django/db/utils.py", line 94, in __exit__
uwsgi_1   |     six.reraise(dj_exc_type, dj_exc_value, traceback)
uwsgi_1   |   File "/usr/local/lib/python3.5/site-packages/django/utils/six.py", line 685, in reraise
uwsgi_1   |     raise value.with_traceback(tb)
uwsgi_1   |   File "/usr/local/lib/python3.5/site-packages/django/db/backends/utils.py", line 65, in execute
uwsgi_1   |     return self.cursor.execute(sql, params)
uwsgi_1   | django.db.utils.DataError: value too long for type character varying(100)

I think it’s related with a bit image name.

What do you think?

[vsoch]

@victorsndvg see https://github.com/singularityhub/sregistry/pull/155 would you be able to test?

[victorsndvg]

@vsoch ,

sorry for the delay, yesterday was a busy day.

I’ve tested it and I got:

[1. Collection return status 200 OK]
[================================] 91/91 MB - 00:00:01
[Return status 200 value too long for type character varying(200)

/var/www/images/mso4sc/test:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa[e-mail] must be less than 255 characters]

Great, it works as expected! Although I don’t like to tell users where the images are going to be stored in the server … for me, to return Collection/container:tag has more sense from the point of view of the sregistry-cli user. What do you thing?

I like to stress sregistry and I did more tests.

• With tags > 450 chars:

nginx_1   | 2018/10/11 06:34:06 [error] 6#6: *25 part header is too long,

• With 255 < tags < 450:

uwsgi_1   | Internal Server Error: /api/uploads/complete/
uwsgi_1   | Traceback (most recent call last):
uwsgi_1   |   File "/usr/local/lib/python3.5/shutil.py", line 538, in move
uwsgi_1   |     os.rename(src, real_dst)
uwsgi_1   | OSError: [Errno 36] File name too long: '/var/www/images/_upload/1/0000000001' -> '/var/www/images/mso4sc/test:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa[e-mail]'
uwsgi_1   | 
uwsgi_1   | During handling of the above exception, another exception occurred:
uwsgi_1   | 
uwsgi_1   | Traceback (most recent call last):
uwsgi_1   |   File "/usr/local/lib/python3.5/site-packages/django/core/handlers/exception.py", line 41, in inner
uwsgi_1   |     response = get_response(request)
uwsgi_1   |   File "/usr/local/lib/python3.5/site-packages/django/core/handlers/base.py", line 249, in _legacy_get_response
uwsgi_1   |     response = self._get_response(request)
uwsgi_1   |   File "/usr/local/lib/python3.5/site-packages/django/core/handlers/base.py", line 187, in _get_response
uwsgi_1   |     response = self.process_exception_by_middleware(e, request)
uwsgi_1   |   File "/usr/local/lib/python3.5/site-packages/django/core/handlers/base.py", line 185, in _get_response
uwsgi_1   |     response = wrapped_callback(request, *callback_args, **callback_kwargs)
uwsgi_1   |   File "/usr/local/lib/python3.5/site-packages/django/views/decorators/csrf.py", line 58, in wrapped_view
uwsgi_1   |     return view_func(*args, **kwargs)
uwsgi_1   |   File "./shub/apps/api/actions/upload.py", line 108, in upload_complete
uwsgi_1   |     size = size)
uwsgi_1   |   File "./shub/apps/api/actions/create.py", line 111, in upload_container
uwsgi_1   |     new_path = move_nginx_upload_to_storage(collection, upload_id, storage)
uwsgi_1   |   File "./shub/apps/api/actions/create.py", line 73, in move_nginx_upload_to_storage
uwsgi_1   |     shutil.move(source, new_path)
uwsgi_1   |   File "/usr/local/lib/python3.5/shutil.py", line 552, in move
uwsgi_1   |     copy_function(src, real_dst)
uwsgi_1   |   File "/usr/local/lib/python3.5/shutil.py", line 251, in copy2
uwsgi_1   |     copyfile(src, dst, follow_symlinks=follow_symlinks)
uwsgi_1   |   File "/usr/local/lib/python3.5/shutil.py", line 115, in copyfile
uwsgi_1   |     with open(dst, 'wb') as fdst:
uwsgi_1   | OSError: [Errno 36] File name too long: '/var/www/images/mso4sc/test:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa[e-mail]'

The first one is only informative, but maybe you want to catch the one related with the Filename too long OS exception.

From the point of view of sregistry-cli users, they always get meaningful error messages (500 or 400 status) when this errors occurs.

What do you think?