Admin error when using some of file storage backends

[machakux]

NotImplementedError: This backend doesn't support absolute paths is raised when opening e-mail messages with attachments in admin interface when using some remote file storage backends (via django-storages).

I think the error is a result of trying to open attachments files directly using open() function; For example as in Message._rehydrate() method.

May be a possibe soulution could be replacing direct calls to open() with open method from django storage backends; For example using something like attachment.document.open('rb') instead of open(attachment.document.path, 'rb')

Sample error log

Internal Server Error: /admin/django_mailbox/message/<id>/change/
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/django/db/models/options.py", line 676, in get_field
  return self.fields_map[field_name]
         ~~~~~~~~~~~~~~~^^^^^^^^^^^^
KeyError: 'envelope_headers'


During handling of the above exception, another exception occurred:

Traceback (most recent call last):

File "/usr/local/lib/python3.11/site-packages/django/contrib/admin/utils.py", line 290, in lookup_field
  f = _get_non_gfk_field(opts, name)
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/django/contrib/admin/utils.py", line 330, in _get_non_gfk_field
  field = opts.get_field(name)
          ^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/django/db/models/options.py", line 678, in get_field
      raise FieldDoesNotExist(
django.core.exceptions.FieldDoesNotExist: Message has no field named 'envelope_headers'

During handling of the above exception, another exception occurred:
    Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/django/core/handlers/exception.py", line 55, in inner
  response = get_response(request)
             ^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/django/core/handlers/base.py", line 220, in _get_response
  response = response.render()
             ^^^^^^^^^^^^^^^^^

 ...

            ^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/django/contrib/admin/helpers.py", line 275, in contents
  f, attr, value = lookup_field(field, obj, model_admin)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/django/contrib/admin/utils.py", line 299, in lookup_field
  value = attr(obj)
          ^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/django_mailbox/admin.py", line 76, in envelope_headers
  email = msg.get_email_object()
          ^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/django_mailbox/models.py", line 798, in get_email_object
  self._email_object = self._rehydrate(flat)
                       ^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/django_mailbox/models.py", line 699, in _rehydrate
  self._rehydrate(part)
File "/usr/local/lib/python3.11/site-packages/django_mailbox/models.py", line 728, in _rehydrate
  with open(attachment.document.path, 'rb') as f:
            ^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/django/db/models/fields/files.py", line 65, in path
  return self.storage.path(self.name)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/django/core/files/storage/base.py", line 137, in path
  raise NotImplementedError("This backend doesn't support absolute paths.")

NotImplementedError: This backend doesn't support absolute paths

[jordanambra]

I can confirm that this is also happening to me, using django-storages and S3/Boto, after upgrading from an older version.

It seems like it changed beginning in 4.10.0, looking at this commit: 7e250b2

@martinmanzo any thoughts on why this had to move to using open() instead of attachment.document.read()?

[pfouque]

Hi @jordanambra,
Thanks for your feedback, According to PR #273 there was an issue when too many files were opened.
I assume it was solved by using "with" which garantee to have only one file opened at once.
There is an open PR #303 that tries to solve this, but it's only rollbacking but doesn't consider the #273 issue.
Maybe you can contribute to this PR or open a new one to solve both issues at once.

[martinmanzo]

Hi @jordanambra, I was having OS errors due to files not being closed after being read and that solution did it for me.

BTW I think you may be right, open(attachment.document.path, 'rb') could be changed for something like:

raw = attachment.document.read()
attachment.document.close()
encode_quopri(BytesIO(raw), output, quotetabs=True, header=False)

And the same for the other changes in 7e250b2

Please note that I was using this in Django 3. Maybe the new versions read() closes the file.

Regards 💪

[pfouque]

Please note that I was using this in Django 3. Maybe the new versions read() closes the file.

I don't think so.

Indeed closing it explicitly should work but maybe we can also use it as a context manager:

with attachment.document.open() as f:
    encode_quopri(BytesIO(f.read()), output, quotetabs=True, header=False)

Both solutions should work!

@martinmanzo & @jordanambra can you give it a try to check both your issues are solved? 🙏

[martinmanzo]

Sure, I'll need a couple of days to free some time and give it a go.