Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Error on creation of dynamic image sizes #4915

Closed
vmario89 opened this issue Mar 22, 2019 · 10 comments
Closed

Error on creation of dynamic image sizes #4915

vmario89 opened this issue Mar 22, 2019 · 10 comments
Labels
A-Media-Repository Uploading, downloading images and video, thumbnailing

Comments

@vmario89
Copy link

vmario89 commented Mar 22, 2019 •
edited by MadLittleMods
Loading

Hi,

in Riot i got the problem that when i want to post a sticker i cannot see the preview images but posting them works:
grafik

It seems that the Matrix instance does not accept the option dynamic_thumbnails.

grafik

An example of the image: the first one is not available but the last two are working.

As i understand the option the URL should return an image because Matrix should generate a cached image from source by the request with parameters width/height.

the image thingy worked once without problems but i dont know which point made the breaking change.

regards, Mario
@turt2live
Copy link
Member

turt2live commented Mar 22, 2019 •
edited
Loading

I'm suspecting this to be a server-side problem. Do you have any large stack traces in the server logs?

(Edit: in my haste, I thought this was riot-web, but the question still applies.)

@vmario89
Copy link
Author

vmario89 commented Mar 22, 2019 •
edited by richvdh
Loading

[edited to fix log formatting by @richvdh]

Hey. no big traces. I put synapse to DEBUG log state but nothing seriousi can see. But Dimension server puts in log:

Mär 22 02:38:29 h2606131.stratoserver.net matrix-dimension[28568]: Error: Can't set headers after they are sent.
Mär 22 02:38:29 h2606131.stratoserver.net matrix-dimension[28568]:     at SendStream.headersAlreadySent (/opt/matrix-dimension/node_modules/send/index.js:390:13)
Mär 22 02:38:29 h2606131.stratoserver.net matrix-dimension[28568]:     at SendStream.send (/opt/matrix-dimension/node_modules/send/index.js:618:10)
Mär 22 02:38:29 h2606131.stratoserver.net matrix-dimension[28568]:     at onstat (/opt/matrix-dimension/node_modules/send/index.js:730:10)
Mär 22 02:38:29 h2606131.stratoserver.net matrix-dimension[28568]:     at FSReqWrap.oncomplete (fs.js:155:5)
Mär 22 02:38:29 h2606131.stratoserver.net matrix-dimension[28568]: Mar-22-2019 02:38:29.636 +01:00 info [Webserver] Incoming request: GET /api/v1/dimension/media/thumbnail/t2bot.io/193408b58f5e1eb72d9bea13f23914e6?width=120&height=120&method=scale&animated=true

https://dimension.fablabchemnitz.de//api/v1/dimension/media/thumbnail/t2bot.io/193408b58f5e1eb72d9bea13f23914e6?width=120&height=120&method=scale&animated=true
leads to
https://matrix.fablabchemnitz.de/_matrix/media/r0/thumbnail/t2bot.io/193408b58f5e1eb72d9bea13f23914e6?width=120&height=120&method=scale&animated=true
which is not available for example

some images are working partially, some completely not

so i am guessing this is more related to Dimension instead of Synapse

@turt2live
Copy link
Member

Dimension isn't handling the error gracefully, but the fact that synapse is returning M_NOT_FOUND for a thumbnail is the problem. There should be at least something in the synapse logs to highlight the request and what happened though.

@L0ric0
Copy link
Contributor

L0ric0 commented Mar 22, 2019

I don't know if it is connected but I get this tracebacks since the last update, the config is mostly the same as vmario89s:

synapse.http.server: [GET-680119] Failed handle request via 'ThumbnailResource': <XForwardedForRequest at 0x7f7bb9a10f60 method='GET' uri='/_matrix/media/v1/thumbnail/web3r.de/oDSvAQybETGHteNLFuZgIOTU?width=120&height=120&method=scale' clientproto='HTTP/1.1' site=8008>
  Traceback (most recent call last):
    File "/usr/lib/python3/dist-packages/synapse/http/server.py", line 81, in wrapped_request_handler
      yield h(self, request)
    File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
      result = result.throwExceptionIntoGenerator(g)
    File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 491, in throwExceptionIntoGenerator
      return g.throw(self.type, self.value, self.tb)
    File "/usr/lib/python3/dist-packages/synapse/rest/media/v1/thumbnail_resource.py", line 78, in _async_render_GET
      width, height, method, m_type
    File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
      result = result.throwExceptionIntoGenerator(g)
    File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 491, in throwExceptionIntoGenerator
      return g.throw(self.type, self.value, self.tb)
    File "/usr/lib/python3/dist-packages/synapse/rest/media/v1/thumbnail_resource.py", line 221, in _select_or_generate_remote_thumbnail
      desired_height, desired_method, desired_type
    File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
      result = result.throwExceptionIntoGenerator(g)
    File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 491, in throwExceptionIntoGenerator
      return g.throw(self.type, self.value, self.tb)
    File "/usr/lib/python3/dist-packages/synapse/rest/media/v1/media_repository.py", line 540, in generate_remote_exact_thumbnail
      t_width, t_height, t_type, t_method, t_len
    File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
      result = result.throwExceptionIntoGenerator(g)
    File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 491, in throwExceptionIntoGenerator
      return g.throw(self.type, self.value, self.tb)
    File "/usr/lib/python3/dist-packages/synapse/storage/_base.py", line 516, in _simple_insert
      self._simple_insert_txn, table, values,
    File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
      result = result.throwExceptionIntoGenerator(g)
    File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 491, in throwExceptionIntoGenerator
      return g.throw(self.type, self.value, self.tb)
    File "/usr/lib/python3/dist-packages/synapse/storage/_base.py", line 408, in runInteraction
      *args, **kwargs
    File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
      result = result.throwExceptionIntoGenerator(g)
    File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 491, in throwExceptionIntoGenerator
      return g.throw(self.type, self.value, self.tb)
    File "/usr/lib/python3/dist-packages/synapse/storage/_base.py", line 457, in runWithConnection
      inner_func, *args, **kwargs
    File "/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 250, in inContext
      result = inContext.theWork()
    File "/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 266, in <lambda>
      inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
    File "/usr/lib/python3/dist-packages/twisted/python/context.py", line 122, in callWithContext
      return self.currentContext().callWithContext(ctx, func, *args, **kw)
    File "/usr/lib/python3/dist-packages/twisted/python/context.py", line 85, in callWithContext
      return func(*args,**kw)
    File "/usr/lib/python3/dist-packages/twisted/enterprise/adbapi.py", line 306, in _runWithConnection
      compat.reraise(excValue, excTraceback)
    File "/usr/lib/python3/dist-packages/twisted/python/compat.py", line 464, in reraise
      raise exception.with_traceback(traceback)
    File "/usr/lib/python3/dist-packages/twisted/enterprise/adbapi.py", line 297, in _runWithConnection
      result = func(conn, *args, **kw)
    File "/usr/lib/python3/dist-packages/synapse/storage/_base.py", line 453, in inner_func
      return func(conn, *args, **kwargs)
    File "/usr/lib/python3/dist-packages/synapse/storage/_base.py", line 329, in _new_transaction
      r = func(txn, *args, **kwargs)
    File "/usr/lib/python3/dist-packages/synapse/storage/_base.py", line 536, in _simple_insert_txn
      txn.execute(sql, vals)
    File "/usr/lib/python3/dist-packages/synapse/storage/_base.py", line 119, in execute
      self._do_execute(self.txn.execute, sql, *args)
    File "/usr/lib/python3/dist-packages/synapse/storage/_base.py", line 149, in _do_execute
      sql, *args
psycopg2.IntegrityError: FEHLER:  doppelter Schlüsselwert verletzt Unique-Constraint »remote_media_cache_thumbnails_media_origin_media_id_thumbna_key«
DETAIL:  Schlüssel »(media_origin, media_id, thumbnail_width, thumbnail_height, thumbnail_type)=(web3r.de, oDSvAQybETGHteNLFuZgIOTU, 120, 120, image/png)« existiert bereits.

@richvdh
Copy link
Member

richvdh commented Mar 22, 2019

@turt2live where do you see M_NOT_FOUND?

@vmario89 please find the "Received request" line in your synapse logs for a URL which doesn't work and share the logs as per https://github.com/matrix-org/synapse/wiki#how-can-i-find-the-lines-corresponding-to-a-given-http-request-in-my-homeserver-log

@vmario89
Copy link
Author

vmario89 commented Mar 22, 2019 •
edited
Loading

Hi @richvdh , the following i found in log. seems to be same behaviour like @L0ric0 has got.

2019-03-22 10:42:59,016 - synapse.http.server - 112 - ERROR - GET-3590- Failed handle request via 'ThumbnailResource': <XForwardedForRequest at 0x7f5490213d68 method='GET' uri='/_matrix/media/r0/thumbnail/t2bot.io/193408b58f5e1eb72d9bea13f23914e6?width=120&height=120&method=scale&animated=true' clientproto='HTTP/1.1' site=8008>
Traceback (most recent call last):
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/synapse/http/server.py", line 81, in wrapped_request_handler yield h(self, request)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks result = result.throwExceptionIntoGenerator(g)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/python/failure.py", line 491, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/synapse/rest/media/v1/thumbnail_resource.py", line 78, in _async_render_GET width, height, method, m_type
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks result = result.throwExceptionIntoGenerator(g)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/python/failure.py", line 491, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/synapse/rest/media/v1/thumbnail_resource.py", line 221, in _select_or_generate_remote_thumbnail desired_height, desired_method, desired_type
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks result = result.throwExceptionIntoGenerator(g)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/python/failure.py", line 491, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/synapse/rest/media/v1/media_repository.py", line 540, in generate_remote_exact_thumbnail t_width, t_height, t_type, t_method, t_len
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks result = result.throwExceptionIntoGenerator(g)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/python/failure.py", line 491, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/synapse/storage/_base.py", line 516, in _simple_insert self._simple_insert_txn, table, values,
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks result = result.throwExceptionIntoGenerator(g)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/python/failure.py", line 491, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/synapse/storage/_base.py", line 408, in runInteraction *args, **kwargs
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks result = result.throwExceptionIntoGenerator(g)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/python/failure.py", line 491, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/synapse/storage/_base.py", line 457, in runWithConnection inner_func, *args, **kwargs
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/python/threadpool.py", line 250, in inContext result = inContext.theWork()
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/python/threadpool.py", line 266, in <lambda> inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/python/context.py", line 122, in callWithContext return self.currentContext().callWithContext(ctx, func, *args, **kw)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/python/context.py", line 85, in callWithContext return func(*args,**kw)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/enterprise/adbapi.py", line 306, in _runWithConnection compat.reraise(excValue, excTraceback)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/python/compat.py", line 464, in reraise raise exception.with_traceback(traceback)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/twisted/enterprise/adbapi.py", line 297, in _runWithConnection result = func(conn, *args, **kw)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/synapse/storage/_base.py", line 453, in inner_func return func(conn, *args, **kwargs)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/synapse/storage/_base.py", line 329, in _new_transaction r = func(txn, *args, **kwargs)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/synapse/storage/_base.py", line 536, in _simple_insert_txn txn.execute(sql, vals)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/synapse/storage/_base.py", line 119, in execute self._do_execute(self.txn.execute, sql, *args)
 File "/opt/venvs/matrix-synapse/lib/python3.5/site-packages/synapse/storage/_base.py", line 149, in _do_execute sql, *args
psycopg2.IntegrityError: duplicate key value violates unique constraint "remote_media_cache_thumbnails_media_origin_media_id_thumbna_key"
DETAIL: Key (media_origin, media_id, thumbnail_width, thumbnail_height, thumbnail_type)=(t2bot.io, 193408b58f5e1eb72d9bea13f23914e6, 120, 120, image/png) already exists.

@richvdh
Copy link
Member

richvdh commented Mar 22, 2019

thanks. can you share any other log lines for GET-3590-.

@vmario89
Copy link
Author

hey. this comes before the error entry:

2019-03-22 10:42:58,815 - synapse.access.http.8008 - 233 - INFO - GET-3590- 131.246.146.10 - 8008 - Received request: GET /_matrix/media/r0/thumbnail/t2bot.io/193408b58f5e1eb72d9bea13f23914e6?width=120&height=120&method=scale&animated=true
2019-03-22 10:42:58,816 - synapse.util.async_helpers - 205 - DEBUG - GET-3590- Acquired uncontended linearizer lock 'media_remote' for key ('t2bot.io', '193408b58f5e1eb72d9bea13f23914e6')
2019-03-22 10:42:58,816 - synapse.metrics - 372 - INFO - - Collecting gc 0
2019-03-22 10:42:58,817 - synapse.storage.txn - 317 - DEBUG - GET-3590- [TXN START] {get_cached_remote_media-764a}
2019-03-22 10:42:58,818 - synapse.storage.txn - 373 - DEBUG - GET-3590- [TXN END] {get_cached_remote_media-764a} 0.001629 sec
2019-03-22 10:42:58,822 - synapse.util.async_helpers - 218 - DEBUG - GET-3590- Releasing linearizer lock 'media_remote' for key ('t2bot.io', '193408b58f5e1eb72d9bea13f23914e6')
2019-03-22 10:42:58,822 - synapse.storage.txn - 317 - DEBUG - GET-3590- [TXN START] {get_remote_media_thumbnails-764b}
2019-03-22 10:42:58,823 - synapse.storage.txn - 373 - DEBUG - GET-3590- [TXN END] {get_remote_media_thumbnails-764b} 0.000840 sec
2019-03-22 10:42:58,832 - synapse.rest.media.v1.thumbnail_resource - 216 - DEBUG - GET-3590- We don't have a thumbnail of that size. Generating
2019-03-22 10:42:59,012 - synapse.rest.media.v1.media_repository - 534 - INFO - GET-3590- Stored thumbnail in file '/var/lib/matrix-synapse/media/remote_thumbnail/t2bot.io/19/34/08b58f5e1eb72d9bea13f23914e6/120-120-image-png'
2019-03-22 10:42:59,013 - synapse.storage.txn - 317 - DEBUG - GET-3590- [TXN START] {store_remote_media_thumbnail-764c}
2019-03-22 10:42:59,015 - synapse.storage._base - 152 - DEBUG - GET-3590- [SQL FAIL] {store_remote_media_thumbnail-764c} duplicate key value violates unique constraint "remote_media_cache_thumbnails_media_origin_media_id_thumbna_key"
DETAIL:  Key (media_origin, media_id, thumbnail_width, thumbnail_height, thumbnail_type)=(t2bot.io, 193408b58f5e1eb72d9bea13f23914e6, 120, 120, image/png) already exists.

2019-03-22 10:42:59,015 - synapse.storage._base - 365 - DEBUG - GET-3590- [TXN FAIL] {store_remote_media_thumbnail-764c} duplicate key value violates unique constraint "remote_media_cache_thumbnails_media_origin_media_id_thumbna_key"
DETAIL:  Key (media_origin, media_id, thumbnail_width, thumbnail_height, thumbnail_type)=(t2bot.io, 193408b58f5e1eb72d9bea13f23914e6, 120, 120, image/png) already exists.

2019-03-22 10:42:59,015 - synapse.storage.txn - 373 - DEBUG - GET-3590- [TXN END] {store_remote_media_thumbnail-764c} 0.001923 sec

@richvdh
Copy link
Member

richvdh commented Mar 22, 2019

Great, thanks. This looks like a duplicate of #2182 so I will close this and add your logs to that issue.

@richvdh richvdh closed this as completed Mar 22, 2019
@richvdh richvdh mentioned this issue Mar 22, 2019
Closed
@turt2live
Copy link
Member

@turt2live where do you see M_NOT_FOUND?

Showed up when I clicked the links yesterday. Seems fixed now though

@MadLittleMods MadLittleMods added the A-Media-Repository Uploading, downloading images and video, thumbnailing label Jun 10, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
A-Media-Repository Uploading, downloading images and video, thumbnailing
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@MadLittleMods @turt2live @richvdh @vmario89 @L0ric0