Rate limit assets list endpoint for unauthenticated users #1899

mvandenburgh · 2024-03-20T20:02:51Z

This PR adds a rate limit of 60 requests per minute to the asset list endpoint for unauthenticated users. Logged-in users are not rate-limited. This is a short-term fix for #1891.

jjnesbitt · 2024-03-20T20:19:59Z

The overall look and feel of it make sense to me, although it seems there's some issues to work through regarding the tests.

mvandenburgh · 2024-03-20T20:41:47Z

The overall look and feel of it make sense to me, although it seems there's some issues to work through regarding the tests.

The tests were getting rate-limited too 😅 I just removed the test, as there doesn't appear to be a clean way to integrate the rate-limiting configuration from the production configuration.

dandiapi/api/views/asset.py

test still failing

yarikoptic · 2024-03-21T18:04:23Z

are there easy means on heroku to rate limit specific IPs? IMHO as a temporary workaround it could be less invasive approach to mitigate whenever we detect flood of requests to list 000026 . There were only 2-3 IPs in the past days which were heavily (a few thousand pagination requests) bombarding 000026 with listing requests each day....

yarikoptic · 2024-03-21T18:06:38Z

also FWIW -- it seems that such extensive requests come from our dandi-cli (attn @jwodder ) which makes it so easy to paginate/get full lists

dandi@drogon:/mnt/backup/dandi/heroku-logs/dandi-api$ for d in {19..21}; do echo  "$d "; grep -e '000026.*/assets/\(?page=\|[^?p0-9a-e]\)' 202403$d-*.log  | grep '\[web'; done | head
19 
20240319-0001.log:2024-03-19T04:10:38.643112+00:00 app[web.1]: 10.1.88.179 - - [19/Mar/2024:04:10:38 +0000] "GET /api/dandisets/000026/versions/draft/assets/ HTTP/1.1" 200 31687 "-" "dandi/0.60.0 requests/2.31.0 CPython/3.8.18"
20240319-0001.log:2024-03-19T04:10:38.788757+00:00 app[web.1]: 10.1.31.245 - - [19/Mar/2024:04:10:38 +0000] "GET /api/dandisets/000026/versions/draft/assets/ HTTP/1.1" 200 31687 "-" "dandi/0.60.0 requests/2.31.0 CPython/3.8.18"
20240319-0001.log:2024-03-19T04:10:39.064803+00:00 app[web.1]: 10.1.11.171 - - [19/Mar/2024:04:10:39 +0000] "GET /api/dandisets/000026/versions/draft/assets/ HTTP/1.1" 200 31687 "-" "dandi/0.60.0 requests/2.31.0 CPython/3.8.18"
20240319-0001.log:2024-03-19T04:10:39.144945+00:00 app[web.1]: 10.1.26.139 - - [19/Mar/2024:04:10:39 +0000] "GET /api/dandisets/000026/versions/draft/assets/ HTTP/1.1" 200 31687 "-" "dandi/0.60.0 requests/2.31.0 CPython/3.8.18"
20240319-0001.log:2024-03-19T04:10:40.338017+00:00 app[web.1]: 10.1.88.90 - - [19/Mar/2024:04:10:40 +0000] "GET /api/dandisets/000026/versions/draft/assets/ HTTP/1.1" 200 31687 "-" "dandi/0.60.0 requests/2.31.0 CPython/3.8.18"
20240319-0001.log:2024-03-19T04:10:40.468877+00:00 app[web.1]: 10.1.22.211 - - [19/Mar/2024:04:10:40 +0000] "GET /api/dandisets/000026/versions/draft/assets/ HTTP/1.1" 200 31687 "-" "dandi/0.60.0 requests/2.31.0 CPython/3.8.18"
20240319-0001.log:2024-03-19T04:10:40.774544+00:00 app[web.1]: 10.1.88.179 - - [19/Mar/2024:04:10:40 +0000] "GET /api/dandisets/000026/versions/draft/assets/?page=2 HTTP/1.1" 200 34108 "-" "dandi/0.60.0 requests/2.31.0 CPython/3.8.18"
20240319-0001.log:2024-03-19T04:10:40.839495+00:00 app[web.1]: 10.1.31.245 - - [19/Mar/2024:04:10:40 +0000] "GET /api/dandisets/000026/versions/draft/assets/?page=3 HTTP/1.1" 200 34281 "-" "dandi/0.60.0 requests/2.31.0 CPython/3.8.18"
20240319-0001.log:2024-03-19T04:10:41.959202+00:00 app[web.1]: 10.1.31.84 - - [19/Mar/2024:04:10:41 +0000] "GET /api/dandisets/000026/versions/draft/assets/?page=4 HTTP/1.1" 200 32943 "-" "dandi/0.60.0 requests/2.31.0 CPython/3.8.18"

so that users might be just running naive scripts getting a full listing over dandiset even when not needing it... not sure if there is a sensible way to mitigate on dandi-cli level though .

dandiapi/api/views/asset.py

yarikoptic · 2024-03-21T21:38:29Z

dandiapi/settings.py

+        # By default, set request rate limit to a very high number, effectively disabling it.
+        configuration.REST_FRAMEWORK['DEFAULT_THROTTLE_RATES'] = {
+            'anon': f'{sys.maxsize}/minute',
+        }


even if you trick client for testing -- how would it appear to people who actually use production instance, i.e.
will client now just fail any request to list 000026 (over 60 pages) without providing any further guidance?

The client doesn't list assets from that endpoint. The file browser uses the /assets/paths endpoint.

sorry -- by client I meant the dandi-cli -- the python library+client, not web UI.

Ah I see. The rate limit only applies to anonymous users, so if you're logged in with the python client, you won't be rate limited. However yes, if not, it would apply to the python client all the same. I'm not sure how the client handles a 429 response.

not sure yet how would handle 429, but ATM dandi-cli tests seems to be full of tracebacks reported back from dandi-archive with 500 response,

e.g. like the top of this one

2024-03-21T22:04:26.7639012Z WARNING dandi:dandiapi.py:229 Will retry: Error 500 while sending GET request to http://localhost:8000/api/dandisets/000008/versions/draft/assets/: <!DOCTYPE html> 2024-03-21T22:04:26.7640125Z <html lang="en"> 2024-03-21T22:04:26.7640430Z <head> 2024-03-21T22:04:26.7641284Z <meta http-equiv="content-type" content="text/html; charset=utf-8"> 2024-03-21T22:04:26.7642092Z <meta name="robots" content="NONE,NOARCHIVE"> 2024-03-21T22:04:26.7642920Z <title>ProgrammingError 2024-03-21T22:04:26.7643504Z at /api/dandisets/000008/versions/draft/assets/</title> 2024-03-21T22:04:26.7644119Z <style type="text/css"> 2024-03-21T22:04:26.7644545Z html * { padding:0; margin:0; } 2024-03-21T22:04:26.7645043Z body * { padding:10px 20px; } 2024-03-21T22:04:26.7645512Z body * * { padding:0; } 2024-03-21T22:04:26.7646253Z body { font:small sans-serif; background-color:#fff; color:#000; } 2024-03-21T22:04:26.7647106Z body>div { border-bottom:1px solid #ddd; } 2024-03-21T22:04:26.7647737Z h1 { font-weight:normal; } 2024-03-21T22:04:26.7648189Z h2 { margin-bottom:.8em; } 2024-03-21T22:04:26.7648626Z h3 { margin:1em 0 .5em 0; } 2024-03-21T22:04:26.7649186Z h4 { margin:0 0 .5em 0; font-weight: normal; } 2024-03-21T22:04:26.7650033Z code, pre { font-size: 100%; white-space: pre-wrap; word-break: break-word; } 2024-03-21T22:04:26.7650765Z summary { cursor: pointer; } 2024-03-21T22:04:26.7651547Z table { border:1px solid #ccc; border-collapse: collapse; width:100%; background:white; } 2024-03-21T22:04:26.7652551Z tbody td, tbody th { vertical-align:top; padding:2px 3px; } 2024-03-21T22:04:26.7653183Z thead th { 2024-03-21T22:04:26.7653798Z padding:1px 6px 1px 3px; background:#fefefe; text-align:left; 2024-03-21T22:04:26.7654731Z font-weight:normal; font-size:11px; border:1px solid #ddd; 2024-03-21T22:04:26.7655317Z } 2024-03-21T22:04:26.7655994Z tbody th { width:12em; text-align:right; color:#666; padding-right:.5em; } 2024-03-21T22:04:26.7656850Z table.vars { margin:5px 10px 2px 40px; width: auto; } 2024-03-21T22:04:26.7657649Z table.vars td, table.req td { font-family:monospace; } 2024-03-21T22:04:26.7658300Z table td.code { width:100%; } 2024-03-21T22:04:26.7658841Z table td.code pre { overflow:hidden; } 2024-03-21T22:04:26.7659364Z table.source th { color:#666; } 2024-03-21T22:04:26.7660232Z table.source td { font-family:monospace; white-space:pre; border-bottom:1px solid #eee; } 2024-03-21T22:04:26.7661203Z ul.traceback { list-style-type:none; color: #222; } 2024-03-21T22:04:26.7661922Z ul.traceback li.cause { word-break: break-word; } 2024-03-21T22:04:26.7662672Z ul.traceback li.frame { padding-bottom:1em; color:#4f4f4f; } 2024-03-21T22:04:26.7663504Z ul.traceback li.user { background-color:#e0e0e0; color:#000 } 2024-03-21T22:04:26.7664238Z div.context { padding:10px 0; overflow:hidden; } 2024-03-21T22:04:26.7665207Z div.context ol { padding-left:30px; margin:0 10px; list-style-position: inside; } 2024-03-21T22:04:26.7666597Z div.context ol li { font-family:monospace; white-space:pre; color:#777; cursor:pointer; padding-left: 2px; } 2024-03-21T22:04:26.7667643Z div.context ol li pre { display:inline; } 2024-03-21T22:04:26.7668804Z div.context ol.context-line li { color:#464646; background-color:#dfdfdf; padding: 3px 2px; } 2024-03-21T22:04:26.7670026Z div.context ol.context-line li span { position:absolute; right:32px; } 2024-03-21T22:04:26.7671183Z .user div.context ol.context-line li { background-color:#bbb; color:#000; } 2024-03-21T22:04:26.7671979Z .user div.context ol li { color:#666; } 2024-03-21T22:04:26.7672714Z div.commands, summary.commands { margin-left: 40px; } 2024-03-21T22:04:26.7673661Z div.commands a, summary.commands { color:#555; text-decoration:none; } 2024-03-21T22:04:26.7674466Z .user div.commands a { color: black; } 2024-03-21T22:04:26.7675089Z #summary { background: #ffc; } 2024-03-21T22:04:26.7675817Z #summary h2 { font-weight: normal; color: #666; } 2024-03-21T22:04:26.7676494Z #explanation { background:#eee; } 2024-03-21T22:04:26.7677288Z #template, #template-not-exist { background:#f6f6f6; } 2024-03-21T22:04:26.7678191Z #template-not-exist ul { margin: 0 0 10px 20px; } 2024-03-21T22:04:26.7679481Z #template-not-exist .postmortem-section { margin-bottom: 3px; } 2024-03-21T22:04:26.7680433Z #unicode-hint { background:#eee; } 2024-03-21T22:04:26.7681311Z #traceback { background:#eee; } 2024-03-21T22:04:26.7682147Z #requestinfo { background:#f6f6f6; padding-left:120px; } 2024-03-21T22:04:26.7683028Z #summary table { border:none; background:transparent; } 2024-03-21T22:04:26.7684116Z #requestinfo h2, #requestinfo h3 { position:relative; margin-left:-100px; } 2024-03-21T22:04:26.7685065Z #requestinfo h3 { margin-bottom:-1em; } 2024-03-21T22:04:26.7685684Z .error { background: #ffc; } 2024-03-21T22:04:26.7686325Z .specific { color:#cc3300; font-weight:bold; } 2024-03-21T22:04:26.7687141Z h2 span.commands { font-size:.7em; font-weight:normal; } 2024-03-21T22:04:26.7687832Z span.commands a:link {color:#5E5694;} 2024-03-21T22:04:26.7688984Z pre.exception_value { font-family: sans-serif; color: #575757; font-size: 1.5em; margin: 10px 0 10px 0; } 2024-03-21T22:04:26.7690113Z .append-bottom { margin-bottom: 10px; } 2024-03-21T22:04:26.7690742Z .fname { user-select: all; } 2024-03-21T22:04:26.7691187Z </style> 2024-03-21T22:04:26.7691505Z 2024-03-21T22:04:26.7691796Z <script> 2024-03-21T22:04:26.7692143Z function hideAll(elems) { 2024-03-21T22:04:26.7692664Z for (var e = 0; e < elems.length; e++) { 2024-03-21T22:04:26.7693326Z elems[e].style.display = 'none'; 2024-03-21T22:04:26.7693821Z } 2024-03-21T22:04:26.7694117Z } 2024-03-21T22:04:26.7694461Z window.onload = function() { 2024-03-21T22:04:26.7695167Z hideAll(document.querySelectorAll('ol.pre-context')); 2024-03-21T22:04:26.7696054Z hideAll(document.querySelectorAll('ol.post-context')); 2024-03-21T22:04:26.7696924Z hideAll(document.querySelectorAll('div.pastebin')); 2024-03-21T22:04:26.7697525Z } 2024-03-21T22:04:26.7697849Z function toggle() { 2024-03-21T22:04:26.7698335Z for (var i = 0; i < arguments.length; i++) { 2024-03-21T22:04:26.7699016Z var e = document.getElementById(arguments[i]); 2024-03-21T22:04:26.7699610Z if (e) { 2024-03-21T22:04:26.7700269Z e.style.display = e.style.display == 'none' ? 'block': 'none'; 2024-03-21T22:04:26.7700958Z } 2024-03-21T22:04:26.7701272Z } 2024-03-21T22:04:26.7701582Z return false; 2024-03-21T22:04:26.7701929Z } 2024-03-21T22:04:26.7702314Z function switchPastebinFriendly(link) { 2024-03-21T22:04:26.7703002Z s1 = "Switch to copy-and-paste view"; 2024-03-21T22:04:26.7703595Z s2 = "Switch back to interactive view"; 2024-03-21T22:04:26.7704316Z link.textContent = link.textContent.trim() == s1 ? s2: s1; 2024-03-21T22:04:26.7705185Z toggle('browserTraceback', 'pastebinTraceback'); 2024-03-21T22:04:26.7705779Z return false; 2024-03-21T22:04:26.7706132Z } 2024-03-21T22:04:26.7706430Z </script> 2024-03-21T22:04:26.7706731Z 2024-03-21T22:04:26.7707016Z </head> 2024-03-21T22:04:26.7707310Z <body> 2024-03-21T22:04:26.7707616Z <div id="summary"> 2024-03-21T22:04:26.7708006Z <h1>ProgrammingError 2024-03-21T22:04:26.7708749Z at /api/dandisets/000008/versions/draft/assets/</h1> 2024-03-21T22:04:26.7709720Z <pre class="exception_value">relation "dandi_cache_table" does not exist 2024-03-21T22:04:26.7711007Z LINE 1: SELECT "cache_key", "value", "expires" FROM "dandi_cache_tab... 2024-03-21T22:04:26.7712010Z ^ 2024-03-21T22:04:26.7712529Z </pre> 2024-03-21T22:04:26.7712857Z <table class="meta"> 2024-03-21T22:04:26.7713127Z 2024-03-21T22:04:26.7713257Z <tr> 2024-03-21T22:04:26.7713643Z <th>Request Method:</th> 2024-03-21T22:04:26.7714000Z <td>GET</td> 2024-03-21T22:04:26.7714400Z </tr> 2024-03-21T22:04:26.7714694Z <tr> 2024-03-21T22:04:26.7715031Z <th>Request URL:</th> 2024-03-21T22:04:26.7715723Z <td>http://localhost:8000/api/dandisets/000008/versions/draft/assets/</td> 2024-03-21T22:04:26.7716490Z </tr> 2024-03-21T22:04:26.7716683Z 2024-03-21T22:04:26.7717076Z <tr> 2024-03-21T22:04:26.7717486Z <th>Django Version:</th> 2024-03-21T22:04:26.7718176Z <td>4.1.13</td> 2024-03-21T22:04:26.7718594Z </tr> 2024-03-21T22:04:26.7718774Z 2024-03-21T22:04:26.7718889Z <tr> 2024-03-21T22:04:26.7719256Z <th>Exception Type:</th> 2024-03-21T22:04:26.7719742Z <td>ProgrammingError</td> 2024-03-21T22:04:26.7720187Z </tr> 2024-03-21T22:04:26.7720370Z 2024-03-21T22:04:26.7720380Z 2024-03-21T22:04:26.7720496Z <tr> 2024-03-21T22:04:26.7720860Z <th>Exception Value:</th> 2024-03-21T22:04:26.7721471Z <td><pre>relation "dandi_cache_table" does not exist 2024-03-21T22:04:26.7722427Z LINE 1: SELECT "cache_key", "value", "expires" FROM "dandi_cache_tab... 2024-03-21T22:04:26.7723284Z ^ 2024-03-21T22:04:26.7723782Z </pre></td> 2024-03-21T22:04:26.7724069Z </tr> 2024-03-21T22:04:26.7724274Z 2024-03-21T22:04:26.7724288Z 2024-03-21T22:04:26.7724412Z <tr> 2024-03-21T22:04:26.7724739Z <th>Exception Location:</th> 2024-03-21T22:04:26.7725835Z <td><span class="fname">/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py</span>, line 89, in _execute</td> 2024-03-21T22:04:26.7726872Z </tr> 2024-03-21T22:04:26.7727049Z 2024-03-21T22:04:26.7727059Z 2024-03-21T22:04:26.7727204Z <tr> 2024-03-21T22:04:26.7727511Z <th>Raised during:</th> 2024-03-21T22:04:26.7728114Z <td>dandiapi.api.views.asset.NestedAssetViewSet</td> 2024-03-21T22:04:26.7728701Z </tr> 2024-03-21T22:04:26.7728890Z 2024-03-21T22:04:26.7729039Z <tr> 2024-03-21T22:04:26.7729371Z <th>Python Executable:</th> 2024-03-21T22:04:26.7729854Z <td>/usr/local/bin/python</td> 2024-03-21T22:04:26.7730312Z </tr> 2024-03-21T22:04:26.7730602Z <tr> 2024-03-21T22:04:26.7730917Z <th>Python Version:</th> 2024-03-21T22:04:26.7731355Z <td>3.11.8</td> 2024-03-21T22:04:26.7731733Z </tr> 2024-03-21T22:04:26.7732035Z <tr> 2024-03-21T22:04:26.7732404Z <th>Python Path:</th> 2024-03-21T22:04:26.7732869Z <td><pre>['/opt/django', 2024-03-21T22:04:26.7733377Z '/opt/django', 2024-03-21T22:04:26.7733778Z '/usr/local/lib/python311.zip', 2024-03-21T22:04:26.7734272Z '/usr/local/lib/python3.11', 2024-03-21T22:04:26.7734970Z '/usr/local/lib/python3.11/lib-dynload', 2024-03-21T22:04:26.7735656Z '/usr/local/lib/python3.11/site-packages', 2024-03-21T22:04:26.7736600Z '__editable__.dandiapi-0.0.post1+g0f9bd08.finder.__path_hook__']</pre></td> 2024-03-21T22:04:26.7737227Z </tr> 2024-03-21T22:04:26.7737501Z <tr> 2024-03-21T22:04:26.7737810Z <th>Server time:</th> 2024-03-21T22:04:26.7738257Z <td>Thu, 21 Mar 2024 20:07:16 +0000</td> 2024-03-21T22:04:26.7738754Z </tr> 2024-03-21T22:04:26.7739075Z </table> 2024-03-21T22:04:26.7739374Z </div> 2024-03-21T22:04:26.7739574Z 2024-03-21T22:04:26.7739581Z 2024-03-21T22:04:26.7739596Z 2024-03-21T22:04:26.7739601Z 2024-03-21T22:04:26.7739750Z <div id="traceback"> 2024-03-21T22:04:26.7740518Z <h2>Traceback <span class="commands"><a href="#" onclick="return switchPastebinFriendly(this);"> 2024-03-21T22:04:26.7741525Z Switch to copy-and-paste view</a></span> 2024-03-21T22:04:26.7742026Z </h2> 2024-03-21T22:04:26.7742365Z <div id="browserTraceback"> 2024-03-21T22:04:26.7742823Z <ul class="traceback"> 2024-03-21T22:04:26.7743233Z 2024-03-21T22:04:26.7743507Z 2024-03-21T22:04:26.7743855Z <li class="frame django"> 2024-03-21T22:04:26.7744309Z 2024-03-21T22:04:26.7745276Z <code class="fname">/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py</code>, line 89, in _execute 2024-03-21T22:04:26.7745923Z 2024-03-21T22:04:26.7746049Z 2024-03-21T22:04:26.7746129Z 2024-03-21T22:04:26.7746387Z <div class="context" id="c140470718659712"> 2024-03-21T22:04:26.7746902Z 2024-03-21T22:04:26.7747299Z <ol start="82" class="pre-context" id="pre140470718659712"> 2024-03-21T22:04:26.7747813Z 2024-03-21T22:04:26.7748745Z <li onclick="toggle('pre140470718659712', 'post140470718659712')"><pre> def _execute(self, sql, params, *ignored_wrapper_args):</pre></li> 2024-03-21T22:04:26.7749421Z 2024-03-21T22:04:26.7750085Z <li onclick="toggle('pre140470718659712', 'post140470718659712')"><pre> self.db.validate_no_broken_transaction()</pre></li> 2024-03-21T22:04:26.7750712Z 2024-03-21T22:04:26.7751345Z <li onclick="toggle('pre140470718659712', 'post140470718659712')"><pre> with self.db.wrap_database_errors:</pre></li> 2024-03-21T22:04:26.7751941Z 2024-03-21T22:04:26.7752528Z <li onclick="toggle('pre140470718659712', 'post140470718659712')"><pre> if params is None:</pre></li> 2024-03-21T22:04:26.7753102Z 2024-03-21T22:04:26.7753776Z <li onclick="toggle('pre140470718659712', 'post140470718659712')"><pre> # params default might be backend specific.</pre></li> 2024-03-21T22:04:26.7754419Z 2024-03-21T22:04:26.7755377Z <li onclick="toggle('pre140470718659712', 'post140470718659712')"><pre> return self.cursor.execute(sql)</pre></li> 2024-03-21T22:04:26.7756256Z 2024-03-21T22:04:26.7757122Z <li onclick="toggle('pre140470718659712', 'post140470718659712')"><pre> else:</pre></li> 2024-03-21T22:04:26.7757863Z 2024-03-21T22:04:26.7758136Z </ol> 2024-03-21T22:04:26.7758453Z 2024-03-21T22:04:26.7758921Z <ol start="89" class="context-line"> 2024-03-21T22:04:26.7760286Z <li onclick="toggle('pre140470718659712', 'post140470718659712')"><pre> return self.cursor.execute(sql, params)</pre> <span>…</span></li> 2024-03-21T22:04:26.7761388Z </ol> 2024-03-21T22:04:26.7761738Z 2024-03-21T22:04:26.7762390Z <ol start='90' class="post-context" id="post140470718659712"> 2024-03-21T22:04:26.7763034Z 2024-03-21T22:04:26.7763819Z <li onclick="toggle('pre140470718659712', 'post140470718659712')"><pre></pre></li> 2024-03-21T22:04:26.7764623Z 2024-03-21T22:04:26.7765817Z <li onclick="toggle('pre140470718659712', 'post140470718659712')"><pre> def _executemany(self, sql, param_list, *ignored_wrapper_args):</pre></li> 2024-03-21T22:04:26.7766844Z 2024-03-21T22:04:26.7767788Z <li onclick="toggle('pre140470718659712', 'post140470718659712')"><pre> self.db.validate_no_broken_transaction()</pre></li> 2024-03-21T22:04:26.7768737Z 2024-03-21T22:04:26.7769641Z <li onclick="toggle('pre140470718659712', 'post140470718659712')"><pre> with self.db.wrap_database_errors:</pre></li> 2024-03-21T22:04:26.7770501Z 2024-03-21T22:04:26.7771617Z <li onclick="toggle('pre140470718659712', 'post140470718659712')"><pre> return self.cursor.executemany(sql, param_list)</pre></li> 2024-03-21T22:04:26.7772651Z 2024-03-21T22:04:26.7773446Z <li onclick="toggle('pre140470718659712', 'post140470718659712')"><pre></pre></li> 2024-03-21T22:04:26.7774174Z 2024-03-21T22:04:26.7774530Z </ol> 2024-03-21T22:04:26.7774850Z 2024-03-21T22:04:26.7775193Z </div> 2024-03-21T22:04:26.7775531Z 2024-03-21T22:04:26.7775734Z 2024-03-21T22:04:26.7775857Z 2024-03-21T22:04:26.7776172Z 2024-03-21T22:04:26.7776484Z <details> 2024-03-21T22:04:26.7776998Z <summary class="commands">Local vars</summary> 2024-03-21T22:04:26.7777578Z 2024-03-21T22:04:26.7778203Z <table class="vars" id="v140470718659712"> 2024-03-21T22:04:26.7778705Z <thead> 2024-03-21T22:04:26.7779299Z <tr> 2024-03-21T22:04:26.7779693Z <th>Variable</th> 2024-03-21T22:04:26.7780166Z <th>Value</th> 2024-03-21T22:04:26.7780543Z </tr> 2024-03-21T22:04:26.7780899Z </thead> 2024-03-21T22:04:26.7781239Z <tbody> 2024-03-21T22:04:26.7781516Z 2024-03-21T22:04:26.7781813Z <tr> 2024-03-21T22:04:26.7782249Z <td>ignored_wrapper_args</td> 2024-03-21T22:04:26.7782800Z <td class="code"><pre>(False, 2024-03-21T22:04:26.7783760Z {'connection': <DatabaseWrapper vendor='postgresql' alias='default'>, 2024-03-21T22:04:26.7785451Z 'cursor': <debug_toolbar.panels.sql.tracking.patch_cursor_wrapper_with_mixin.<locals>.DjDTCursorWrapper object at 0x7fc1e3497cd0>})</pre></td> 2024-03-21T22:04:26.7786665Z </tr> 2024-03-21T22:04:26.7787068Z 2024-03-21T22:04:26.7787413Z <tr> 2024-03-21T22:04:26.7787788Z <td>params</td> 2024-03-21T22:04:26.7788644Z <td class="code"><pre>[':1:throttle_anon_172.18.0.1']</pre></td> 2024-03-21T22:04:26.7789412Z </tr> 2024-03-21T22:04:26.7789735Z 2024-03-21T22:04:26.7790045Z <tr> 2024-03-21T22:04:26.7790375Z <td>self</td> 2024-03-21T22:04:26.7791554Z <td class="code"><pre><debug_toolbar.panels.sql.tracking.patch_cursor_wrapper_with_mixin.<locals>.DjDTCursorWrapper object at 0x7fc1e3497cd0></pre></td> 2024-03-21T22:04:26.7792737Z </tr> 2024-03-21T22:04:26.7793063Z 2024-03-21T22:04:26.7793363Z <tr> 2024-03-21T22:04:26.7793678Z <td>sql</td> 2024-03-21T22:04:26.7794708Z <td class="code"><pre>('SELECT "cache_key", "value", "expires" FROM "dandi_cache_table" WHERE ' 2024-03-21T22:04:26.7795903Z '"cache_key" IN (%s)')</pre></td> 2024-03-21T22:04:26.7796471Z </tr> 2024-03-21T22:04:26.7796841Z 2024-03-21T22:04:26.7797197Z </tbody> 2024-03-21T22:04:26.7797561Z </table> 2024-03-21T22:04:26.7797921Z </details> 2024-03-21T22:04:26.7798308Z 2024-03-21T22:04:26.7798601Z </li> 2024-03-21T22:04:26.7798936Z 2024-03-21T22:04:26.7799251Z 2024-03-21T22:04:26.7799580Z <li class="cause"><h3> 2024-03-21T22:04:26.7800011Z 2024-03-21T22:04:26.7800606Z The above exception (relation "dandi_cache_table" does not exist 2024-03-21T22:04:26.7801692Z LINE 1: SELECT "cache_key", "value", "expires" FROM "dandi_cache_tab... 2024-03-21T22:04:26.7802551Z ^ 2024-03-21T22:04:26.7803109Z ) was the direct cause of the following exception: 2024-03-21T22:04:26.7803700Z 2024-03-21T22:04:26.7804001Z </h3></li> 2024-03-21T22:04:26.7804305Z 2024-03-21T22:04:26.7804608Z <li class="frame django"> 2024-03-21T22:04:26.7804963Z 2024-03-21T22:04:26.7806014Z <code class="fname">/usr/local/lib/python3.11/site-packages/django/core/handlers/exception.py</code>, line 56, in inner 2024-03-21T22:04:26.7807053Z 2024-03-21T22:04:26.7807238Z 2024-03-21T22:04:26.7807379Z 2024-03-21T22:04:26.7807800Z <div class="context" id="c140470718665088"> 2024-03-21T22:04:26.7808332Z 2024-03-21T22:04:26.7808956Z <ol start="49" class="pre-context" id="pre140470718665088"> 2024-03-21T22:04:26.7809551Z 2024-03-21T22:04:26.7810621Z <li onclick="toggle('pre140470718665088', 'post140470718665088')"><pre></pre></li> 2024-03-21T22:04:26.7811423Z 2024-03-21T22:04:26.7812371Z <li onclick="toggle('pre140470718665088', 'post140470718665088')"><pre> return inner</pre></li> 2024-03-21T22:04:26.7813436Z 2024-03-21T22:04:26.7814206Z <li onclick="toggle('pre140470718665088', 'post140470718665088')"><pre> else:</pre></li> 2024-03-21T22:04:26.7814912Z 2024-03-21T22:04:26.7815640Z <li onclick="toggle('pre140470718665088', 'post140470718665088')"><pre></pre></li> 2024-03-21T22:04:26.7816324Z 2024-03-21T22:04:26.7817119Z <li onclick="toggle('pre140470718665088', 'post140470718665088')"><pre> @wraps(get_response)</pre></li> 2024-03-21T22:04:26.7818003Z 2024-03-21T22:04:26.7818959Z <li onclick="toggle('pre140470718665088', 'post140470718665088')"><pre> def inner(request):</pre></li> 2024-03-21T22:04:26.7819883Z 2024-03-21T22:04:26.7820697Z <li onclick="toggle('pre140470718665088', 'post140470718665088')"><pre> try:</pre></li> 2024-03-21T22:04:26.7821535Z 2024-03-21T22:04:26.7821907Z </ol> 2024-03-21T22:04:26.7822218Z 2024-03-21T22:04:26.7822723Z <ol start="56" class="context-line"> 2024-03-21T22:04:26.7824089Z <li onclick="toggle('pre140470718665088', 'post140470718665088')"><pre> response = get_response(request)</pre> <span>…</span></li> 2024-03-21T22:04:26.7825052Z </ol> 2024-03-21T22:04:26.7825396Z 2024-03-21T22:04:26.7825984Z <ol start='57' class="post-context" id="post140470718665088"> 2024-03-21T22:04:26.7826551Z 2024-03-21T22:04:26.7827469Z <li onclick="toggle('pre140470718665088', 'post140470718665088')"><pre> except Exception as exc:</pre></li> 2024-03-21T22:04:26.7828333Z 2024-03-21T22:04:26.7829328Z <li onclick="toggle('pre140470718665088', 'post140470718665088')"><pre> response = response_for_exception(request, exc)</pre></li> 2024-03-21T22:04:26.7830217Z 2024-03-21T22:04:26.7831003Z <li onclick="toggle('pre140470718665088', 'post140470718665088')"><pre> return response</pre></li> 2024-03-21T22:04:26.7831758Z 2024-03-21T22:04:26.7832394Z <li onclick="toggle('pre140470718665088', 'post140470718665088')"><pre></pre></li> 2024-03-21T22:04:26.7832992Z 2024-03-21T22:04:26.7833760Z <li onclick="toggle('pre140470718665088', 'post140470718665088')"><pre> return inner</pre></li> 2024-03-21T22:04:26.7834479Z 2024-03-21T22:04:26.7835101Z <li onclick="toggle('pre140470718665088', 'post140470718665088')"><pre></pre></li> 2024-03-21T22:04:26.7835698Z 2024-03-21T22:04:26.7836000Z </ol> 2024-03-21T22:04:26.7836281Z 2024-03-21T22:04:26.7836554Z </div> 2024-03-21T22:04:26.7836826Z 2024-03-21T22:04:26.7836955Z 2024-03-21T22:04:26.7837072Z 2024-03-21T22:04:26.7837294Z 2024-03-21T22:04:26.7837569Z <details> 2024-03-21T22:04:26.7837987Z <summary class="commands">Local vars</summary> 2024-03-21T22:04:26.7838437Z 2024-03-21T22:04:26.7838778Z <table class="vars" id="v140470718665088"> 2024-03-21T22:04:26.7839206Z <thead> 2024-03-21T22:04:26.7839523Z <tr> 2024-03-21T22:04:26.7839811Z <th>Variable</th> 2024-03-21T22:04:26.7840168Z <th>Value</th> 2024-03-21T22:04:26.7840527Z </tr> 2024-03-21T22:04:26.7840828Z </thead> 2024-03-21T22:04:26.7841113Z <tbody> 2024-03-21T22:04:26.7841434Z 2024-03-21T22:04:26.7841952Z <tr> 2024-03-21T22:04:26.7842285Z <td>exc</td> 2024-03-21T22:04:26.7843824Z <td class="code"><pre>ProgrammingError('relation "dandi_cache_table" does not exist\nLINE 1: SELECT "cache_key", "value", "expires" FROM "dandi_cache_tab...\n ^\n')</pre></td> 2024-03-21T22:04:26.7845578Z </tr> 2024-03-21T22:04:26.7845896Z 2024-03-21T22:04:26.7846167Z <tr> 2024-03-21T22:04:26.7846549Z <td>get_response</td> 2024-03-21T22:04:26.7847643Z <td class="code"><pre><bound method BaseHandler._get_response of <django.core.handlers.wsgi.WSGIHandler object at 0x7fc1ee1488d0>></pre></td> 2024-03-21T22:04:26.7848625Z </tr> 2024-03-21T22:04:26.7848925Z 2024-03-21T22:04:26.7849209Z <tr> 2024-03-21T22:04:26.7849529Z <td>request</td> 2024-03-21T22:04:26.7850312Z <td class="code"><pre><WSGIRequest: GET '/api/dandisets/000008/versions/draft/assets/'></pre></td> 2024-03-21T22:04:26.7851110Z </tr> 2024-03-21T22:04:26.7851400Z 2024-03-21T22:04:26.7851636Z </tbody> 2024-03-21T22:04:26.7851933Z </table> 2024-03-21T22:04:26.7852226Z </details> 2024-03-21T22:04:26.7852509Z 2024-03-21T22:04:26.7852780Z </li> 2024-03-21T22:04:26.7853076Z 2024-03-21T22:04:26.7853329Z 2024-03-21T22:04:26.7853611Z <li class="frame django"> 2024-03-21T22:04:26.7853992Z 2024-03-21T22:04:26.7854959Z <code class="fname">/usr/local/lib/python3.11/site-packages/django/core/handlers/base.py</code>, line 197, in _get_response 2024-03-21T22:04:26.7855886Z 2024-03-21T22:04:26.7856059Z 2024-03-21T22:04:26.7856148Z 2024-03-21T22:04:26.7856513Z <div class="context" id="c140470718664960"> 2024-03-21T22:04:26.7856969Z 2024-03-21T22:04:26.7857478Z <ol start="190" class="pre-context" id="pre140470718664960"> 2024-03-21T22:04:26.7857987Z 2024-03-21T22:04:26.7858636Z <li onclick="toggle('pre140470718664960', 'post140470718664960')"><pre></pre></li> 2024-03-21T22:04:26.7859315Z 2024-03-21T22:04:26.7860261Z <li onclick="toggle('pre140470718664960', 'post140470718664960')"><pre> if response is None:</pre></li> 2024-03-21T22:04:26.7861220Z 2024-03-21T22:04:26.7862413Z <li onclick="toggle('pre140470718664960', 'post140470718664960')"><pre> wrapped_callback = self.make_view_atomic(callback)</pre></li> 2024-03-21T22:04:26.7863682Z 2024-03-21T22:04:26.7865045Z <li onclick="toggle('pre140470718664960', 'post140470718664960')"><pre> # If it is an asynchronous view, run it in a subthread.</pre></li> 2024-03-21T22:04:26.7866304Z 2024-03-21T22:04:26.7867642Z <li onclick="toggle('pre140470718664960', 'post140470718664960')"><pre> if asyncio.iscoroutinefunction(wrapped_callback):</pre></li> 2024-03-21T22:04:26.7869073Z 2024-03-21T22:04:26.7870417Z <li onclick="toggle('pre140470718664960', 'post140470718664960')"><pre> wrapped_callback = async_to_sync(wrapped_callback)</pre></li> 2024-03-21T22:04:26.7871642Z 2024-03-21T22:04:26.7872604Z <li onclick="toggle('pre140470718664960', 'post140470718664960')"><pre> try:</pre></li> 2024-03-21T22:04:26.7873496Z 2024-03-21T22:04:26.7873876Z </ol> 2024-03-21T22:04:26.7874258Z 2024-03-21T22:04:26.7874742Z <ol start="197" class="context-line"> 2024-03-21T22:04:26.7876777Z <li onclick="toggle('pre140470718664960', 'post140470718664960')"><pre> response = wrapped_callback(request, *callback_args, **callback_kwargs)</pre> <span>…</span></li> 2024-03-21T22:04:26.7878341Z </ol> 2024-03-21T22:04:26.7878687Z 2024-03-21T22:04:26.7879331Z <ol start='198' class="post-context" id="post140470718664960"> 2024-03-21T22:04:26.7880002Z 2024-03-21T22:04:26.7881040Z <li onclick="toggle('pre140470718664960', 'post140470718664960')"><pre> except Exception as e:</pre></li> 2024-03-21T22:04:26.7882041Z 2024-03-21T22:04:26.7883336Z <li onclick="toggle('pre140470718664960', 'post140470718664960')"><pre> response = self.process_exception_by_middleware(e, request)</pre></li> 2024-03-21T22:04:26.7884542Z 2024-03-21T22:04:26.7885577Z <li onclick="toggle('pre140470718664960', 'post140470718664960')"><pre> if response is None:</pre></li> 2024-03-21T22:04:26.7886586Z 2024-03-21T22:04:26.7887561Z <li onclick="toggle('pre140470718664960', 'post140470718664960')"><pre> raise</pre></li> 2024-03-21T22:04:26.7888504Z 2024-03-21T22:04:26.7889295Z <li onclick="toggle('pre140470718664960', 'post140470718664960')"><pre></pre></li> 2024-03-21T22:04:26.7890086Z 2024-03-21T22:04:26.7891320Z <li onclick="toggle('pre140470718664960', 'post140470718664960')"><pre> # Complain if the view returned None (a common error).</pre></li> 2024-03-21T22:04:26.7892480Z 2024-03-21T22:04:26.7892829Z </ol> 2024-03-21T22:04:26.7893185Z 2024-03-21T22:04:26.7893509Z </div> 2024-03-21T22:04:26.7893845Z 2024-03-21T22:04:26.7894039Z 2024-03-21T22:04:26.7894166Z 2024-03-21T22:04:26.7894466Z 2024-03-21T22:04:26.7894804Z <details> 2024-03-21T22:04:26.7895318Z <summary class="commands">Local vars</summary> 2024-03-21T22:04:26.7895897Z 2024-03-21T22:04:26.7896318Z <table class="vars" id="v140470718664960"> 2024-03-21T22:04:26.7896871Z <thead> 2024-03-21T22:04:26.7897237Z <tr> 2024-03-21T22:04:26.7897628Z <th>Variable</th> 2024-03-21T22:04:26.7898116Z <th>Value</th> 2024-03-21T22:04:26.7898551Z </tr> 2024-03-21T22:04:26.7898923Z </thead> 2024-03-21T22:04:26.7899296Z <tbody> 2024-03-21T22:04:26.7899644Z 2024-03-21T22:04:26.7899985Z <tr> 2024-03-21T22:04:26.7900377Z <td>callback</td> 2024-03-21T22:04:26.7901236Z <td class="code"><pre><function NestedAssetViewSet at 0x7fc1e9d0af20></pre></td> 2024-03-21T22:04:26.7901953Z </tr> 2024-03-21T22:04:26.7902284Z 2024-03-21T22:04:26.7902515Z <tr> 2024-03-21T22:04:26.7902874Z <td>callback_args</td> 2024-03-21T22:04:26.7903310Z <td class="code"><pre>()</pre></td> 2024-03-21T22:04:26.7903765Z </tr> 2024-03-21T22:04:26.7904056Z 2024-03-21T22:04:26.7904333Z <tr> 2024-03-21T22:04:26.7904661Z <td>callback_kwargs</td> 2024-03-21T22:04:26.7905511Z <td class="code"><pre>{'versions__dandiset__pk': '000008', 'versions__version': 'draft'}</pre></td> 2024-03-21T22:04:26.7906321Z </tr> 2024-03-21T22:04:26.7906610Z 2024-03-21T22:04:26.7906887Z <tr> 2024-03-21T22:04:26.7907219Z <td>middleware_method</td> 2024-03-21T22:04:26.7908722Z <td class="code"><pre><bound method CsrfViewMiddleware.process_view of <CsrfViewMiddleware get_response=convert_exception_to_response.<locals>.inner>></pre></td> 2024-03-21T22:04:26.7910158Z </tr> 2024-03-21T22:04:26.7910496Z 2024-03-21T22:04:26.7911073Z <tr> 2024-03-21T22:04:26.7911461Z <td>request</td> 2024-03-21T22:04:26.7912616Z <td class="code"><pre><WSGIRequest: GET '/api/dandisets/000008/versions/draft/assets/'></pre></td> 2024-03-21T22:04:26.7913440Z </tr> 2024-03-21T22:04:26.7913810Z 2024-03-21T22:04:26.7914151Z <tr> 2024-03-21T22:04:26.7914548Z <td>response</td> 2024-03-21T22:04:26.7915022Z <td class="code"><pre>None</pre></td> 2024-03-21T22:04:26.7915535Z </tr> 2024-03-21T22:04:26.7915843Z 2024-03-21T22:04:26.7916137Z <tr> 2024-03-21T22:04:26.7916485Z <td>self</td> 2024-03-21T22:04:26.7917433Z <td class="code"><pre><django.core.handlers.wsgi.WSGIHandler object at 0x7fc1ee1488d0></pre></td> 2024-03-21T22:04:26.7918374Z </tr> 2024-03-21T22:04:26.7918743Z 2024-03-21T22:04:26.7919133Z <tr> 2024-03-21T22:04:26.7919572Z <td>wrapped_callback</td> 2024-03-21T22:04:26.7920385Z <td class="code"><pre><function NestedAssetViewSet at 0x7fc1e9d0af20></pre></td> 2024-03-21T22:04:26.7921187Z </tr> 2024-03-21T22:04:26.7921557Z 2024-03-21T22:04:26.7921890Z </tbody> 2024-03-21T22:04:26.7922286Z </table> 2024-03-21T22:04:26.7922656Z </details> 2024-03-21T22:04:26.7923012Z 2024-03-21T22:04:26.7923373Z </li> 2024-03-21T22:04:26.7923668Z 2024-03-21T22:04:26.7923923Z 2024-03-21T22:04:26.7924225Z <li class="frame django"> 2024-03-21T22:04:26.7924606Z 2024-03-21T22:04:26.7925667Z <code class="fname">/usr/local/lib/python3.11/site-packages/django/views/decorators/csrf.py</code>, line 55, in wrapped_view 2024-03-21T22:04:26.7926733Z 2024-03-21T22:04:26.7926889Z 2024-03-21T22:04:26.7927034Z 2024-03-21T22:04:26.7927367Z <div class="context" id="c140470806240640"> 2024-03-21T22:04:26.7927799Z 2024-03-21T22:04:26.7928420Z <ol start="48" class="pre-context" id="pre140470806240640"> 2024-03-21T22:04:26.7929018Z 2024-03-21T22:04:26.7929815Z <li onclick="toggle('pre140470806240640', 'post140470806240640')"><pre></pre></li> 2024-03-21T22:04:26.7930575Z 2024-03-21T22:04:26.7931585Z <li onclick="toggle('pre140470806240640', 'post140470806240640')"><pre>def csrf_exempt(view_func):</pre></li> 2024-03-21T22:04:26.7932488Z 2024-03-21T22:04:26.7933968Z <li onclick="toggle('pre140470806240640', 'post140470806240640')"><pre> """Mark a view function as being exempt from the CSRF view protection."""</pre></li> 2024-03-21T22:04:26.7935296Z 2024-03-21T22:04:26.7935982Z <li onclick="toggle('pre140470806240640', 'post140470806240640')"><pre></pre></li> 2024-03-21T22:04:26.7936679Z 2024-03-21T22:04:26.7937867Z <li onclick="toggle('pre140470806240640', 'post140470806240640')"><pre> # view_func.csrf_exempt = True would also work, but decorators are nicer</pre></li> 2024-03-21T22:04:26.7938891Z 2024-03-21T22:04:26.7939984Z <li onclick="toggle('pre140470806240640', 'post140470806240640')"><pre> # if they don't have side effects, so return a new function.</pre></li> 2024-03-21T22:04:26.7941109Z 2024-03-21T22:04:26.7942084Z <li onclick="toggle('pre140470806240640', 'post140470806240640')"><pre> def wrapped_view(*args, **kwargs):</pre></li> 2024-03-21T22:04:26.7943020Z 2024-03-21T22:04:26.7943372Z </ol> 2024-03-21T22:04:26.7943971Z 2024-03-21T22:04:26.7944515Z <ol start="55" class="context-line"> 2024-03-21T22:04:26.7946047Z <li onclick="toggle('pre140470806240640', 'post140470806240640')"><pre> return view_func(*args, **kwargs)</pre> <span>…</span></li> 2024-03-21T22:04:26.7947044Z </ol> 2024-03-21T22:04:26.7947352Z 2024-03-21T22:04:26.7947901Z <ol start='56' class="post-context" id="post140470806240640"> 2024-03-21T22:04:26.7948653Z 2024-03-21T22:04:26.7949353Z <li onclick="toggle('pre140470806240640', 'post140470806240640')"><pre></pre></li> 2024-03-21T22:04:26.7949937Z 2024-03-21T22:04:26.7950707Z <li onclick="toggle('pre140470806240640', 'post140470806240640')"><pre> wrapped_view.csrf_exempt = True</pre></li> 2024-03-21T22:04:26.7951456Z 2024-03-21T22:04:26.7952330Z <li onclick="toggle('pre140470806240640', 'post140470806240640')"><pre> return wraps(view_func)(wrapped_view)</pre></li> 2024-03-21T22:04:26.7953177Z 2024-03-21T22:04:26.7953467Z </ol> 2024-03-21T22:04:26.7953777Z 2024-03-21T22:04:26.7954057Z </div> 2024-03-21T22:04:26.7954329Z 2024-03-21T22:04:26.7954501Z 2024-03-21T22:04:26.7954589Z 2024-03-21T22:04:26.7954835Z 2024-03-21T22:04:26.7955135Z <details> 2024-03-21T22:04:26.7955552Z <summary class="commands">Local vars</summary> 2024-03-21T22:04:26.7956014Z 2024-03-21T22:04:26.7956392Z <table class="vars" id="v140470806240640"> 2024-03-21T22:04:26.7956856Z <thead> 2024-03-21T22:04:26.7957151Z <tr> 2024-03-21T22:04:26.7957508Z <th>Variable</th> 2024-03-21T22:04:26.7957927Z <th>Value</th> 2024-03-21T22:04:26.7958274Z </tr> 2024-03-21T22:04:26.7958587Z </thead> 2024-03-21T22:04:26.7958884Z <tbody> 2024-03-21T22:04:26.7959150Z 2024-03-21T22:04:26.7959437Z <tr> 2024-03-21T22:04:26.7959748Z <td>args</td> 2024-03-21T22:04:26.7960512Z <td class="code"><pre>(<WSGIRequest: GET '/api/dandisets/000008/versions/draft/assets/'>,)</pre></td> 2024-03-21T22:04:26.7961327Z </tr> 2024-03-21T22:04:26.7961647Z 2024-03-21T22:04:26.7961900Z <tr> 2024-03-21T22:04:26.7962176Z <td>kwargs</td> 2024-03-21T22:04:26.7963009Z <td class="code"><pre>{'versions__dandiset__pk': '000008', 'versions__version': 'draft'}</pre></td> 2024-03-21T22:04:26.7963913Z </tr> 2024-03-21T22:04:26.7964240Z 2024-03-21T22:04:26.7964532Z <tr> 2024-03-21T22:04:26.7964850Z <td>view_func</td> 2024-03-21T22:04:26.7965536Z <td class="code"><pre><function NestedAssetViewSet at 0x7fc1e9d0ae80></pre></td> 2024-03-21T22:04:26.7966219Z </tr> 2024-03-21T22:04:26.7966557Z 2024-03-21T22:04:26.7966824Z </tbody> 2024-03-21T22:04:26.7967136Z </table> 2024-03-21T22:04:26.7967461Z </details> 2024-03-21T22:04:26.7967739Z 2024-03-21T22:04:26.7967993Z </li> 2024-03-21T22:04:26.7968280Z 2024-03-21T22:04:26.7968533Z 2024-03-21T22:04:26.7968834Z <li class="frame user"> 2024-03-21T22:04:26.7969219Z 2024-03-21T22:04:26.7970050Z <code class="fname">/usr/local/lib/python3.11/site-packages/rest_framework/viewsets.py</code>, line 125, in view 2024-03-21T22:04:26.7970817Z 2024-03-21T22:04:26.7970982Z 2024-03-21T22:04:26.7971073Z 2024-03-21T22:04:26.7971384Z <div class="context" id="c140470718556864"> 2024-03-21T22:04:26.7971843Z 2024-03-21T22:04:26.7972568Z <ol start="118" class="pre-context" id="pre140470718556864"> 2024-03-21T22:04:26.7973276Z 2024-03-21T22:04:26.7974115Z <li onclick="toggle('pre140470718556864', 'post140470718556864')"><pre> setattr(self, method, handler)</pre></li> 2024-03-21T22:04:26.7974896Z 2024-03-21T22:04:26.7975537Z <li onclick="toggle('pre140470718556864', 'post140470718556864')"><pre></pre></li> 2024-03-21T22:04:26.7976110Z 2024-03-21T22:04:26.7976945Z <li onclick="toggle('pre140470718556864', 'post140470718556864')"><pre> self.request = request</pre></li> 2024-03-21T22:04:26.7977705Z 2024-03-21T22:04:26.7978507Z <li onclick="toggle('pre140470718556864', 'post140470718556864')"><pre> self.args = args</pre></li> 2024-03-21T22:04:26.7979241Z 2024-03-21T22:04:26.7980062Z <li onclick="toggle('pre140470718556864', 'post140470718556864')"><pre> self.kwargs = kwargs</pre></li> 2024-03-21T22:04:26.7980810Z 2024-03-21T22:04:26.7981436Z <li onclick="toggle('pre140470718556864', 'post140470718556864')"><pre></pre></li> 2024-03-21T22:04:26.7982015Z 2024-03-21T22:04:26.7982797Z <li onclick="toggle('pre140470718556864', 'post140470718556864')"><pre> # And continue as usual</pre></li> 2024-03-21T22:04:26.7983515Z 2024-03-21T22:04:26.7983818Z </ol> 2024-03-21T22:04:26.7984112Z 2024-03-21T22:04:26.7984470Z <ol start="125" class="context-line"> 2024-03-21T22:04:26.7985585Z <li onclick="toggle('pre140470718556864', 'post140470718556864')"><pre> return self.dispatch(request, *args, **kwargs)</pre> <span>…</span></li> 2024-03-21T22:04:26.7986488Z </ol> 2024-03-21T22:04:26.7986792Z 2024-03-21T22:04:26.7987301Z <ol start='126' class="post-context" id="post140470718556864"> 2024-03-21T22:04:26.7987855Z 2024-03-21T22:04:26.7988662Z <li onclick="toggle('pre140470718556864', 'post140470718556864')"><pre></pre></li> 2024-03-21T22:04:26.7989451Z 2024-03-21T22:04:26.7990539Z <li onclick="toggle('pre140470718556864', 'post140470718556864')"><pre> # take name and docstring from class</pre></li> 2024-03-21T22:04:26.7991486Z 2024-03-21T22:04:26.7992578Z <li onclick="toggle('pre140470718556864', 'post140470718556864')"><pre> update_wrapper(view, cls, updated=())</pre></li> 2024-03-21T22:04:26.7993538Z 2024-03-21T22:04:26.7994220Z <li onclick="toggle('pre140470718556864', 'post140470718556864')"><pre></pre></li> 2024-03-21T22:04:26.7994913Z 2024-03-21T22:04:26.7995912Z <li onclick="toggle('pre140470718556864', 'post140470718556864')"><pre> # and possible attributes set by decorators</pre></li> 2024-03-21T22:04:26.7996817Z 2024-03-21T22:04:26.7997714Z <li onclick="toggle('pre140470718556864', 'post140470718556864')"><pre> # like csrf_exempt from dispatch</pre></li> 2024-03-21T22:04:26.7998650Z 2024-03-21T22:04:26.7999007Z </ol> 2024-03-21T22:04:26.7999341Z 2024-03-21T22:04:26.7999688Z </div> 2024-03-21T22:04:26.8000004Z 2024-03-21T22:04:26.8000209Z 2024-03-21T22:04:26.8000326Z 2024-03-21T22:04:26.8000637Z 2024-03-21T22:04:26.8000929Z <details> 2024-03-21T22:04:26.8001460Z <summary class="commands">Local vars</summary> 2024-03-21T22:04:26.8002017Z 2024-03-21T22:04:26.8002429Z <table class="vars" id="v140470718556864"> 2024-03-21T22:04:26.8002943Z <thead> 2024-03-21T22:04:26.8003331Z <tr> 2024-03-21T22:04:26.8003985Z <th>Variable</th> 2024-03-21T22:04:26.8004500Z <th>Value</th> 2024-03-21T22:04:26.8005172Z </tr> 2024-03-21T22:04:26.8005490Z </thead> 2024-03-21T22:04:26.8005810Z <tbody> 2024-03-21T22:04:26.8006122Z 2024-03-21T22:04:26.8006428Z <tr> 2024-03-21T22:04:26.8006815Z <td>action</td> 2024-03-21T22:04:26.8007357Z <td class="code"><pre>'list'</pre></td> 2024-03-21T22:04:26.8007891Z </tr> 2024-03-21T22:04:26.8008217Z 2024-03-21T22:04:26.8008519Z <tr> 2024-03-21T22:04:26.8008821Z <td>actions</td> 2024-03-21T22:04:26.8009744Z <td class="code"><pre>{'get': 'list', 'head': 'list', 'post': 'create'}</pre></td> 2024-03-21T22:04:26.8010754Z </tr> 2024-03-21T22:04:26.8011129Z 2024-03-21T22:04:26.8011461Z <tr> 2024-03-21T22:04:26.8011863Z <td>args</td> 2024-03-21T22:04:26.8012353Z <td class="code"><pre>()</pre></td> 2024-03-21T22:04:26.8012905Z </tr> 2024-03-21T22:04:26.8013257Z 2024-03-21T22:04:26.8013614Z <tr> 2024-03-21T22:04:26.8013977Z <td>cls</td> 2024-03-21T22:04:26.8014890Z <td class="code"><pre><class 'dandiapi.api.views.asset.NestedAssetViewSet'></pre></td> 2024-03-21T22:04:26.8015837Z </tr> 2024-03-21T22:04:26.8016186Z 2024-03-21T22:04:26.8016504Z <tr> 2024-03-21T22:04:26.8016874Z <td>handler</td> 2024-03-21T22:04:26.8018012Z <td class="code"><pre><bound method NestedAssetViewSet.list of <dandiapi.api.views.asset.NestedAssetViewSet object at 0x7fc1e3497410>></pre></td> 2024-03-21T22:04:26.8019215Z </tr> 2024-03-21T22:04:26.8019570Z 2024-03-21T22:04:26.8019882Z <tr> 2024-03-21T22:04:26.8020216Z <td>initkwargs</td> 2024-03-21T22:04:26.8021161Z <td class="code"><pre>{'basename': 'asset', 'detail': False, 'suffix': 'List'}</pre></td> 2024-03-21T22:04:26.8022159Z </tr> 2024-03-21T22:04:26.8022510Z 2024-03-21T22:04:26.8022846Z <tr> 2024-03-21T22:04:26.8023285Z <td>kwargs</td> 2024-03-21T22:04:26.8024269Z <td class="code"><pre>{'versions__dandiset__pk': '000008', 'versions__version': 'draft'}</pre></td> 2024-03-21T22:04:26.8025304Z </tr> 2024-03-21T22:04:26.8025670Z 2024-03-21T22:04:26.8026016Z <tr> 2024-03-21T22:04:26.8026399Z <td>method</td> 2024-03-21T22:04:26.8026986Z <td class="code"><pre>'head'</pre></td> 2024-03-21T22:04:26.8027603Z </tr> 2024-03-21T22:04:26.8027959Z 2024-03-21T22:04:26.8028284Z <tr> 2024-03-21T22:04:26.8028737Z <td>request</td> 2024-03-21T22:04:26.8029522Z <td class="code"><pre><WSGIRequest: GET '/api/dandisets/000008/versions/draft/assets/'></pre></td> 2024-03-21T22:04:26.8030314Z </tr> 2024-03-21T22:04:26.8030604Z 2024-03-21T22:04:26.8030870Z <tr> 2024-03-21T22:04:26.8031166Z <td>self</td> 2024-03-21T22:04:26.8031873Z <td class="code"><pre><dandiapi.api.views.asset.NestedAssetViewSet object at 0x7fc1e3497410></pre></td> 2024-03-21T22:04:26.8032652Z </tr> 2024-03-21T22:04:26.8032943Z 2024-03-21T22:04:26.8033265Z </tbody> 2024-03-21T22:04:26.8033791Z </table> 2024-03-21T22:04:26.8034148Z </details> 2024-03-21T22:04:26.8034453Z 2024-03-21T22:04:26.8034925Z </li> 2024-03-21T22:04:26.8035195Z 2024-03-21T22:04:26.8035456Z 2024-03-21T22:04:26.8035764Z <li class="frame user"> 2024-03-21T22:04:26.8036106Z 2024-03-21T22:04:26.8036990Z <code class="fname">/usr/local/lib/python3.11/site-packages/rest_framework/views.py</code>, line 509, in dispatch 2024-03-21T22:04:26.8037831Z 2024-03-21T22:04:26.8037994Z 2024-03-21T22:04:26.8038088Z

This is the rendered HTML that you posted in that comment

The reason for this is that the CLI tests are improperly configured. They need to run ./manage.py createcachetable before running the tests, as documented in the development guide. I believe @mvandenburgh is making a PR to the CLI codebase now.

As for why we're just seeing this now with the CLI tests, it's likely due to the fact that rate limiting uses that cache table to determine who and when to rate limit. The CLI tests must not have hit any endpoint that makes uses of that cache table until now.

@jjnesbitt @yarikoptic just to confirm this - I opened a PR to the CLI making that fix, and repointed the CLI tests at that branch, and the tests pass now- https://github.com/dandi/dandi-archive/actions/runs/8392624314/job/22985832345?pr=1899

dandiapi/api/views/asset.py

yarikoptic · 2024-03-22T14:03:17Z

PR was approved, but I think it is "not ready".
IMHO this should not be deployed until dandi-cli is capable of handling it since it

seems to make detrimental effect on CI: were dandi-cli reported fails analyzed? I mentioned above that some fails seems to be actually 500s - errors on the server likely due to introduced here changes!
might very negatively affect users in typical legit usage patterns: anyone checked explicitly effects on dandi-cli? again looking at dandi-cli CI fails could be of help.

dandibot · 2024-03-25T17:50:04Z

🚀 PR was released in v0.3.81 🚀

mvandenburgh requested review from waxlamp and jjnesbitt March 20, 2024 20:02

mvandenburgh added patch Increment the patch version when merged release Create a release when this pr is merged labels Mar 20, 2024

mvandenburgh force-pushed the rate-limit-asset-list branch from bd3d544 to b8f197c Compare March 20, 2024 20:40

jjnesbitt reviewed Mar 21, 2024

View reviewed changes

dandiapi/api/views/asset.py Outdated Show resolved Hide resolved

mvandenburgh force-pushed the rate-limit-asset-list branch 4 times, most recently from 856f938 to bbd21b4 Compare March 21, 2024 15:42

mvandenburgh requested a review from jjnesbitt March 21, 2024 15:42

jjnesbitt previously approved these changes Mar 21, 2024

View reviewed changes

jjnesbitt self-requested a review March 21, 2024 16:38

mvandenburgh force-pushed the rate-limit-asset-list branch from bbd21b4 to c5e9583 Compare March 21, 2024 18:03

mvandenburgh force-pushed the rate-limit-asset-list branch 4 times, most recently from 78114f8 to 9b64ad2 Compare March 21, 2024 19:40

jjnesbitt reviewed Mar 21, 2024

View reviewed changes

dandiapi/api/views/asset.py Outdated Show resolved Hide resolved

mvandenburgh force-pushed the rate-limit-asset-list branch from e15446e to 7051d15 Compare March 21, 2024 20:03

jjnesbitt approved these changes Mar 21, 2024

View reviewed changes

yarikoptic reviewed Mar 21, 2024

View reviewed changes

jjnesbitt reviewed Mar 22, 2024

View reviewed changes

dandiapi/api/views/asset.py Show resolved Hide resolved

yarikoptic mentioned this pull request Mar 22, 2024

Design and implement rate limiting policy #1902

Open

mvandenburgh mentioned this pull request Mar 22, 2024

Add missing command to dandi-archive docker compose fixture dandi/dandi-cli#1421

Merged

Rate limit assets list endpoint for logged out users

55cbc02

mvandenburgh force-pushed the rate-limit-asset-list branch from fd95fba to e653997 Compare March 22, 2024 16:04

mvandenburgh removed patch Increment the patch version when merged release Create a release when this pr is merged labels Mar 22, 2024

mvandenburgh force-pushed the rate-limit-asset-list branch from e653997 to 55cbc02 Compare March 22, 2024 16:51

mvandenburgh merged commit c3666bc into master Mar 22, 2024
11 checks passed

mvandenburgh deleted the rate-limit-asset-list branch March 22, 2024 17:42

This was referenced Mar 22, 2024

Add missing setting for DRF throttle class #1903

Merged

Revert rate limiting of asset list endpoint #1905

Merged

dandibot added the released This issue/pull request has been released. label Mar 25, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Rate limit assets list endpoint for unauthenticated users #1899

Rate limit assets list endpoint for unauthenticated users #1899

mvandenburgh commented Mar 20, 2024

jjnesbitt commented Mar 20, 2024

mvandenburgh commented Mar 20, 2024

yarikoptic commented Mar 21, 2024

yarikoptic commented Mar 21, 2024

yarikoptic Mar 21, 2024

jjnesbitt Mar 21, 2024

yarikoptic Mar 21, 2024

jjnesbitt Mar 22, 2024

yarikoptic Mar 22, 2024

jjnesbitt Mar 22, 2024 •

edited

Loading

mvandenburgh Mar 22, 2024

yarikoptic commented Mar 22, 2024

dandibot commented Mar 25, 2024

Rate limit assets list endpoint for unauthenticated users #1899

Rate limit assets list endpoint for unauthenticated users #1899

Conversation

mvandenburgh commented Mar 20, 2024

jjnesbitt commented Mar 20, 2024

mvandenburgh commented Mar 20, 2024

yarikoptic commented Mar 21, 2024

yarikoptic commented Mar 21, 2024

yarikoptic Mar 21, 2024

Choose a reason for hiding this comment

jjnesbitt Mar 21, 2024

Choose a reason for hiding this comment

yarikoptic Mar 21, 2024

Choose a reason for hiding this comment

jjnesbitt Mar 22, 2024

Choose a reason for hiding this comment

yarikoptic Mar 22, 2024

Choose a reason for hiding this comment

jjnesbitt Mar 22, 2024 • edited Loading

Choose a reason for hiding this comment

mvandenburgh Mar 22, 2024

Choose a reason for hiding this comment

yarikoptic commented Mar 22, 2024

dandibot commented Mar 25, 2024

jjnesbitt Mar 22, 2024 •

edited

Loading