Could not find starting boundary in request.post() and reader.next()

[The-CJ]

Long story short

I want to upload a image and one or multiple arguments to it.
Seems to work on some images, some have empty payload and some throwing "Could not find starting boundary" error

Expected behaviour

It reads the post content if possible and :

• throws a HTTPRequestEntityTooLarge if to large, or
• have a finished post body to work

same for the multipart() , reader.next() method

Actual behaviour

request.post() throws a "ValueError: Could not find starting boundary b'------stuff'"
request.multipart() -> reader.next() throws a "ValueError: Could not find starting boundary b'------stuff'"
requests with a body > 2,8MB image have a empty request.post() and return reader.next() = None should trow a HTTPRequestEntityTooLarge error

Seems to work find if no file is involved or file < 1MB
(no, i never changed the client_max_size, should be 1024*2)

Client (Chrome)

action: change_avatar
file: (binary)

Client Raw

------WebKitFormBoundaryRyPZoBqLT0tIpjV2
Content-Disposition: form-data; name="action"

change_avatar
------WebKitFormBoundaryRyPZoBqLT0tIpjV2
Content-Disposition: form-data; name="file"; filename="example_file.png"
Content-Type: image/png


------WebKitFormBoundaryRyPZoBqLT0tIpjV2--

Server

Error handling request
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/aiohttp/web_protocol.py", line 418, in start
    resp = await task
  File "/usr/local/lib/python3.6/dist-packages/aiohttp/web_app.py", line 458, in _handle
    resp = await handler(request)
  File "/root/Dev/Phaazebot_dev/_API_/Admin.py", line 105, in control
    r = await reader.next()
  File "/usr/local/lib/python3.6/dist-packages/aiohttp/multipart.py", line 561, in next
    await self._read_until_first_boundary()
  File "/usr/local/lib/python3.6/dist-packages/aiohttp/multipart.py", line 627, in _read_until_first_boundary
    % (self._boundary))
ValueError: Could not find starting boundary b'------WebKitFormBoundaryRyPZoBqLT0tIpjV2'

Steps to reproduce

Client

function upload_avatar() {
  var f = $('#new_avatar');
  var u = {
    "action": "change_avatar",
    "file": f[0].files[0],
  };
  var p = "/api/admin/control";
  function s(data) {
    _hide_loading();
    _show_message(data.msg, "green");
  }
  function fa(data) {
    _hide_loading();
    _show_message(data.msg ? data.msg : "Critical Error", "red");
  }
  _show_loading("Uploading...");
  upload_file(u, p, s, fa);
}

function upload_file(args, url, success_function, fail_function) {
  if (!(url && args)) {return false;}

  var formData = new FormData();
  for (var upl in args) {
    formData.append(upl, args[upl]);
  }
  var request = new XMLHttpRequest();
  request.onload = function () {
    if (200 <= request.status && request.status < 300) {
      success_function( JSON.parse(request.responseText) );
    } else if (request.status >= 500) {
      fail_function( Array() );
    } else {
      fail_function( JSON.parse(request.responseText) );
    }
  }
  request.onerror = function () {
    fail_function(JSON.parse(request.responseText));
  }
  request.open("POST", url);
  request.send(formData);
}

Server

from aiohttp import web
server = web.Application()
server.router.add_route('*',   '/api/admin/control', example_upload)

#example 1
async def example_upload(request):
        post = await request.post() # <- Critical Error here

       action = post["action"]
       file = post["file"]

       . . .

       return web.Response(
                 body=json.dumps(dict(msg="stuff", status=400)),
                 status=400,
                 content_type='application/json'
        )

#example 2
async def example_upload(request):
        reader = await request.multipart()
        r = await reader.next() # <-- Critical Error here#

       . . .

        return web.Response(
                 body=json.dumps(dict(msg="stuff", status=400)),
                 status=400,
                 content_type='application/json'
        )



web.run_app(server, handle_signals=False, ssl_context=get_ssl_stuff(), port=443, print=False)

Your environment

Ubuntu 18.04.1 LTS (GNU/Linux 4.15.0-42-generic x86_64)
VM Server
Python 3.6.7
aiohttp 3.5.4

server

[aio-libs-bot]

GitMate.io thinks the contributor most likely able to help you is @asvetlov.

Possibly related issues are #187 (Request.POST preloading), #161 (aiohttp can't send files by POST request correct), #1765 (request.post() ignores blank values), #1557 (POST request always follow redirects), and #114 (Exception with MultiDict in POST request).

[asvetlov]

Pull Request is welcome!

[The-CJ]

sorry if i ask, but of what...

[kxepal]

Hm.... @The-CJ , could you share via gist full raw HTTP request that triggers that error? Steps to reproduce are good, but I fear I couldn't reproduce issues by them.

[The-CJ]

Yeah Sure. Full raw request send by client:
Workes: https://gist.github.com/The-CJ/294cd55c67704b6c442daf857100e470
Don't Work: https://gist.github.com/The-CJ/144f8c48854010bf487d6cc1f9793aa0

[kxepal]

Awesome! Thank you! I quick looked on them and noticed that second one looks like truncated: there is no close boundary nor json Discord response. Is that dump complete? Because otherwise the error is correct.

[The-CJ]

the second one has no response, because the code of my server never reached the response part, it failed at await request.post()
both gist are full logs of a curl request,
(and yes the error response in the first is wanted, that should also the the case in the second, but it issnt, because, yeah... welcome back to the main issue)

[The-CJ]

no_success.txt
here the full Failed request, (just saw its truncated in gist, sorry)

[The-CJ]

Well, i was testing some more stuff, and tryed running example code from my home pc
(Win 10, Py 3.7, aiohttp 3.5.4)
And everything workes fine, without a problem...

https://gist.github.com/The-CJ/e582041992dcebf7a70af5cd79f738ee

both, the normal and the ajax way work, it also response with "413 Request Entity Too Large" if needed, workes totally fine
but not on my server,
one think i might not have mention, i have the server run in a sidethread with own asyncio loop.
is there everything else i could/should test?

[zhoujinhai]

Well, i was testing some more stuff, and tryed running example code from my home pc
(Win 10, Py 3.7, aiohttp 3.5.4)
And everything workes fine, without a problem...

https://gist.github.com/The-CJ/e582041992dcebf7a70af5cd79f738ee

both, the normal and the ajax way work, it also response with "413 Request Entity Too Large" if needed, workes totally fine
but not on my server,
one think i might not have mention, i have the server run in a sidethread with own asyncio loop.
is there everything else i could/should test?

Hello, I am have the same problem, my server is failed at await request.post(). Have you solved it? your link （https://gist.github.com/The-CJ/e582041992dcebf7a70af5cd79f738ee ） could not be opened