How to upload multiple images as form-data with same key #1157

gersanco · 2020-05-14T13:59:12Z

Basic Info

Faraday Version: 1.0.1
Ruby Version: 2.6.1

Issue description

Hello, I'm developing an app and I need to upload multiples images as form-data with the same name. (https://developer.epages.com/beyond-docs/#upload_multiple_product_images)
This the curl call that I need to implement:

$ curl 'https://URL?fileName=file.png&fileName=file2.png' -i -X POST \
    -H 'Content-Type: multipart/form-data' \
    -H 'Authorization: Bearer <Access token>' \
    -F 'image=@/home/picture/file.png' \
    -F 'image=@/home/picture/file2.png'

How can I transcribe it to faraday?

I followed this issue #830 but I've been able to upload 1 file only if I try to upload multiples doesn't work.

Steps to reproduce

This is what I have:

# files = ["/home/pictures/file.png", "/home/pictures/file2.png"]
upload_files = files.map{ |file| Faraday::FilePart.new(file, 'image/png') }
request.body = { image: upload_files }

The text was updated successfully, but these errors were encountered:

iMacTia · 2020-05-14T16:18:08Z

@gersanco couple of questions from me:

does it work if you separate the images and use different names?
When you say that it doesn't work, do you have any additional details? Which request is being sent exactly and how does it differs from the one you expect?

gersanco · 2020-05-14T16:45:16Z

@iMacTia

That should be work. But I can't change the endpoint.
I get this from logs

One file

I, [2020-05-14T18:37:32.232373 #55747]  INFO -- request: POST https://team42.beyondshop.cloud/api/products/898783a5-0e3f-4061-a87f-a7d9f02b7e19/images?fileName=file1.png
I, [2020-05-14T18:37:32.232479 #55747]  INFO -- request: User-Agent: "Faraday v1.0.1"
Authorization: "Bearer [FILTERED]"
Content-Type: "multipart/form-data; boundary=-----------RubyMultipartPost-2480117c84209277bc50c5e022f8af84"
Content-Length: "2081"
I, [2020-05-14T18:37:32.237859 #55747]  INFO -- request: #<Faraday::CompositeReadIO:0x00005614c7fed738
 @index=0,
 @ios=
  [#<CompositeReadIO:0x00005614c7fed8c8
    @index=0,
    @ios=
     [#<StringIO:0x00005614c7fed940>,
      #<UploadIO:0x00005614c7feede0
       @content_type="image/png",
       @io=#<File:/home/gsanemeterio/Pictures/dummyImage600x400.png>,
       @local_path="/home/gsanemeterio/Pictures/dummyImage600x400.png",
       @opts={},
       @original_filename="dummyImage600x400.png">,
      #<StringIO:0x00005614c7fed918>]>,
   #<StringIO:0x00005614c7fed760>],
 @parts=
  [#<Parts::FilePart:0x00005614c7fedd50
    @foot="\r\n",
    @head=
     "-------------RubyMultipartPost-2480117c84209277bc50c5e022f8af84\r\n" +
     "Content-Disposition: form-data; name=\"image\"; filename=\"dummyImage600x400.png\"\r\n" +
     "Content-Length: 1783\r\n" +
     "Content-Type: image/png\r\n" +
     "Content-Transfer-Encoding: binary\r\n" +
     "\r\n",
    @io=
     #<CompositeReadIO:0x00005614c7fed8c8
      @index=0,
      @ios=
       [#<StringIO:0x00005614c7fed940>,
        #<UploadIO:0x00005614c7feede0
         @content_type="image/png",
         @io=#<File:/home/gsanemeterio/Pictures/dummyImage600x400.png>,
         @local_path="/home/gsanemeterio/Pictures/dummyImage600x400.png",
         @opts={},
         @original_filename="dummyImage600x400.png">,
        #<StringIO:0x00005614c7fed918>]>,
    @length=2014>,
   #<Parts::EpiloguePart:0x00005614c7fed7d8
    @io=#<StringIO:0x00005614c7fed760>,
    @part=
     "-------------RubyMultipartPost-2480117c84209277bc50c5e022f8af84--\r\n">]>

I received:

I, [2020-05-14T18:37:32.951945 #55747]  INFO -- response: Status 201
I, [2020-05-14T18:37:32.952129 #55747]  INFO -- response: date: "Thu, 14 May 2020 16:37:32 GMT"
content-type: "application/hal+json;charset=UTF-8"

When I try to upload 2 files:

I, [2020-05-14T18:43:09.467840 #56117]  INFO -- request: POST https://team42.beyondshop.cloud/api/products/898783a5-0e3f-4061-a87f-a7d9f02b7e19/images?fileName=file1.png&fileName=file2.png
I, [2020-05-14T18:43:09.467975 #56117]  INFO -- request: User-Agent: "Faraday v1.0.1"
Authorization: "Bearer [FILTERED]"
Content-Type: "multipart/form-data; boundary=-----------RubyMultipartPost-c9a99aa70b46a6e9fb6fcb2b7a5ee92a"
Content-Length: "3413"
I, [2020-05-14T18:43:09.470968 #56117]  INFO -- request: #<Faraday::CompositeReadIO:0x0000558f34bc8108
 @index=0,
 @ios=
  [#<CompositeReadIO:0x0000558f34bc9760
    @index=0,
    @ios=
     [#<StringIO:0x0000558f34bc97d8>,
      #<UploadIO:0x0000558f34bc5c00
       @content_type="image/png",
       @io=#<File:/home/gsanemeterio/Pictures/dummyImage600x400.png>,
       @local_path="/home/gsanemeterio/Pictures/dummyImage600x400.png",
       @opts={},
       @original_filename="dummyImage600x400.png">,
      #<StringIO:0x0000558f34bc9788>]>,
   #<CompositeReadIO:0x0000558f34bc89f0
    @index=0,
    @ios=
     [#<StringIO:0x0000558f34bc8ba8>,
      #<UploadIO:0x0000558f34bc58e0
       @content_type="image/png",
       @io=#<File:/home/gsanemeterio/Pictures/dummyImage400x400.png>,
       @local_path="/home/gsanemeterio/Pictures/dummyImage400x400.png",
       @opts={},
       @original_filename="dummyImage400x400.png">,
      #<StringIO:0x0000558f34bc8a18>]>,
   #<StringIO:0x0000558f34bc8130>],
 @parts=
  [#<Parts::FilePart:0x0000558f34bca020
    @foot="\r\n",
    @head=
     "-------------RubyMultipartPost-c9a99aa70b46a6e9fb6fcb2b7a5ee92a\r\n" +
     "Content-Disposition: form-data; name=\"image[]\"; filename=\"dummyImage600x400.png\"\r\n" +
     "Content-Length: 1783\r\n" +
     "Content-Type: image/png\r\n" +
     "Content-Transfer-Encoding: binary\r\n" +
     "\r\n",
    @io=
     #<CompositeReadIO:0x0000558f34bc9760
      @index=0,
      @ios=
       [#<StringIO:0x0000558f34bc97d8>,
        #<UploadIO:0x0000558f34bc5c00
         @content_type="image/png",
         @io=#<File:/home/gsanemeterio/Pictures/dummyImage600x400.png>,
         @local_path="/home/gsanemeterio/Pictures/dummyImage600x400.png",
         @opts={},
         @original_filename="dummyImage600x400.png">,
        #<StringIO:0x0000558f34bc9788>]>,
    @length=2016>,
   #<Parts::FilePart:0x0000558f34bc95a8
    @foot="\r\n",
    @head=
     "-------------RubyMultipartPost-c9a99aa70b46a6e9fb6fcb2b7a5ee92a\r\n" +
     "Content-Disposition: form-data; name=\"image[]\"; filename=\"dummyImage400x400.png\"\r\n" +
     "Content-Length: 1097\r\n" +
     "Content-Type: image/png\r\n" +
     "Content-Transfer-Encoding: binary\r\n" +
     "\r\n",
    @io=
     #<CompositeReadIO:0x0000558f34bc89f0
      @index=0,
      @ios=
       [#<StringIO:0x0000558f34bc8ba8>,
        #<UploadIO:0x0000558f34bc58e0
         @content_type="image/png",
         @io=#<File:/home/gsanemeterio/Pictures/dummyImage400x400.png>,
         @local_path="/home/gsanemeterio/Pictures/dummyImage400x400.png",
         @opts={},
         @original_filename="dummyImage400x400.png">,
        #<StringIO:0x0000558f34bc8a18>]>,
    @length=1330>,
   #<Parts::EpiloguePart:0x0000558f34bc8180
    @io=#<StringIO:0x0000558f34bc8130>,
    @part=
     "-------------RubyMultipartPost-c9a99aa70b46a6e9fb6fcb2b7a5ee92a--\r\n">]>

And I received:

I, [2020-05-14T18:43:09.987429 #56117]  INFO -- response: Status 400
I, [2020-05-14T18:43:09.987527 #56117]  INFO -- response: date: "Thu, 14 May 2020 16:43:09 GMT"
content-type: "application/json;charset=utf-8"
transfer-encoding: "chunked"
connection: "keep-alive"
x-content-type-options: "nosniff"
x-xss-protection: "1; mode=block"
cache-control: "no-cache, no-store, max-age=0, must-revalidate"
pragma: "no-cache"
expires: "0"
strict-transport-security: "max-age=31536000 ; includeSubDomains"
x-frame-options: "DENY"
server: "epages-beyond"
x-request-time: "0.093"
x-b3-traceid: "339ccc52972519f644fbccc69c578736"
x-hello-human: "Come work with us! https://developer.epages.com/devjobs/"
I, [2020-05-14T18:43:09.987584 #56117]  INFO -- response: {
  "errorId" : "upload-validation-error",
  "details" : { },
  "message" : "At least one file should be uploaded.",
  "traceId" : "339ccc52972519f644fbccc69c578736"
}

iMacTia · 2020-05-15T15:22:18Z

Thanks @gersanco, I believe the issue might be here:

@head=
     "-------------RubyMultipartPost-c9a99aa70b46a6e9fb6fcb2b7a5ee92a\r\n" +
     "Content-Disposition: form-data; name=\"image[]\"; filename=\"dummyImage600x400.png\"\r\n" +
     "Content-Length: 1783\r\n" +
     "Content-Type: image/png\r\n" +
     "Content-Transfer-Encoding: binary\r\n" +
     "\r\n",

The server expects name to probably be image rather than image[] (even though the latter makes more sense, since it's an array with multiple values).
I'll need to have a deeper look at how to go around that, bear with me.

iMacTia · 2020-05-15T15:26:43Z

Ah, I forgot about the good old FlatParamsEncoder 😄.
@gersanco could you please try passing request: { params_encoder: Faraday::FlatParamsEncoder } option when initialising your Faraday connection?

Something like this:

Faraday.new('...', request: { params_encoder: Faraday::FlatParamsEncoder })

If this still doesn't work, please share the logs again

gersanco · 2020-05-15T16:26:20Z

Hi @iMacTia
You mean instead of request.options[:params_encoder] = Faraday::FlatParamsEncoder right?
I received the same:

I, [2020-05-15T18:24:46.033468 #6282]  INFO -- request: POST https://team42.beyondshop.cloud/api/products/a653f979-900b-4d1d-bd8e-6b2553c24114/images?fileName=file1.png&fileName=file2.png
I, [2020-05-15T18:24:46.033521 #6282]  INFO -- request: User-Agent: "Faraday v1.0.1"
Authorization: "Bearer [FILTERED]"
Content-Type: "multipart/form-data; boundary=-----------RubyMultipartPost-add0dcb4dbcf0997ceb608f9416c2017"
Content-Length: "3413"
I, [2020-05-15T18:24:46.034686 #6282]  INFO -- request: #<Faraday::CompositeReadIO:0x000055a1250b9110
 @index=0,
 @ios=
  [#<CompositeReadIO:0x000055a1250ba650
    @index=0,
    @ios=
     [#<StringIO:0x000055a1250ba768>,
      #<UploadIO:0x000055a12509d848
       @content_type="image/png",
       @io=#<File:/home/gsanemeterio/Pictures/dummyImage600x400.png>,
       @local_path="/home/gsanemeterio/Pictures/dummyImage600x400.png",
       @opts={},
       @original_filename="dummyImage600x400.png">,
      #<StringIO:0x000055a1250ba678>]>,
   #<CompositeReadIO:0x000055a1250b94d0
    @index=0,
    @ios=
     [#<StringIO:0x000055a1250b96d8>,
      #<UploadIO:0x000055a12509d168
       @content_type="image/png",
       @io=#<File:/home/gsanemeterio/Pictures/dummyImage400x400.png>,
       @local_path="/home/gsanemeterio/Pictures/dummyImage400x400.png",
       @opts={},
       @original_filename="dummyImage400x400.png">,
      #<StringIO:0x000055a1250b9598>]>,
   #<StringIO:0x000055a1250b91d8>],
 @parts=
  [#<Parts::FilePart:0x000055a1250bb898
    @foot="\r\n",
    @head=
     "-------------RubyMultipartPost-add0dcb4dbcf0997ceb608f9416c2017\r\n" +
     "Content-Disposition: form-data; name=\"image[]\"; filename=\"dummyImage600x400.png\"\r\n" +
     "Content-Length: 1783\r\n" +
     "Content-Type: image/png\r\n" +
     "Content-Transfer-Encoding: binary\r\n" +
     "\r\n",
    @io=
     #<CompositeReadIO:0x000055a1250ba650
      @index=0,
      @ios=
       [#<StringIO:0x000055a1250ba768>,
        #<UploadIO:0x000055a12509d848
         @content_type="image/png",
         @io=#<File:/home/gsanemeterio/Pictures/dummyImage600x400.png>,
         @local_path="/home/gsanemeterio/Pictures/dummyImage600x400.png",
         @opts={},
         @original_filename="dummyImage600x400.png">,
        #<StringIO:0x000055a1250ba678>]>,
    @length=2016>,
   #<Parts::FilePart:0x000055a1250ba010
    @foot="\r\n",
    @head=
     "-------------RubyMultipartPost-add0dcb4dbcf0997ceb608f9416c2017\r\n" +
     "Content-Disposition: form-data; name=\"image[]\"; filename=\"dummyImage400x400.png\"\r\n" +
     "Content-Length: 1097\r\n" +
     "Content-Type: image/png\r\n" +
     "Content-Transfer-Encoding: binary\r\n" +
     "\r\n",
    @io=
     #<CompositeReadIO:0x000055a1250b94d0
      @index=0,
      @ios=
       [#<StringIO:0x000055a1250b96d8>,
        #<UploadIO:0x000055a12509d168
         @content_type="image/png",
         @io=#<File:/home/gsanemeterio/Pictures/dummyImage400x400.png>,
         @local_path="/home/gsanemeterio/Pictures/dummyImage400x400.png",
         @opts={},
         @original_filename="dummyImage400x400.png">,
        #<StringIO:0x000055a1250b9598>]>,
    @length=1330>,
   #<Parts::EpiloguePart:0x000055a1250b9340
    @io=#<StringIO:0x000055a1250b91d8>,
    @part=
     "-------------RubyMultipartPost-add0dcb4dbcf0997ceb608f9416c2017--\r\n">]>

I, [2020-05-15T18:24:46.485595 #6282]  INFO -- response: Status 400
I, [2020-05-15T18:24:46.485653 #6282]  INFO -- response: date: "Fri, 15 May 2020 16:24:46 GMT"
content-type: "application/json;charset=utf-8"
transfer-encoding: "chunked"
connection: "keep-alive"
x-content-type-options: "nosniff"
x-xss-protection: "1; mode=block"
cache-control: "no-cache, no-store, max-age=0, must-revalidate"
pragma: "no-cache"
expires: "0"
strict-transport-security: "max-age=31536000 ; includeSubDomains"
x-frame-options: "DENY"
server: "epages-beyond"
x-request-time: "0.075"
x-b3-traceid: "63fd2194f3b8d485200625b5a513f3d0"
x-hello-human: "Come work with us! https://developer.epages.com/devjobs/"
I, [2020-05-15T18:24:46.485683 #6282]  INFO -- response: {
  "errorId" : "upload-validation-error",
  "details" : { },
  "message" : "At least one file should be uploaded.",
  "traceId" : "63fd2194f3b8d485200625b5a513f3d0"
}

gersanco · 2020-05-15T16:31:34Z

Maybe it helps you if I send you how I make the request

request.url(session.api_url + path)
request.headers['Authorization'] = "Bearer #{ session.access_token }" unless session.access_token.nil?
# I had this before your last comment
# request.options[:params_encoder] = Faraday::FlatParamsEncoder
request.params = params.to_h.camelize_keys
upload_files = files.map{ |file| Faraday::FilePart.new(file, 'image/png') }
request.body = { image: upload_files }

iMacTia · 2020-05-23T16:05:36Z

@gersanco I've spent some time on this over the week but with no luck.
I assume this is probably an unusual case and I'm not really sure how to help.
If I get some more time I'll look again into it, but should you finally find a solution please share it here as it may be useful for future readers.

technoweenie · 2020-06-11T16:04:49Z

It's coming from here:

faraday/lib/faraday/request/multipart.rb

Lines 80 to 82 in d77c9ef

    
           def process_params(params, prefix = nil, pieces = nil, &block) 
        
             params.inject(pieces || []) do |all, (key, value)| 
        
               key = "#{prefix}[#{key}]" if prefix

Here's what happens:

#process_params is called on the form parameters, { image: upload_files } for example.
#process_params is called again on the upload_files array. It uses image as the prefix, and no key since it's an array.

@iMacTia The param encoders have a single encode method. What do you think about introducing something like #array_key(key) on FlatParamsEncoder and NestedParamsEncoder? The implementations will be really basic, but it would let us use the params encoders in the multipart middleware.

iMacTia · 2020-06-20T15:18:39Z

@technoweenie thanks for pointing me in the right direction! That makes sense, so there's a bit of duplication between the params encoders and the multipart middleware 🤔.
Your solution makes sense, although it creates a bit of dependency between them... that said I'm not sure I can come up with a better solution without going for a major refactoring 😂, so I'll probably take it this time 😄

gersanco · 2020-07-08T06:57:00Z

Hello,
Do you think this fix will be available in a future version?

iMacTia · 2020-07-08T07:12:39Z

@gersanco Absolutely! I managed to reproduce this issue in a test already and I’m now looking for a way to make this work based on the suggestion above 👍

gersanco · 2020-07-08T07:18:56Z

@iMacTia Perfect, thanks so much 😃

Fixes #1157

olleolleolle added the info Generic question on how to use Faraday label May 14, 2020

iMacTia added a commit that referenced this issue Jul 8, 2020

Introduces flat_encode option for multipart adapter.

925c346

Fixes #1157

iMacTia mentioned this issue Jul 8, 2020

Introduces flat_encode option for multipart adapter. #1163

Merged

iMacTia added a commit that referenced this issue Jul 9, 2020

Introduces flat_encode option for multipart adapter.

7b87d46

Fixes #1157

iMacTia closed this as completed in #1163 Jul 9, 2020

iMacTia added a commit that referenced this issue Jul 9, 2020

Introduces flat_encode option for multipart adapter. (#1163)

5acab36

Fixes #1157

iMacTia mentioned this issue Jul 22, 2020

Support for send array of files in the same value. #1169

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

How to upload multiple images as form-data with same key #1157

How to upload multiple images as form-data with same key #1157

gersanco commented May 14, 2020 •

edited by iMacTia

Loading

iMacTia commented May 14, 2020

gersanco commented May 14, 2020 •

edited by iMacTia

Loading

iMacTia commented May 15, 2020

iMacTia commented May 15, 2020

gersanco commented May 15, 2020

gersanco commented May 15, 2020 •

edited

Loading

iMacTia commented May 23, 2020

technoweenie commented Jun 11, 2020

iMacTia commented Jun 20, 2020

gersanco commented Jul 8, 2020

iMacTia commented Jul 8, 2020

gersanco commented Jul 8, 2020

How to upload multiple images as form-data with same key #1157

How to upload multiple images as form-data with same key #1157

Comments

gersanco commented May 14, 2020 • edited by iMacTia Loading

Basic Info

Issue description

Steps to reproduce

iMacTia commented May 14, 2020

gersanco commented May 14, 2020 • edited by iMacTia Loading

iMacTia commented May 15, 2020

iMacTia commented May 15, 2020

gersanco commented May 15, 2020

gersanco commented May 15, 2020 • edited Loading

iMacTia commented May 23, 2020

technoweenie commented Jun 11, 2020

iMacTia commented Jun 20, 2020

gersanco commented Jul 8, 2020

iMacTia commented Jul 8, 2020

gersanco commented Jul 8, 2020

gersanco commented May 14, 2020 •

edited by iMacTia

Loading

gersanco commented May 14, 2020 •

edited by iMacTia

Loading

gersanco commented May 15, 2020 •

edited

Loading