Image provider for remote public images: RemoteImageProvider

[marklagendijk]

Currently ImageSharp does not yet include a RemoteImageProvider: an image provider which downloads images from a publicly available remote location. Since I need one I am considering implementing it, and making a pull request for it.

Proposed design

RemoteImageProvider

• ProcessingBehavior: All
• Match: by default: paths starting with /remote. Can be overridden to custom function.
• IsValidRequest: by default: allow configuring a list of allowed domains. This means that unless you configure these domains, it will do nothing (security by default). Can be overridden to custom function.
• GetAsync: returns a RemoteImageResolver

RemoteImageResolver

• OpenReadAsync: executes a HTTP GET request and returns the stream of the response body.
• GetMetaDataAsync: executes a HTTP HEAD request and returns the date from the Last-Modified header and content length from the Content-Length header (which is the content length of the body of the GET request).

Url format

The tricky question is whether the remote url should be read from a query param or a path param.

Query param:
/remote?{COMMANDS}&src=http%3A%2F%2Fexample.com%2Fimage.jpg

Path param
/remote/http://example.com/image.jpg?{COMMANDS}

Considerations:

• Query param: requires escaping by the user of ImageSharp.
• Path param: is valid according to spec, but may trigger overzealous security protections. Does .NET core have such protections?

@JimBobSquarePants, @dlemstra: Could you please review this design? Since the functionality is so common, I would expect that it would be fine to include it in ImageSharp.Web directly, right?

[marklagendijk]

I have a working POC, see this gist: https://gist.github.com/marklagendijk/e5fd1934391a515f8c974c87024ed39c

Some extra questions:

• The HttpClient can be created using IHttpClientFactory.CreateClient(string name), see documentation. This way users can configure the HttpClient via dependency injection, so that they can customize it. If no matching client config is found, it will give back a client with the default config. Question: do we want to use this mechanism to allow the user to configure custom values for: DefaultRequestHeaders, Timeout, MaxResponseContentBufferSize? Probably we can register default config for the name, which the user can override.
• In RemoteImageResolver.GetMetaDataAsync it may happen that no LastModified or ContentLength are included on the response. What should happen in such case? Throw an exception? If so, what kind of exception?
• I currenlty specified IRemoteImageUrlValidator with AllowedDomainsRemoteImageUrlValidator as default implementation. Do you like this approach, or would you prefer something different?

[JimBobSquarePants]

I'm afraid this is not something I want to bring into the core offering. It was a conscious decision not to do so.

There's a few reasons:

1. It adds a dependency on HttpClientFactory which then brings additional configuration complexity
2. To be truly effective you would have to introduce an instance per domain. Each potential domain brings authentication issues, url parsing issues and other oddities.

It would be a mistake attempting to bring something into the codebase that is designed as a catch-all. If someone wants to introduce a new IImageProvider they can develop and plugin an unlimited collection independently.

[marklagendijk]

I agree that trying to implement a catch-all is a bad idea.
However, I do believe that if we define clearly what use cases are supported and which not, we can still make something that is simple and useful.

1. It adds a dependency on `HttpClientFactory` which then brings additional configuration complexity

In the POC use a single named client configuration. I think this is reasonable for most use cases (and other use cases are out of scope). With this the user only has to do something like:

services.AddHttpClient(RemoteImageResolver.HttpClientName, client =>
{
    client.Timeout = TimeSpan.FromSeconds(5);
    client.MaxResponseContentBufferSize = 8388608; //8mb
});

2. To be truly effective you would have to introduce an instance per domain. Each potential domain brings authentication issues, url parsing issues and other oddities.

You mean effective functionality wise instead of performance wise, correct? In that case I would say that the oddities are out of scope.

I think that with the POC I'm close to something that is useful for 80% of the use cases:

• It can download from publicly available urls. Authentication is out of scope.
• It has support for a domain-based allowlist. Since the validator is an interface other use cases can easily be implemented by the users.
• DefaultRequestHeaders, Timeout, MaxResponseContentBufferSize can easily be configured by the user.

What do you think? Am I oversimplifying things? Am I missing things?

[JimBobSquarePants]

It adds a dependency on HttpClientFactory which then brings additional configuration complexity

By including IHttpClientFactory we require a reference to Microsoft.Extensions.Http. I don't want to add that dependency since it is administered out of bounds. For .NET Core 3.1+ we have a single dependency. Microsoft.AspNetCore.App which I'd like to maintain.

With the provider you're suggesting only URLs without any additional querystring parameters would be supported. That rules out Facebook and other media providers which was a common use case for the old provider in ImageProcessor.Web (In fact, why it was built). This is too limiting and if I added the provider someone would soon come along asking for additional support.

I designed the API to be easily extendible precisely so that I wouldn't have to maintain multiple providers. Each one is a developmental burden I simply do not have time to take on.

So I'm afraid it's going to have to be a no for now. Maybe I'll change my mind in the future but you are more than welcome to release your own extensions in the interim.

[marklagendijk]

I see your point. Making something that is easy to use, which also fits most use cases is non trivial.
For now I'll make a custom solution which fits our own purpose. Maybe I'll try to make it generally usable in the future.