Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support StabilityAi Image Edit Api #1344

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

inpink
Copy link
Contributor

@inpink inpink commented Sep 11, 2024

Hello, this PR implements StabilityAi Image Edit - Remove Background API's client.


[The result of the execution]

Before After
before after

[Background]

StabilityAI provides various features for editing images via API requests.
Currently, Spring AI supports image generation using text, but lacks functionality for image editing.
This PR adds the StabilitAiEditApi class, which implements the Remove Background API from StabilityAI at the raw level.
The latest version of StabilityAI is V2 Beta, and for the API, please refer to StabilityAi Image Edit - Remove Background API.

[PR contents]

Component Details
Request DTO StabilityAiEditApi.RemoveBackgroundRequest record
Request Headers StabilityAiEditApi.ImageEditHeaders record
Response DTO Generic (StabilityAiEditApi.StructuredResponse record, byte[])
REST Call removeBackground method in StabilityAiEditApi Class
Test StabilityAiEditApiIT Class

  • request:
Field Type Description
image byte[] A binary string representing the image from which users want to remove the background.
output_format OutputFormat Enum The format of the output image. PNG or WEBP.

Users can submit an image as byte[] in formats such as png or webp.
Raw binary image data can be submitted using multipart form data.
An HTTP request is sent using RestClient.
Using ImageEditHeaders, users can add headers. Headers added at the constructor level are included in the RestClient builder, while those added in the HTTP request sending method removeBackground are applied only to that specific request.


  • response:
Response Type Description
StructuredResponse If a base64-encoded image is requested, the response is mapped to this class, containing the image, finish reason, and seed.
byte[] If a raw binary image is requested, the response is mapped to a byte array containing the image data.

StructuredResponse Class

Field Type Description
b64Image String The generated image, encoded in base64 format.
finishReason String The reason the generation finished. Possible values are:
- SUCCESS
- CONTENT_FILTERED
seed String The seed used as random noise for this generation, which can help reproduce the result.

The response type can be set using the Accept header in the request.
Depending on the requested response type, it returns either a base64-encoded image or raw binary image data.
If a base64-encoded image is requested, the JSON object is mapped to the StructuredResponse class. If a raw binary image is requested, it is mapped to byte[]. Java Generics are used to dynamically determine the response type, which the user specifies in the responseType parameter when calling the removeBackground method.


  • tests:
    Both tests ensure the functionality of the API in handling different response types (byte[] and StructuredResponse) and confirm that the data is processed and returned as expected.
  1. removeBackgroundToRawImage Test:
Test Case Description
Raw Binary Image Test This test verifies that the API correctly processes raw binary image data. A sample image (test.png) is submitted in byte[] format.
Method Tested removeBackground method
Expected Outcome The test ensures that the method returns raw binary image data as a byte[].
Validation The resulting image is saved and validated to confirm that it is not null.
  1. removeBackgroundToBase64Image Test:
Test Case Description
Base64-Encoded Image Test This test checks if the API properly handles the request to return a base64-encoded image. A sample image (test.png) is submitted with the output format WEBP.
Method Tested removeBackground method
Expected Outcome The response is expected as a JSON object mapped to the StructuredResponse class, containing a base64-encoded image.
Assertions The test asserts that the image, seed, and finish reason are present, and that the finish reason is "SUCCESS".
Validation The base64-encoded image is decoded and saved as a WEBP file for further validation.

[Related Issue]



I’m pleased to submit this PR to Spring AI.
Please let me know if any revisions are needed, and I will address them promptly.
I would be happy to assist with future maintenance as well, if necessary.

- Add StabilityAiEditApi that implements the Background-Remove endpoint.
- Add ITs
@markpollack
Copy link
Member

wow,thanks, getting to the PRs now, a nice addition! will merge soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants