-
Notifications
You must be signed in to change notification settings - Fork 74
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Rendering the image from the clipboard #62
Comments
I think you should use the Rust ImageData bytes to construct a new JS
ImageData:
https://developer.mozilla.org/en-US/docs/Web/API/ImageData/ImageData
Then pass the JS ImageData to the putImageData function of the canvas:
https://developer.mozilla.org/en-US/docs/Web/API/CanvasRenderingContext2D/putImageData
…On Wed, 6 Jul 2022 at 08:54, Faraz Patankar ***@***.***> wrote:
Hey y'all, Rust newbie here. For context, I am building a desktop app with
Tauri <https://tauri.app/> and using arboard to read/write images from/to
the clipboard. The writing part works fine but the part I am struggling
with is taking the image I read from the clipboard and rendering it on the
client so the user can view and edit the image.
Things I've tried in JS:
- Converting the image.bytes into a blob/base64 string.
- Rendering the image.bytes into a canvas.
Things I've tried in Rust:
- Writing the image.bytes to a file using this approach
<https://www.reddit.com/r/rust/comments/ooloe7/how_to_grab_an_image_from_the_clipboard_and_copy/>
.
In Rust, the file gets saved by when I open it, I just see this
<https://user-images.githubusercontent.com/10681116/177487082-0ed2ef3f-be4c-4586-b5aa-36611734155b.png>
.
I think the issue is probably with my understanding of what image.bytes
returns and how to use it to actually convert it into a PNG or any other
format that I can use to display the image. I know the image itself is in
the form of this ImageData
<https://docs.rs/arboard/2.1.1/arboard/struct.ImageData.html> struct but
I don't really understand much beyond that.
Could someone help walk me through this or give me some pointers so I can
further experiment with implementing this feature within my app?
Additional context: My dev environment is MacOS 12.4 and my frontend is
written in React.
—
Reply to this email directly, view it on GitHub
<#62>, or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AB7PKCBCFFEH52C4IEKB2L3VSUUUFANCNFSM52YSYJIA>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
Thanks for the response, @ArturKovacs. I believe I have tried doing what you're suggesting but to no avail. Any chance you could share a code example? Here's what I've tried to do in JS: export async function bytesToBlob(
Image: Image,
): Promise<{ blob: Blob; src: string }> {
const canvas = document.createElement("canvas");
const context = canvas.getContext("2d")!;
canvas.width = Image.width;
canvas.height = Image.height;
const imgData = context.createImageData(Image.width, Image.height);
imgData.data.set(Image.bytes);
context.putImageData(imgData, 10, 10);
return new Promise(resolve => {
canvas.toBlob(file =>
resolve({
blob: file!,
src: URL.createObjectURL(file!),
}),
);
});
} |
First of all, you should decide where you want to render it.
|
So the code I shared just happened to be the last thing I tried. I have tried rendering it to a I have also tried rendering the image to a file using the fn get_image_from_clipboard() {
let mut ctx = Clipboard::new().unwrap();
let image = ctx.get_image().unwrap();
let imgbuf = ImageBuffer::from_raw(
image.width as u32,
image.height as u32,
image.bytes.into_owned(),
)
.unwrap();
let imgbuf = DynamicImage::ImageRgba8(imgbuf);
imgbuf.save("clipboard-image.png");
} My end goal is to display the image to the user within the UI so they can edit it. I tried saving the image to file after I couldn't get it to render with JS hoping to read that file on the client but that approach gives me the same output as well. |
I have a feeling that you are copying the file and not the pixel values. The two things are not the same. Note that arboard does not currently support copying and pasting files. Here are two example situation where the user would copy the pixel values (which is supported by arboard)
The Rust code you pasted above should work correctly. So if that doesn't work, then the problem might be somewhere else. (Again, maybe you are trying to copy the file) If I were in your place, I would create a 1 pixel image where you know the exact RGB values of the pixel (with a GIMP or something like that), and copy that image to the clipboard, and then use For example the 1 pixel image where the pixel is red should have the following bytes:
|
Okay, that makes a lot of sense. I was indeed copying the file as an easy way to test the implementation when in reality, I'd imagine if the user has the file saved, they'd rather drag-n-drop or open the file through the file select UI I have provided.
Edit: Lastly, just to confirm, is there an easier way to render this image on the client that I haven't thought of and is different from the two options I have listed above? Perhaps something that does not involve using the canvas? |
Even if there is, it wouldn't be a good way of dealing with this problem. The optimal way of dealing with this is to allow users to paste/drag-n-drop files. And actually both the drag-n-drop and the clipboard is the responsibility of that part of the code which manages the windows and input. So in your case handling the clipboard and drag-n-drop should be a feature of Tauri. So I suggest that you open an issue for them if this is not supported by Tauri. (This also means that optimally,
This one: "Sending the image data to the client, rendering a canvas, and converting it to an image." You should not write to the disk unless you want information to be preserved after the application is closed.
If you want to allow the user to edit the image, then I think the best approach is to use the canvas and try to avoid file operations as much as possible. |
i replace from_raw -> from_vec and workd on macos fn get_image_from_clipboard() {
let mut ctx = Clipboard::new().unwrap();
let image = ctx.get_image().unwrap();
let imgbuf = ImageBuffer::from_vec(
image.width as u32,
image.height as u32,
image.bytes.into_owned(),
)
.unwrap();
let imgbuf = DynamicImage::ImageRgba8(imgbuf);
imgbuf.save("clipboard-image.png");
} |
Since it looks like this was figured out and a working solution was found, I'm going to close this in an effort to keep the issue tracker tidy. |
Hey y'all, Rust newbie here. For context, I am building a desktop app with Tauri and using
arboard
to read/write images from/to the clipboard. The writing part works fine but the part I am struggling with is taking the image I read from the clipboard and rendering it on the client so the user can view and edit the image.Things I've tried in JS:
image.bytes
into a blob/base64 string.image.bytes
into a canvas.Things I've tried in Rust:
image.bytes
to a file using this approach.In Rust, the file gets saved by when I open it, I just see this.
I think the issue is probably with my understanding of what
image.bytes
returns and how to use it to actually convert it into a PNG or any other format that I can use to display the image. I know the image itself is in the form of thisImageData
struct but I don't really understand much beyond that.Could someone help walk me through this or give me some pointers so I can further experiment with implementing this feature within my app?
Additional context: My dev environment is MacOS 12.4 and my frontend is written in React.
The text was updated successfully, but these errors were encountered: