Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

draw image from stream #14

Closed
brainlessboy opened this issue May 29, 2018 · 27 comments
Closed

draw image from stream #14

brainlessboy opened this issue May 29, 2018 · 27 comments

Comments

@brainlessboy
Copy link

Hi (me again), I got it to work and its really cool, this opens a few very cool possibilities.
I am using the npm library canvas to draw offscreen images and animations, now to put them on the pitft i need to write a .png file before i can pass them over to the pitft, this is fairly slow and somewhat ugly and unnecessary (see following code) ... would it be possible to pass a stream directly to fb.image(...) ?

var Buffer = {
update: function (canvas) {
var out = fs.createWriteStream(image_path);
var stream = canvas.pngStream();
stream.pipe(out);
out.on('finish', function () {
fb.image(0, 0, image_path);
});
}
};
@vesteraas
Copy link
Owner

vesteraas commented May 29, 2018 via email

@brainlessboy
Copy link
Author

thanks for the tip, but i have not been able figured that out yet, i went throught doc and examples. there is some Buffer stuff, but i found nothing on how to use the framebuffer ...

@vesteraas
Copy link
Owner

This discussion looks promising: Automattic/node-canvas#1172

My guess is that external framebuffer support for node-canvas is just around the corner.

@zbjornson
Copy link

zbjornson commented May 30, 2018 •
edited
Loading

The branch in Automattic/node-canvas#1094 is a better bet for raw FB access; I don't think that will end up in the main node-canvas branch anytime soon. v2.x of canvas has RGB16_565 support already, and you can get the raw Buffer with canvas.toBuffer("raw") -- never used a FB myself, but maybe it's as simple as writing that RGB16_565 buffer to /dev/fb1?

(Edit: sounds like not quite that simple, maybe. https://unix.stackexchange.com/questions/20458/how-to-use-dev-fb0-as-a-console-from-userspace-or-output-text-to-it)

@vesteraas
Copy link
Owner

vesteraas commented May 30, 2018 via email

@brainlessboy
Copy link
Author

yes! i search through their repository and came across recent 'Added framebuffer device support' as well it seems there is some work going on, very exciting
to close this issue, pitft installation on node.js 10.3.0 flawless!!
so again, many thanks for your valuable tip, you can close this issue.

PS: i will stick to your library, its less weight and its fast :-) e.v. will adapt my code to this.

@brainlessboy
Copy link
Author

hmm, just read the feedback from zbjornson , i really can't see how the code would look like ...
any way, many thanks

@brainlessboy
Copy link
Author

me again :-) i was to quick, the install procedure works but ... when executing pitft it fails at this command

var fb = pitft("/dev/fb1");

with

FATAL ERROR: v8::ToLocalChecked Empty MaybeLocal.
Aborted

cheers

@vesteraas
Copy link
Owner

I've tested my updated library on node versions 8.x, 9.x and 10.x, and it works. Could you give me more details about your setup?

@zbjornson
Copy link

zbjornson commented May 30, 2018 •
edited
Loading

@brainlessboy I don't know much about FBs nor do I have a device or X server to test with right now, so this may be completely wrong, but here's a shot:

// Using canvas 2.x
const {createCanvas} = require("canvas");
const canvas = createCanvas(200, 200, {pixelFormat: "RGB16_565"});

// draw stuff on the canvas...

const buff = canvas.toBuffer("raw"); // buff is a Buffer containing RGB15_565 data

Then I think you can just write that to the device, like this:

const fs = require("fs");
const fb = fs.openSync("/dev/fb1", "w");
fs.writeSync(fb, buff, 0, buff.byteLength, 0);

This skips over using ioctl to read info about the FB. There are Node.js bindings for ioctl that might help if you need it, but I assume you know what your FB looks like in this case.

If that works, that's not as ugly as I thought it might be. I do want to find and document a nice way to work with FBs because it's a recurring issue in node-canvas. Options include adding it to node-canvas itself, writing a C++ plugin for node-canvas or writing a JS wrapper that does something like the above.

Please let us know if that works!

@vesteraas
Copy link
Owner

vesteraas commented May 30, 2018 via email •
edited
Loading

@brainlessboy
Copy link
Author

i am working with a raspberrypi 3 with raspbian light so not entierly sure what would help, e.v. the log from the node-gyp rebuild?

pi@raspberrypi:~/spghty/node_modules/pitft $ sudo node-gyp rebuild
gyp info it worked if it ends with ok
gyp info using node-gyp@3.6.2
gyp info using node@10.3.0 | linux | arm
gyp info spawn /usr/bin/python2
gyp info spawn args [ '/usr/lib/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args 'binding.gyp',
gyp info spawn args '-f',
gyp info spawn args 'make',
gyp info spawn args '-I',
gyp info spawn args '/home/pi/spghty/node_modules/pitft/build/config.gypi',
gyp info spawn args '-I',
gyp info spawn args '/usr/lib/node_modules/node-gyp/addon.gypi',
gyp info spawn args '-I',
gyp info spawn args '/root/.node-gyp/10.3.0/include/node/common.gypi',
gyp info spawn args '-Dlibrary=shared_library',
gyp info spawn args '-Dvisibility=default',
gyp info spawn args '-Dnode_root_dir=/root/.node-gyp/10.3.0',
gyp info spawn args '-Dnode_gyp_dir=/usr/lib/node_modules/node-gyp',
gyp info spawn args '-Dnode_lib_file=/root/.node-gyp/10.3.0/<(target_arch)/node.lib',
gyp info spawn args '-Dmodule_root_dir=/home/pi/spghty/node_modules/pitft',
gyp info spawn args '-Dnode_engine=v8',
gyp info spawn args '--depth=.',
gyp info spawn args '--no-parallel',
gyp info spawn args '--generator-output',
gyp info spawn args 'build',
gyp info spawn args '-Goutput_dir=.' ]
gyp info spawn make
gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
make: Entering directory '/home/pi/spghty/node_modules/pitft/build'
CXX(target) Release/obj.target/pitft/src/pitft.o
CXX(target) Release/obj.target/pitft/src/framebuffer.o
SOLINK_MODULE(target) Release/obj.target/pitft.node
COPY Release/pitft.node
make: Leaving directory '/home/pi/spghty/node_modules/pitft/build'
gyp info ok

@vesteraas
Copy link
Owner

vesteraas commented May 30, 2018 via email

@zbjornson
Copy link

@vesteraas neat! The image shows up, but then node crashes with EFBIG? Is it easy for you to post the ioctl info from your device? (I guess the important one is comparing buff.byteLength to the FB screen size.)

@brainlessboy
Copy link
Author

brainlessboy commented May 30, 2018 •
edited
Loading

@ vesteraas you are right, a little tinkering, i exchanged the pitft with a Pomironi Hyper Screen ... then i had to sudo apt-get install fbi (this adds quiet an amount of libs that deal with framebuffer, specially cairo)

after that fbi install and sending an image to the screen works (the pitft heart ;-), but using pitft gets me now a different error -> 'Segmentation fault'

@vesteraas
Copy link
Owner

@zbjornson ,

The buffer.byteLength has a value of 614400. Printing out canvas.width and canvas.height reports 480 and 320, but a ctx.fillRect(0,0,240,160) produces a rectangle that fills the entire screen, when it should only fill a quarter of the screen.

Not sure about the ioctl stuff.

@zbjornson
Copy link

Thanks @vesteraas -- looks like a bug in node-canvas, the buffer shouldn't be that big. Will look into it.

@zbjornson
Copy link

Ah actually -- I had a typo in my example. It should be pixelFormat: "RGB16_565", not 15_565. Does that fix it?

@brainlessboy
Copy link
Author

brainlessboy commented May 30, 2018 •
edited
Loading

@zbjornson @vesteraas IT WORKS!! and it runs very smooth at 800x480!
i use this gadget hooked to the raspberry https://github.com/pimoroni/hyperpixel

  • fbi needs to be installed specially for the extra libraries.
  • the node script needs to run in sudo (otherwise no access to the framebuffer)
  • and you need to get the right device (in my case with the HyperPixel its 0 not 1 or 2)

with your help i was able to actually stream a 3d javascript canvas based animated visualisation to a screen without X or a Browser! this also works with standard hdmi if no screen hat is available and only raspbian stretch. this opens up quiet a few possibilities ...

@vesteraas
Copy link
Owner

@zbjornson changing the pixel format to RGB16_565 did not help.

@vesteraas
Copy link
Owner

@brainlessboy the hyperpixel looks like a neat display! I will order one myself!

@vesteraas
Copy link
Owner

@zbjornson I was on version 1.6.x... Everything works, after npm install canvas@next and installing some dependencies. Thank you!

@zbjornson
Copy link

\o/ fantastic, thanks 😀

@brainlessboy
Copy link
Author

so after successful tinkering with the HyperPixel and your advise :-) i started with a slightly different setup, exact same software but a different PITFT (PiTFT 2.2 display with 320x240 16-bit color pixels)
on the node.js side i do the same thing, but now i do get a scrambled image (the motif is recognisable but scrambled/shifted/interlaced) and i get the following error message:

EFBIG: file too large, write

i am not sure where to tweak ... either on the canvas side?

const canvas = createCanvas(320, 240,{pixelFormat: "RGB16_565"});

or on the buffer write side ...

const buff = canvas.toBuffer("raw");

e.v. you have tip :-)

@zbjornson
Copy link

@brainlessboy are you definitely using canvas v2.x? (npm i canvas@next or yarn add canvas@next)

I put a full example here:
https://github.com/Automattic/node-canvas/wiki/Using-with-framebuffer-devices

@brainlessboy
Copy link
Author

@zbjornson exacctly
npm i canvas@next (i do however have to rebuild node-gyp manually after installation)
i have the exact same code you propose. (works super with the HyperPixel TFT)
i also tested the smaller 2.2 PITFT with fbi and a jpg image (this is a test adafruit is proposing) so the wiring must be correct. Also an image is sent to the display but its kind of shifted/interlaced, this probably means the pixel information is not correct either to much information in the buffer or not enough ... the PITFT is a 16 bit display e.v. this is a hint ... may be i should build the buffer in a different way ...

@brainlessboy
Copy link
Author

brainlessboy commented Jun 6, 2018 •
edited
Loading

any idea what i could try? or any recommendation on what direction i could tinker? like playing with the stream, alter it or what ever?

@m-s-sh m-s-sh mentioned this issue Sep 23, 2018
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@vesteraas @zbjornson @brainlessboy