Do not repeat, dangerous, you can only use laser pointer - 1mW!
Here I will post information for creating a laser device.
- A warning!!
- How It Works
- General information
- Dimensions
- Galvanometer setting
- Determining the coordinates of an object
- Demonstrations
- We need more FPS
- Security questions
- Discussion and Future work
- Publication and Citation
- Contacts
The main limiting factor in the development of this technology is the danger of the laser may damage the eyes. The laser can enter a blood vessel and clog it, it can get into a blind spot where nerves from all over the eye go to the brain, you can burn out a line of "pixels" And then the damaged retina can begin to flake off, and this is the path to complete and irreversible loss of vision. This is dangerous because a person may not notice at the beginning of damage from a laser hit: there are no pain receptors there, the brain completes objects in damaged areas (remapping of dead pixels), and only when the damaged area becomes large enough person starts to notice that some objects not visible. We can develop additional security systems, such as human detection, audio sensors, etc. But in any case, we are not able to make the installation 100% safe, since even a laser can be reflected and damage the eye of a person who is not in the field of view of the device and at a distant distance. Therefore, this technology should not be used at home. My strong recommendation - don't use the power laser! I recommend making a device that will track an object using a safe laser pointer.
To detect x,y coordinates initially we used Haar cascades in RaspberryPI after that yolov4-tiny in Jetson nano.
For Y coordinates - stereo vision.
Calculation necessary value for the angle of mirrors.
RaspberryPI/JetsonNano by SPI sends a command for galvanometer via DAC mcp4922. Electrical scheme (here). From mcp4922 bibolar analog signal go to amplifair. Finally, we have -12 and + 12 V for control positions of the mirrors.
The principle of operation
Single board computer to processes the digital signal from the camera and determines positioning to the object, and transmits the digital signal to the analog display - 3, where digital-to-analog converts the signal to the range of 0-5V. Using a board with an operational amplifier, we get a bipolar voltage, from which the boards with the motor driver for the galvanometer are powered - 4, from where the signal goes to galvanometers -7. The galvanometer uses mirrors to change the direction of the laser - 6. The system is powered by the power supply - 5. Cameras 2 determine the distance to the object. The camera detects mosquito and transmits data to the galvanometer, which sets the mirrors in the correct position, and then the laser turns on.
1 - PI cameras, 2 - galvanometer, 3 - Jetson nano, 4 - adjusting the position to the object, 5 - laser device, 6 - power supply, 7 - galvanometer driver boards, 8 - analog conversion boards
In practice, the maximum deflection angle of the mirrors is set at the factory, but before use, it is necessary to check, for example, according to the documentation, our galvanometer had a step width of 30, but as it turned out we have only 20
Maximum and minimum positions of galvanometer mirrors:
a - lower position - 350 for x mirror;
b - upper position - 550 for x mirror;
c - lower position - 00 for y mirror;
d - upper position - 250 for y mirror;
We created GUI, source here.
At the expense of computer vision, the position of the object in the X, Y plane is determined - based on which its ROI area is taken. Then we use stereo vision to compile a depth map and for a given ROI with the NumPy library tool - np.average we calculated the average value for the pixels of this area, which will allow us to calculate the distance to the object.
You can find more detail in the published paper in preprint - Low-Cost Stereovision System (Disparity Map) For Few Dollars
The laser beam obeys all the optical laws of physics, therefore, depending on the design of the galvanometer, the required angle of inclination of the mirror – α, can be calculated through the geometrical formulas. In our case, through the tangent of the angle α, where it is equal to the ratio of the opposing side – X(Y) (position calculated by deep learning) to the adjacent side - Z (calculated by stereo vision).
For single boards, computers are actual problems with FPS. For one object with Jetson was reached the next result for the Yolov4-tiny model.
Framework
with Keras: 4-5 FPS
with Darknet: 12-15 FPS
with Darknet Tensor RT: 24-27 FPS
with Darknet DeepStream: 23-26 FPS
with tkDNN: 30-35 FPS
You can find more detail in the published paper in arxiv - Increasing FPS for single board computers and embedded computers in 2021 (Jetson nano and YOVOv4-tiny). Practice and review
In this video - a laser (the red point) tries to catch a yellow LED. It is an adjusting process but in fact, instead, a yellow LED can be a mosquito, and instead, the red laser can be a powerful laser.
In the video below, you can see, how the YoloV4tiny detected a cockroach and after that - turn on the laser and send a signal to the galvanometer to set the position of the mirror. When using a more powerful laser, the efficiency is much higher. But the video cannot be shot with a powerful laser, the light is too bright.
An additional device - a security module that will turn off the laser:
- Use additional cameras to fix people
- Audio sensors to capture voice and noise
- To mechanically shoot down the laser
- To use a thermal camera if there is any warm effect, turn it off - this is probably also possible to protect against fires consider not to overheat.
- Teach the system to record the process of laser reflection from any random glass or other mirror surfaces (maybe before turning on the power laser - for checking turn on the simple laser).
We can try light up the room to reflect the mosquito - and then use the library functions - OpenCV in range or haar cascades to detect the object. With a bright background, they will be detected without problems. This is for low-power single-board computers - Raspberry, Orange, Banana, etc. For jetson nano, we can use yolov4-tiny which, using the tkDNN library, is able to give 30-35 FPS
Use a lower the laser power as much as possible. The laser should burn the wings of mosquitos but should be safe for the eyes. That is to do research on the topic of laser power, laser wavelength, and their efficiency for mosquitos. This is for safety, the lower the power, the better.
Laser control on a stationary computer. The IP camera installed next to the laser only transmits video to the computer, and the computer already analyzes it on a powerful processor video card and transmits back coordinates for the laser via Wi-Fi. In this case, we can use very powerful computing processors.
Make the device completely on our electronic boards. It is a galvanometer for a laser show and changes positions 20,000 times per second, which is why there are such powerful and big drivers for motors. It is useful to make a small PCB board to change the position of the laser only 200 times per second. In finally, so to speak, the pocket version.
- Ildar, R. (2021). Machine vision for low-cost remote control of mosquitoes by power laser. Journal of Real-Time Image Processing
availabe here https://doi.org/10.1007/s11554-021-01079-x - Rakhmatulin I, Andreasen C. (2020). A Concept of a Compact and Inexpensive Device for Controlling Weeds with Laser Beams. Agronomy
availabe here https://doi.org/10.3390/agronomy10101616 - Rakhmatuiln I, Kamilaris A, Andreasen C. Deep Neural Networks to Detect Weeds from Crops in Agricultural Environments in Real-Time: A Review. Remote Sensing. 2021; 13(21):4486. https://doi.org/10.3390/rs13214486
- Rakhmatuiln, I., Lihoreau, M., & Pueyo, J. (2022). Selective neutralisation and deterring of cockroaches with laser automated by machine vision. Oriental Insects, https://doi.org/10.1080/00305316.2022.2121777
For any questions write to me by mail - ildarr2016@gmail.com