Pied Piper Updates
Since the last deployment in Fall of 2021, the Pied Piper traps have received several major design changes aimed at making the traps easier to deploy, easier to construct, and more capable. This update provides an overview of all of the most significant hardware and software improvements that have been made.
The battery life of the traps has been extended to 3 months, and this was primarily achieved through the implementation of solar power. The traps now use an external 6V 1A solar panel (and an associated solar charge controller) to recharge their batteries while deployed, which can be attached to the imaging box at one of two locations, depending on the mounting configuration. The image below shows a trap deployed at Stag Hollow Vineyards (during a deployment that is ongoing at the time of this update) with the solar panel visible:
The solar panel produces enough power to allow the traps to operate indefinitely even under occasional sub-optimal weather conditions. The addition of the solar panel has also allowed the battery capacity of the traps to be reduced, which has allowed the traps to become smaller, lighter, and less expensive.
An additional hardware change that has improved the battery life of the traps is the elimination of the AudioFX soundboard for playing back recorded mating calls. This was primarily done to reduce cost, but it has the added benefit of reducing the average current consumption of the trap by ~6 mA, which allows the aforementioned smaller batteries to continue to provide reliable power. The soundboard has been replaced by a small, lower-power amplifier for driving the vibration exciter, and the audio signal is now produced by the microcontroller's onboard DAC.
The mechanical design of the traps has been simplified to allow them to be cheaper and easier to fabricate. The traps no longer have the complicated gasket system used in the previous version to isolate the vibration exciter from the rest of the case. The previous system (shown below) required a significant amount of time and effort to construct, and also required the use of multiple specialized hand tools.
The new design has the vibration exciter installed directly on the inside surface of the case, without any mechanical isolation. This new method is still effective at transferring vibrations from the trap to the attached substrate because the mass of the traps themselves has been reduced. The mass of the traps was reduced primarily by using smaller batteries (as mentioned in the previous section) and by using a smaller, lighter waterproof case.
The reduced mass of the traps now allows them to support a wider range of mounting configurations; most notably, they can be mounted directly on vineyard trellis systems without any external support, as shown below:
Additional design simplifications that have been made are:
- Heavier use of 3D-printing and laser-cutting to reduce manual labor: The traps can now be built with minimal need for careful measuring, cutting, and drilling, which reduces both the time required to build the traps and the likelihood of human error. Examples of this include the inclusion of a new laser-cut backplate for mounting all of the internal electronics, and a laser-cut imaging box.
- No need to install an acrylic window: The previous design used an opaque plastic case which needed to have a window cut out and installed to allow the camera module to see the substrate. The new design uses a case that has a transparent lid, which eliminates this step during construction.
The traps are now configured to automatically test every major hardware subsystem (including the camera module, camera flash, SD card, audio output amplifier, and vibration exciter) every time they are powered on, and provide immediate indication of any problems using the microcontroller’s NeoPixel LED. This eliminates the need to use an external device to configure and test the traps through the serial console during deployment, and thus significantly reduces the amount of time required to deploy each trap.
Another new capability enabled by the the hardware self-test routine is the ability to gracefully recovery from battery depletion: if the traps ever lose power due to an extended period of overcast weather, they are able to power on and resume data collection as soon as weather improves and the batteries begin to recharge.
The quality of the substrate-borne audio recording has been improved substantially. This was done to improve the accuracy of the mating call detection algorithm and reduce the amount of digital post-processing required to convert the audio into a human-listenable format. The quality improvement is achieved primarily through a doubling of the sampling frequency (from 2048 Hz to 4096 Hz) and the inclusion of a more aggressive low-pass filter at the input of the microphone preamplifier. These changes allow the recorded audio to have a much higher bandwidth (and thus allow higher-frequency features of treehopper mating calls to be present) and have less blind time resulting from clipping. The clipping was primarily caused by occasional, sudden changes in signal level from environmental noise that would briefly overload the preamplifier, and the low-pass filter (along with better tuning of the preamplifier’s automatic gain control settings) are now able to eliminate most of this noise.
The traps now use a newer, higher-resolution camera module to take images. The new cameras are configured to have a much shorted focal length than the previous version, which allows more of the substrate closer to the electronics box to be in-focus. The new cameras also perform better in low-light conditions, which allows for power savings from not needing to provide large amounts of light for illuminating the substrate.
The new camera modules are able to communicate directly with the microcontroller to save image data to the microSD card connected to the Hypnos board. This means that there is now only one microSD card that must be removed and erased when re-deploying the traps, rather than two (the previous camera module had a separate SD card that needed to be serviced when collecting data and re-deploying).
All four units have been retrieved from Stag Hollow Vineyards, and their recorded data has been recovered. The data is currently pending analysis by Vaughn Walton. The Pied Piper units appear to have held up well in the field, although Unit #3 seems to have experienced some problems with its camera module, as it only recorded a small number of corrupted images. The causes of Unit #3's apparent camera module failure are being investigated, and the possibility of recovering usable images from the unit is being explored (as it's possible that the files are still present on the unit's uSD card, but are just inaccessible).
Four Pied Piper units have been deployed at Stag Hollow Vineyards, and will remain in the field for at least two weeks (and possibly left until as late as mid-October). One of these units uses the recently completed hardware revision; this unit will be functionally identical to the others with regards to its ability to detect, lure, and image insects, although it will have a significantly longer battery life (see previous updates regarding the hardware revision for why). This field deployment is particularly important, because it will be used to validate the final hardware revision prior to publication. It is also the first time that the new system has been deployed at a vineyard, which is the intended setting for the system.
Image of the revised unit currently deployed at Stag Hollow Vineyards:
Vincent Vaughn
All four Pied Piper units are being deployed at Oak Creek again in order to perform field testing and validation. Various software changes have been made since the last deployment in order to maximize the chances of success, which include:
- Increasing the frequency of control images
- Increasing the frequency of intermittent playback
- Increasing the frequency and duration of detection images
- Better tuning of the detection algorithm
Image of one of the units currently deployed at Oak Creek
Vincent Vaughn
All four Pied Piper units deployed on July 2nd have been recovered, and the data they collected has been analyzed. The units detected thirteen total mating call detections, all of which appear to be genuine (which indicates that the problem of a high false positive rate affecting earlier versions of the system has been eliminated). However, the units were not able to provide much conclusive photographic evidence of these detections, because of a combination of issues with the imaging system that occurred:
- Plant growth over the deployment period caused the vibration exciter wire and contact microphone to become obscured over time.
- The units tended to shift around, which caused the vibration exciter wire and contact microphone to exit the field of view of the camera in some cases.
- Insects tended to appear a significant distance away from the camera.
In order to remedy these problems for the next deployment (which will occur in early August), we will operate the traps entirely in the closed configuration, reduce the length of the vibration exciter wire (thus bringing the camera closer to the substrate), and make some minor hardware revisions.
Spectrogram of audio responsible for one of the positive detections:
Example of plant growth obscuring the camera towards the end of the deployment:
Vincent Vaughn
Beginning on Friday July 2nd, initial deployment of all four existing Pied Piper units began at the Oak Creek Center for Urban Horticulture, located near OSU's campus. By direction of Vaughn Walton, the units were placed at somewhat shaded locations near boundary environments (places where a clear, sunny area abruptly ends and dense foliage begins), as these are the locations that treehoppers are most likely to be found. The units were left for a full 24 hours after they were initially placed, and were subsequently serviced the next day to ensure that no immediate problems arose; they will be left to collect data for approximately two weeks before they are retrieved.
Images of all four units currently deployed in the field:
Vincent Vaughn
Two new Pied Piper units are being constructed, and deployment of the system is expected to begin in early Summer. The camera has now been fully integrated into the devices, and only awaits some 3D printed components to be completed.
Images of the new system. Note that the vibration exciter is now incorporated into the electronics case, which makes the device significantly less cumbersome to set-up.
Vincent Vaughn
In order to simplify the design of the trap, the camera is now being fully integrated into the waterproof case containing the rest of the system's electronics, rather than placing it into an external module. This will make the system cheaper and slightly easier to build.
A circular hole will be cut into the top lid of the case, and a small acrylic window will be installed in order to make a waterproof window for the camera.
The insect trap is now being redesigned to include a camera, which will be used to image insects. This change is being made due to concerns regarding the efficacy of the trap, as it has only been tested under laboratory conditions and it may not be able to prevent insects from escaping. Two new designs are being investigated: one involves imaging insects in the open while they're still on the substrate, and the other is more similar to the current system, and involves luring insects into a hanging imaging chamber.
Hanging design:
"Open" design:
Vincent Vaughn
In order to reduce the false positive rate of the detection algorithm, various measures to reduce environmental noise in the audio feed have been implemented (see previous update). In order to determine whether these measures are effective, a Pied Piper unit was deployed outdoors for two days at a location where there were definitely no treehoppers present. The unit recorded no detections during this deployment; the only detections reported by the unit were from tests performed with simulated mating calls just before it was left for the two days. This indicates that the false positive rate of the detection algorithm has successfully been reduced.
Pied Piper unit installed on English Ivy during light field testing:
Vincent Vaughn
When Pied Piper units were deployed during summer of 2020, it was found that the detection algorithm tended to produce false positives. This was largely the result of environmental noise (mainly from wind) interfering with the audio signal. To reduce this noise, we have implemented a digital filter and fine-tuned the settings of the preamp IC used to condition incoming audio. Testing has revealed that these measures are successful in reducing noise in the audio signal: a Pied Piper unit was set up outdoors during windy conditions, and it was still able to detect simulated treehopper mating calls.
Spectrogram of Pied Piper audio feed under noisy conditions:
Note that the mating calls (the swept horizontal streaks) are still clearly visible.
Vincent Vaughn
In order to reduce power consumption and improve battery life of the PiedPiper unit, the Adafruit AudioFX sound board is being removed. It is currently used for storing and playing back mating call audio when an insect is detected, and we intend to replace it with the onboard DAC on the M4 Express. This requires a minor hardware modification to bypass the AudioFX, as well as various software changes required to interface with the onboard DAC. Audio playback will then work by reading a wave file from the SD card on the Adalogger and continuously writing its samples to the analog output of the M4 Express, which will be wired directly to the amplifier.
Microcontroller analog output wired directly to amplifier input:
Vincent Vaughn
To further accelerate development of the detection algorithm without access to live specimens, we have begun collecting samples of ambient environmental noise. These samples will be spliced onto our existing mating call audio, which will allow us to fine-tune the detection algorithm to reduce its false-positive rate.
The noise is being collected from nearby English ivy (which should provide a reasonable analogue for a grapevine) under various weather conditions.
Vincent Vaughn
The initial results of the digital filter are very promising towards eliminating noise outside the target spectrum of 100-250 Hz for the system's detection algorithm. The filter was tested on White Gaussian Noise, the male treehopper mating call, and the male treehopper mating call with additive white gaussian noise. The images below show the original time and frequency domain plots of the input signal and the filtered output signal.
Original and filtered version of male treehopper signal
Original and filtered version of White Gaussian Noise
Original and filtered version of male treehopper signal with additive White Gaussian Noise
As seen in the above images, the filter decreases the strength of frequencies outside of the 100-250Hz and can help to eliminate high frequency noise from the input of the system. Further testing with a higher order filter is in the process of being performed and will be updated soon.
Elijah Shumway
A 1st order butterworth bandpass filter is in the process of being tested for its efficacy at removing high frequency and electrical noise from the input to the system. The filter will be tested against a variety of signals including pure white noise, the male treehopper mating call, and the male mating call with added noise. The magnitude and frequency plots will be compared to determine the effectiveness of the filter at removing excess noise from the incoming signal
Elijah Shumway
A spectrogram tool (and a corresponding new operating mode) has been developed for plotting live audio data from a Pied Piper unit. The tool can be used to visualize how digital processing affects the incoming signal, and to help with calibrating settings on the audio conditioning IC.
Example of audio spectrum data:
The tool is implemented in Python, and it is run on a computer connected via serial COM port to a Pied Piper unit. The unit must be put into the LiveSpectrogram operating mode in order to begin streaming data, which is achieved by setting the board's DIP switches to [1 0 0]. Both raw audio samples and audio spectrum data can be displayed.
Vincent Vaughn
New power modes have been implemented into the SAMD51 architecture of the FeatherM4. Currently supported for Pied Piper are the OFF, HIBERNATE, and BACKUP modes. There are still issues with the Watchdog timer in the IDLE mode that cause the board to wake immediately. The Hypnos board will soon be integrated to wake the Pied Piper system after a set amount of time, drastically improving the power draw of the system.
Elijah Shumway
In order to facilitate testing without access to live insects, two new methods have been developed. The first generates a replica of the male mating call with a user controlled signal to noise ratio(SNR). This will allow the team to test the response of the system under controlled noisy situations, allowing us to judge the minimum SNR required for a successful detection. The second is a tool that allows for live spectrogram readings of an incoming signal into the Pied Piper system. This will allow the team to visualize any incoming signal in the frequency domain, furthering the effectiveness of detection and the ability to adjust the audio conditioning IC parameters.
Elijah Shumway
A new version of the detection algorithm is in the process of development and testing. This new algorithm detects the time distance between peaks instead of the number of peaks per second to eliminate false positives caused by high frequency noise. Additionally, a digital low pass filter is being considered to eliminate magnitudes at high frequencies to focus the detection algorithm only on the frequency band of the Treehoppers. Further updates of the effectiveness of this algorithm vs the original will be posted as development and testing continues
Elijah Shumway
After the second round of field testing, the collected data was analyzed. This data showed a large quantity of false positives that had not showed up in lab testing. Most of these false positives can be attributed to high frequency peaks in amplitude from noise on the vine that tripped the peak detection component of the detection algorithm. This discovery has led to the detection algorithm to be reanalyzed for efficacy. The peak detection algorithm is being refined to calculate the time difference between peaks instead of counting the number of peaks in a set time period to attempt to remove these false positives
Elijah Shumway
Second round of testing is currently being conducted to address some of the issues identified in the initial field testing. Additionally, battery capacity will be measured to determine project lifetime.
Elijah Shumway
Two Pied Piper systems were placed out for field testing at Stag Hollow Vineyards. During this phase of testing, the Pied Piper was left in full detection mode to attempt to determine the presence of and attract treehoppers from the grape vines. One system was attached directly to a vine on the vineyard, while another was attached to a crab apple tree directly adjacent to the property
Elijah Shumway
In order to prepare for field testing the ability to record was implemented onto the Pied Piper system. During detection, a half second recording is generated by sampling incoming values from the contact microphone. When an insect is detected, this recording is saved to the built in SD card to be processed and analyzed later
Elijah Shumway
A new PCB for the Pied Piper has been designed and is in the process of being manufactured. The new board includes breakouts for the individual potentiometers that control the audio conditioner allowing for ease of access with the user. Additionally, a DIP switch has been added to control different modes of the system. Finally, the board has been designed to run off of a single 12 volt power supply to reduce the number of external connections needed.
Elijah Shumway
The new vibration exciter and trap were tested, and the exciter can easily propagate signals further than one meter through a substrate. Additionally, the Pied Piper system was able to successfully detect a simulated male mating call from over a meter away and triggered the female mating call for playback.
Elijah Shumway
The trap for the Pied Piper has been modeled and printed and is now ready for use. Construction of the full Pied Piper system is underway and is on its way to be ready for full system testing by the following week.
Elijah Shumway
The previous LRA used for vibration excitement has been traded out for a 30 mm Dayton audio exciter. This audio exciter is much stronger and can propagate the mimic signal much farther through the substrate. Additionally, the trap for the system is in the process of being 3D modeled.
Elijah Shumway
A new contact mic housing has been designed to provide a stronger connection between the contact microphone and the substrate. Once printed, this new enclosure will be tested to determine the accuracy of the system.
Elijah Shumway
After successful initial tests with insect detection, further testing was performed to determine the accuracy of the system. In these tests, a linear resonant actuator(LRA) was used to simulate an insect. The LRA was tuned against previous recordings using a laser vibrometer to ensure the LRA was producing the same amplitude and frequency spectrum of the insect in question. After tuning to the correct parameters, the Pied Piper system was attached to the substrate to test its accuracy in recognizing the insect at various distances from the LRA. At each distance the system was tested against 6 repetitions of the insects mating call, and the number of successful detections were counted. After testing, it was determined that the Pied Piper system can detect the male mating call with 86.67% accuracy up to a distance of 25 cm. Further testing is planned to determine accuracy at longer distances and to gain insight into the overall accuracy of the system
Elijah Shumway
Redesigned the PCB to use an LM386 audio amplifier instead of the MAX9744. This switch cut the size of the PCB approximately in half, and will decrease the power required the run the system in playback mode. An attached gain and volume control will allow the board to be tuned to match the amplitude of the physical insect.
Elijah Shumway
Discovered an issue in the code that caused the shift register to behave strangely. The TPIC6B595 is now able to correctly turn on different inputs to the Audio FX board, allowing for a maximum of 8 different audio files to be played. Additionally, all of the Audio pins are now mapped to their respective inputs from the control board i.e. sending a 1 to the shift register will cause Audio pin 1 to play.
Elijah Shumway
Improved the frequency checking portion of the insect detection algorithm. The code now more effectively measures the ending frequency of the mating call to the initial frequency. This makes the algorithm more accurate and has lowered the amount of false positives generated.
Elijah Shumway
Discovered a bug in the code that was causing false positives when running detection. Fixed the bug and adjusted detection parameters to be more effective at only detecting the insect in question
Elijah Shumway
After the implementation of the SSM2166 audio conditioner IC, the sensitivity of the contact mic was greatly increased, allowing for the detection of extremely quiet sources of vibrations. This increased sensitivity allowed the system to be able to accurately detect the insect's mating call created through a vibration motor. Further testing needs to be conducted in order to increase the accuracy and efficiency of the system towards detecting insects, but it has been successful in detecting an insect mating call through a substrate.
Elijah Shumway
A component to better connect the contact mic to a substrate has been modeled and printed. This component will allow the user to attach a contact mic securely to various widths of substrates and more effectively measure vibrations across the substrate.
Elijah Shumway
The Pied Piper has gone from components on a bread board, to a condensed PCB that houses all of the hardware for the system. An audio input conditioner IC with an integrated amplifier has been installed on the PCB in order to increase the clarity of any incoming vibrations, hopefully allowing the Pied Piper to more effectively recognize insects on a substrate. Additionally, 3D printed parts to house the vibration motor and the contact mics have been modeled and printed. These parts allow for a solid contact between the components and the substrate, allowing for a stronger signal to be detected through the substrate.
Elijah Shumway
The vibration motor enclosure has been modeled and printed. This piece allows for solid contact between the vibration motor and the substrate, allowing for a stronger signal to be applied to the substrate
Elijah Shumway
The custom PCB for housing the Pied Piper microcontrollers and ICs has been designed and ordered. Further testing into improving the system's sensitivity will be conducted on the PCB's arrival
Elijah Shumway
The detection algorithm can now detect the presence of insects in half the time as before, with no loss of accuracy
Elijah Shumway