Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: enable use of TensorFlow XNNPACK delegate for accelerating infe… #303

Merged
merged 1 commit into from
Oct 20, 2024
Merged

feat: enable use of TensorFlow XNNPACK delegate for accelerating infe… #303

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions internal/birdnet/birdnet.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ import (

"github.com/tphakala/birdnet-go/internal/conf"
"github.com/tphakala/go-tflite"
"github.com/tphakala/go-tflite/delegates/xnnpack"
)

// Embedded TensorFlow Lite model data.
Expand Down Expand Up @@ -95,6 +96,10 @@ func (bn *BirdNET) initializeModel() error {

// Configure interpreter options.
options := tflite.NewInterpreterOptions()
// If OS is Linux and XNNPACK library is available, enable XNNPACK delegate
if runtime.GOOS == "linux" && CheckXNNPACKLibrary() && bn.Settings.BirdNET.UseXNNPACK {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Undefined function CheckXNNPACKLibrary()

The function CheckXNNPACKLibrary() is called but not defined or imported. This will result in a compile-time error due to the undefined function.

Consider adding an implementation for CheckXNNPACKLibrary():

+// CheckXNNPACKLibrary checks if the XNNPACK library is available on the system.
+func CheckXNNPACKLibrary() bool {
+    // Implement logic to check for the presence of the XNNPACK library.
+    // For example, check if the 'libXNNPACK.a' file exists in '/usr/lib'.
+    if _, err := os.Stat("/usr/lib/libXNNPACK.a"); err == nil {
+        return true
+    }
+    return false
+}

Committable suggestion was skipped due to low confidence.

options.AddDelegate(xnnpack.New(xnnpack.DelegateOptions{NumThreads: int32(threads)}))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Add error handling when creating the XNNPACK delegate

When creating the XNNPACK delegate with xnnpack.New(...), it's important to handle potential errors to prevent unexpected runtime failures.

Modify the code to handle errors during delegate creation:

-if runtime.GOOS == "linux" && CheckXNNPACKLibrary() && bn.Settings.BirdNET.UseXNNPACK {
-    options.AddDelegate(xnnpack.New(xnnpack.DelegateOptions{NumThreads: int32(threads)}))
+if runtime.GOOS == "linux" && CheckXNNPACKLibrary() && bn.Settings.BirdNET.UseXNNPACK {
+    delegate, err := xnnpack.New(xnnpack.DelegateOptions{NumThreads: int32(threads)})
+    if err != nil {
+        return fmt.Errorf("failed to create XNNPACK delegate: %w", err)
+    }
+    options.AddDelegate(delegate)

Committable suggestion was skipped due to low confidence.

}
options.SetNumThread(threads)
options.SetErrorReporter(func(msg string, user_data interface{}) {
fmt.Println(msg)
Expand Down
27 changes: 27 additions & 0 deletions internal/birdnet/birdnet_utils.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
package birdnet

import (
"os"
"path/filepath"
)

// CheckXNNPACKLibrary checks for the presence of libXNNPACK.a in typical Debian Linux library paths
func CheckXNNPACKLibrary() bool {
libraryPaths := []string{
"/usr/lib",
"/usr/local/lib",
"/lib",
"/lib/x86_64-linux-gnu",
"/usr/lib/x86_64-linux-gnu",
}

for _, path := range libraryPaths {
fullPath := filepath.Join(path, "libXNNPACK.a")
if _, err := os.Stat(fullPath); err == nil {
// XNNPACK library is present
return true
}
}

return false
}
1 change: 1 addition & 0 deletions internal/conf/config.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -193,6 +193,7 @@ type BirdNETConfig struct {
RangeFilter RangeFilterSettings // range filter settings
ModelPath string // path to external model file (empty for embedded)
LabelPath string // path to external label file (empty for embedded)
UseXNNPACK bool // true to use XNNPACK delegate for inference acceleration
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codebase verification

Implement validation logic for the UseXNNPACK field.

While the UseXNNPACK field is defined and a default value is likely set in defaults.go, there is no validation logic to ensure its correctness. To maintain the stability and reliability of inference acceleration, please:

  1. Add validation for UseXNNPACK in the appropriate configuration validation functions.
  2. Ensure that default values are correctly handled and that the application gracefully manages any misconfigurations related to this field.
🔗 Analysis chain

Verify default configuration and validation for the new field.

While the addition of the UseXNNPACK field should not break existing functionality, it's important to ensure that:

  1. A default value for UseXNNPACK is set in the setDefaultConfig() function (defined in defaults.go).
  2. Any necessary validation logic for this field is added to the ValidateSettings() function.

To confirm these points, please run the following commands:

If these checks don't yield results, consider updating the respective files to include appropriate default and validation logic for the new UseXNNPACK field.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Check for default configuration
grep -n "UseXNNPACK" defaults.go

# Check for validation logic
grep -n "UseXNNPACK" internal/conf/validation.go

Length of output: 187

Script:

#!/bin/bash
# Search for defaults.go
fd "defaults.go" .

# Search for validation.go
fd "validation.go" internal/conf/

Length of output: 80

Script:

#!/bin/bash
# Search for usage of UseXNNPACK across the entire codebase
rg "UseXNNPACK" --  --type go

Length of output: 118

Script:

#!/bin/bash
# Search for usage of UseXNNPACK across the entire codebase within Go files
rg "UseXNNPACK" --type=go

Length of output: 258

}

// RangeFilterSettings contains settings for the range filter
Expand Down
3 changes: 3 additions & 0 deletions internal/conf/config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,9 @@ birdnet:
rangefilter:
model: latest # model to use for range filter: "latest" or "legacy" for previous model
threshold: 0.01 # rangefilter species occurrence threshold
modelpath: "" # path to external model file (empty for embedded)
labelpath: "" # path to external label file (empty for embedded)
usexnnpack: false # true to use XNNPACK delegate for inference acceleration

# Realtime processing settings
realtime:
Expand Down
3 changes: 3 additions & 0 deletions internal/conf/defaults.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,9 @@ func setDefaultConfig() {
viper.SetDefault("birdnet.longitude", 0.000)
viper.SetDefault("birdnet.rangefilter.model", "latest")
viper.SetDefault("birdnet.rangefilter.threshold", 0.01)
viper.SetDefault("birdnet.modelpath", "")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codebase verification

birdnet.modelpath is not utilized in the backend code.

The birdnet.modelpath configuration is set with an empty default value in internal/conf/defaults.go and referenced in the frontend views/settings/mainSettings.html. However, there are no backend implementations that utilize this configuration. This may lead to scenarios where a model path is expected but not handled, potentially causing runtime issues.

  • Ensure that the backend code consumes birdnet.modelpath appropriately.
  • If the configuration is intended for future use, consider adding a TODO comment or implementing the necessary backend logic.
  • Alternatively, remove the configuration if it's not required.
🔗 Analysis chain

LGTM. Verify handling of empty model path.

The addition of birdnet.modelpath with an empty default value allows for flexible configuration. This aligns with the PR objectives.

Ensure that the code handling this configuration properly checks for an empty path before attempting to load the model. Run the following script to verify:

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Check for proper handling of empty model path

# Test: Search for usage of birdnet.modelpath configuration
rg -A 5 'viper\.Get.*birdnet\.modelpath'

Length of output: 3506

Script:

#!/bin/bash
# Description: Search for any references to birdnet.modelpath in the codebase

rg 'birdnet\.modelpath'

Length of output: 283

viper.SetDefault("birdnet.labelpath", "")
viper.SetDefault("birdnet.usexnnpack", false)

// Realtime configuration
viper.SetDefault("realtime.interval", 15)
Expand Down
Loading