Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature webarkit trackers and beyond -> PlanarTracker integration #16

Open
wants to merge 37 commits into
base: dev
Choose a base branch
from
Open

Feature webarkit trackers and beyond -> PlanarTracker integration #16

wants to merge 37 commits into from

Conversation

kalwalt
Copy link
Member

@kalwalt kalwalt commented May 1, 2023

I will discuss here changes and improves in the WebARKitTracker code.
This is the follow up of the development started on my fork see this PR kalwalt#4
First i need to solve issue webarkit/webarkit-testing#11 this is a must.
My findings: probably the issue arise because the unique_ptr share the same track method.
Second i think the abstract class is not needed to derive the other classes (WebARKitOrbTracker and WebARKitAkazeTracker classes) i can simplify the code implementing a new method setDetectorType to switch the detector (Orb or Akaze). Will be nice also to implemement a Pimpl pattern for the WebARKitTracker class.

kalwalt and others added 21 commits May 15, 2020 16:20
@kalwalt
restoring ARMulti
70eb898
@kalwalt
video.c for javascript5 and similar
0947a3a
@kalwalt
removing other not necessary files
c315ed5
@kalwalt
android.h is not necessary
6773be1
@kalwalt
removing AR/gsub_es.h, gsub_es2.h, gsub_lite.h
7946e7a
@kalwalt
removing Eden lib
c390f77
@kalwalt
removing gsub.h and gsubConfig.h
f04ec81
@kalwalt
Merge branch 'dev'
fcf13e2
@kalwalt
Merge branch 'fix-webarkitLog'
683d401
@kalwalt
merge dev with auto_ptr fix
1462dd4
@kalwalt
removing dynamic throw's see #11
0f4a704
@kalwalt
adding try catch to avoid dynamic throw #11
6074362
@kalwalt
removing dymanic throw from image.h & .cpp see #11
8f42e14
@kalwalt
Eigen 3.4.0 version
20c369f
@kalwalt
added fatal log errors
0472a3d
@kalwalt
updated Readme
07baf8b
@kalwalt
removing config.h
0eb8bc1
@kalwalt
WebARKitTrackers from
63d3f05 
kalwalt#4
@kalwalt
free() is not needed at all
1de18a2
@kalwalt
better processFrame functions
018c055
@kalwalt
move track method to WebARKitTracker class
5237407
@kalwalt kalwalt added enhancement New feature or request C/C++ code concerning the C/C++ code design and improvements Emscripten labels May 1, 2023
@kalwalt kalwalt self-assigned this May 1, 2023
@kalwalt
Copy link
Member Author

kalwalt commented May 5, 2023 •
edited
Loading

I have solved the issue #11 with these commits 5237407 and 323b6df
As mentioned in the header of the PR i added the pImpl pattern to WebARKitTracker class.
In the near future i will try to improve the code further.
For example: I'm using ORB to detect features corners from the the target image, but maybe we can calculate these with the OpenCV method goodFeaturesToTrack() and add an inner ROI to exclude points too close to the border? I'm looking into ArtoolkitX implementation https://github.com/artoolkitx/artoolkitx/blob/master/Source/ARX/OCVT/HarrisDetector.cpp and in specific at the FindCorners method https://github.com/artoolkitx/artoolkitx/blob/1ac3e04b8e0caf9449216182a41264d19879199b/Source/ARX/OCVT/HarrisDetector.cpp#L46C48-L60

@kalwalt
New HarrisDetector and WebARKItFeatureDetector
c876779 
classes
- code under testing phase, but no errors while compiling
@kalwalt
createFeatureMask method
831fada
@kalwalt
Copy link
Member Author

kalwalt commented May 10, 2023

I started to integrate the PlanarTracker code from artoolkitx into the WebARKitTracker class. If it works well on desktop and Mobile native devices, it will works also with Emscripten. There are no reason that it can not works. Probably theere will be differences in performances until we'll not add simd support during compilation of OpenCV libs and during the compilation of the final stage. But this this is another chapter of the development.

@kalwalt
adapted code from artoolkitx
f98b9c0 
- HomographyInfo, TrackableInfo, TrackedPoint, TrackingPointSelector,
- more configs
-  Points and GetHomograpyInliers
@kalwalt kalwalt mentioned this pull request May 10, 2023
Open
@kalwalt
SelectTrackablesPoints, RunOpticalFlow and
c9246e4 
UpdateTrackableHomography from artoolkitx
@kalwalt
other methods:
0abe7f4 
GetVerticesFromPoint  GetVerticesFromTopCorner
GetTemplateRoi IsRoiValidForFrame InflateRoi
@kalwalt
ProcessFrameData_w and other functions
0d02b23 
- code can be compiled but can not be used yet
@kalwalt
new loadARParam, IsTrackableVisible
045d4fe 
GetTrackablePose functions
@kalwalt
Copy link
Member Author

kalwalt commented May 12, 2023

I think will open another PR starting from commit 03b8f4e before applying code from PlanarTracker (artoolkitx). This last code is not stable and is not guaranteed.

@kalwalt kalwalt changed the title Feature webarkit trackers Feature webarkit trackers and beyond -> PlanarTracker integration May 12, 2023
kalwalt
kalwalt commented May 13, 2023
View reviewed changes
WebARKit/WebARKitTrackers/WebARKitOpticalTracking/WebARKitTracker.cpp Outdated
@@ -400,7 +406,8 @@ class WebARKitTracker::WebARKitTrackerImpl {
return featureMask;
}

bool CanDetectNewFeatures() { return (_currentlyTrackedMarkers < _maxNumberOfMarkersToTrack); }
//bool CanDetectNewFeatures() { return (_currentlyTrackedMarkers < _maxNumberOfMarkersToTrack); }
Copy link
Member Author

@kalwalt kalwalt May 13, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had to override this because _maxNumberOfMarkersToTrack result = 0 !! so the boolean comparison is resulting always false and the detection routine never start. _maxNumberOfMarkersToTrack is initialized to 1, but if i print is value is always 0, so there is a bug somewhere....

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

solved with commit 4e3ce3a

@kalwalt
some cout's for debugging and provide
6c3d0c6 
cv::Mat as reference instead
@kalwalt kalwalt force-pushed the feature-webarkit-trackers branch from f301840 to 6c3d0c6 Compare May 13, 2023 21:31
@kalwalt
Copy link
Member Author

kalwalt commented May 13, 2023

Now the code start to detect and track, i can see in the console the message New Marker detected and a message with the pose matrix.
But i have to find the reason of this bug #16 (comment) regarding _maxNumberOfMarkersToTrack

kalwalt added a commit to webarkit/webarkit-testing that referenced this pull request May 14, 2023
@kalwalt
fix for webarkit/WebARKitLib#16 (comment)
2c4d0ce
@kalwalt kalwalt mentioned this pull request May 14, 2023
Merged
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@kalwalt kalwalt

Labels
C/C++ code concerning the C/C++ code design and improvements Emscripten enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@kalwalt