Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reflective mouse click #51

Closed
wants to merge 4 commits into from

Conversation

jaki729
Copy link

@jaki729 jaki729 commented Dec 1, 2023

Coordinate Translation: The feature begins by converting percentage-based coordinates received from user commands into precise pixel positions on the screen. This translation ensures accurate mouse movement.

Reflective Assessment: Upon reaching the calculated position, the system evaluates accuracy by potentially capturing a screenshot of the target area. This snapshot serves as a reference point for precise clicking.

Accuracy Thresholding: Implementing defined accuracy thresholds or visual recognition techniques, the system checks if the calculated position aligns accurately with the target. If below a set threshold, it dynamically readjusts the mouse closer for precision.

Precision Adjustment: To fine-tune accuracy, the system employs iterative adjustments to move the mouse closer to the intended target. It repeatedly repositions the cursor until it reaches a precise point for clicking.

Smart Click Execution: Once the system confirms the accuracy or has successfully readjusted for precision, it executes the click action. This approach emulates human-like interaction, significantly enhancing accuracy while allowing for adjustments to compensate for potential deviations.

@jaki729
Copy link
Author

jaki729 commented Dec 1, 2023

Reflective Mouse Click Mode:

Objective Enhancement: Introduces a 'reflect and correct' approach for click accuracy, mimicking human interaction for precise clicks.
Mode Functionality: Evaluates position accuracy before clicking, readjusting if necessary to ensure precision.
Activation Control: Offers an optional -accurate terminal flag, allowing users to toggle this mode as needed.
Accuracy Improvement: Aims to improve click precision by compensating for inaccuracies, enhancing the overall interaction accuracy.
Performance Consideration: Balancing performance gains against potential latency due to current model operations is crucial, given the increased time for accuracy evaluation.

@michaelhhogue
Copy link
Collaborator

@jaki729 I don't quite see the net value in reflective_mouse_click. It appears to simply refactor what click_at_percentage is already doing, but with the removal of the circular movement. Can you explain the advantages of reflective_mouse_click?

@michaelhhogue michaelhhogue added the question Further information is requested label Dec 2, 2023
@michaelhhogue
Copy link
Collaborator

Closing as #57 added reflective mouse clicks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants