Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reactions Extraction from SQL Databas #79

Open
wants to merge 3 commits into
base: dev
Choose a base branch
from

Conversation

tlcameron3
Copy link

Original Description: I went ahead and grabbed the Reactions data for each message, added the respective lines to the query, and added that data onto the dictionary object in extract.py. Did not interface this updated dictionary object with the HTML output at all. Tested with JSON extraction, worked just fine.

Reactions are understood in the SQL database in two ways:
With an associated sender_row_jid: this will be seen in the message_add_on table, and ties relevant information as who performed the reaction. I wanted sender information associated with each reaction, so this worked just fine. The message_add_on also references the message line id, which is what I used to join to the main table. Meanwhile, the message_add_on_reaction simply contains the reaction (an emoji) and the timestamp, as well as a reference to the message_add_on table with its primary key. In this case, only reactions by other users in group chats are accounted for.

The second way the reaction sender is understood is through a "-1" field for the jid. This indicates a reaction associated with the WhatsApp account holder: either a reaction FROM the user to ANY other message (dm or group message), or a reaction to a message in dms with the user (so, the other dm participant reacting to the user's message). This meant doing additional logic in the information that I extracted using group_concatenate in the SQL query, and parsing the concatenated string accordingly in the associated logic. This also means both cases had to have different methods of extracting the reaction sender tag.

Let me know if you would want additional changes to be made in order for this to be merged with main. In my communication on WhatsApp, it became apparent that reactions (usually thumbs-up or hearts) would take the place of a confirmation message (where "okay" or "sounds good" usually sufficed), meaning some conversational information was lost when not taking reactions into account.

The changes account for three additional query lines, and one if-statement of logic, as well as adding "reactions" to the base class attributes. I would be welcome to assist in any other changes to make this branch compatible with your project.

P.S. I fixed the pull request to only include relevant files, previously it included my result files from testing the extraction.

Best,
tlc

@KnugiHK
Copy link
Owner

KnugiHK commented Jan 17, 2024

Thanks for your contribution again. I contacted GitHub support and they removed the previous PR. I will look into your commits later.

@KnugiHK KnugiHK self-requested a review January 17, 2024 12:19
@KnugiHK KnugiHK added the enhancement New feature or request label Jan 17, 2024
@KnugiHK KnugiHK changed the base branch from main to dev February 12, 2024 10:17
Copy link
Owner

@KnugiHK KnugiHK left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might not work in all case. One of my old WhatsApp database does not have tables with name message_add_on*.

@mattattackDM
Copy link

Any idea how emoji reactions are stored in the ios db? I've been searching for a while, but can't seem to find it.

@mattattackDM
Copy link

mattattackDM commented Apr 3, 2024

Any idea how emoji reactions are stored in the ios db? I've been searching for a while, but can't seem to find it.

Never mind, I found it. Reactions are stored as a blob inside the Zwamessageinfo table.

@rounakdatta
Copy link

Can we proceed with this PR?

@KnugiHK
Copy link
Owner

KnugiHK commented Jun 22, 2024

Can we proceed with this PR?

Unfortunately, it requires additional changes and further review to ensure reliable extraction of reactions.

Even if the PR is merged, reactions won't appear in the outputs because this PR only handles the extraction of reactions from database.

@KnugiHK KnugiHK linked an issue Sep 15, 2024 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[FEATURE] Possible to add emoji reaction information from messages?
4 participants