[$1000] Chat - Certain characters are considered as 6 characters within the message box

[lanitochka17]

If you haven’t already, check out our contributing guidelines for onboarding and email contributors@expensify.com to request to join our Slack channel!

---

Action Performed:

1. Insert a text with a length exceeding 15,000 characters
2. Type characters such as: " , ç , à , & , é
3. The total increments by 6 for each occurrence

Expected Result:

The overall count should increase by 1

Actual Result:

The overall count increased by 6

Workaround:

Unknown

Platforms:

Which of our officially supported platforms is this issue occurring on?

• Android / native
• Android / Chrome
• iOS / native
• iOS / Safari
• MacOS / Chrome / Safari
• MacOS / Desktop

Version Number: 1.3.57-5

Reproducible in staging?: Yes

Reproducible in production?: Yes

If this was caught during regression testing, add the test name, ID and link from TestRail:

Email or phone of affected tester (no customers):

Logs: https://stackoverflow.com/c/expensify/questions/4856

Notes/Photos/Videos: Any additional supporting documentation

6-char-Expensify.mp4

Recording.6026.mp4

Expensify/Expensify Issue URL:

Issue reported by: @hichamcc

Slack conversation: https://expensify.slack.com/archives/C049HHMV9SM/p1692470777922139

View all open jobs on GitHub

Upwork Automation - Do Not Edit

• Upwork Job URL: https://www.upwork.com/jobs/~01ef4e9f58ef6fe5dd
• Upwork Job ID: 1696338819616731136
• Last Price Increase: 2023-08-29

[melvin-bot]

Triggered auto assignment to @MitchExpensify (Bug), see https://stackoverflow.com/c/expensify/questions/14418 for more details.

[melvin-bot]

Bug0 Triage Checklist (Main S/O)

• This "bug" occurs on a supported platform (ensure Platforms in OP are ✅)
• This bug is not a duplicate report (check E/App issues and #expensify-bugs)
  • If it is, comment with a link to the original report, close the issue and add any novel details to the original issue instead
• This bug is reproducible using the reproduction steps in the OP. S/O
  • If the reproduction steps are clear and you're unable to reproduce the bug, check with the reporter and QA first, then close the issue.
  • If the reproduction steps aren't clear and you determine the correct steps, please update the OP.
• This issue is filled out as thoroughly and clearly as possible
  • Pay special attention to the title, results, platforms where the bug occurs, and if the bug happens on staging/production.
• I have reviewed and subscribed to the linked Slack conversation to ensure Slack/Github stay in sync

[akinwale]

Proposal

Please re-state the problem that we are trying to solve in this issue.

When calculating the total comment length, characters with diacritics are being counted as Unicode characters (6 chars in length).

What is the root cause of that problem?

In getCommentLength method implementation found in ReportUtils, the regex used to replace the comment is [^ -~] which replaces all characters greater than ASCII code 127 with a Unicode representation. This also converts characters with diacritics which causes a single Latin character such as ç, à or é to be counted as a Unicode character.

App/src/libs/ReportUtils.js

Lines 2886 to 2890 in d09a1b6

	function getCommentLength(textComment) {
	return getParsedComment(textComment)
	.replace(/[^ -~]/g, '\\u????')
	.trim().length;
	}

What changes do you think we should make in order to solve the problem?

Update the regex to exclude latin characters as well while calculating the comment length. This will ensure that characters with diacritics are counted as expected. Note that this will have to be checked with the backend implementation to confirm that it is working correctly.

return getParsedComment(textComment)
-        .replace(/[^ -~]/g, '\\u????')
+        .replace(/[^ -~\p{Script=Latin}]/gu, '\\u????')
       .trim().length;

The regex also has to be extended or updated to exclude other characters such as ", although this is not strictly an ASCII character. These characters will have to be determined, perhaps during the review phase after a PR is created.

What alternative solutions did you explore? (Optional)

If the backend counts characters with diacritics as 6 in length, then we can update the getParsedComment to show the correct "expected count" for presentation on the frontend instead. However, the logic should ideally match whatever the backend implementation is.

[melvin-bot]

Job added to Upwork: https://www.upwork.com/jobs/~01ef4e9f58ef6fe5dd

[melvin-bot]

Current assignee @MitchExpensify is eligible for the External assigner, not assigning anyone new.

[melvin-bot]

Triggered auto assignment to Contributor-plus team member for initial proposal review - @Ollyws (External)

[redstar504]

This was intentional due to the bug #13988 which was fixed by PR #14752.

[akinwale]

This was intentional due to the bug #13988 which was fixed by PR #14752.

Ah, this makes a lot of sense now. I was really curious as to why the length was being calculated this way in the first place. Thanks for the context.

[MitchExpensify]

Ah got it thanks @redstar504 ! Closing in that case