Search results showing duplicates
Ticketclosed
Merriott Lyle Distribution: documents · Search results showing duplicates
Search results showed duplicate records when accounts had inconsistent name formatting; root cause was deduplication hash including spaces; manual cleanup offered while permanent fix developed.
What happened
What this says
Current to 12 Oct 26Search results showed duplicate records when accounts had inconsistent name formatting; root cause was deduplication hash including spaces; manual cleanup offered while permanent fix developed.
Problem: The branch team's spreadsheet import left trailing spaces in account names, but the Xero sync pulled fresh data without spaces. The deduplication logic couldn't match them as the same record. Handling: Connor Webb traced the root cause and offered immediate manual cleanup with a permanent fix to strip whitespace before hashing. Watch: This affects all customers with messy data imports and is high priority to address in code.
Details
Updated 9 Oct 26Trail
created · Tariq Saleem 19 Mar 2025 12:03We've got a problem with the brain search. When users search for an account by postcode, they're getting back the same record twice, sometimes three times. This is breaking the flow for the branch teams, especially when they're trying to pull up customer history. I ran a test on a few postcodes and it's consistent. Is this a data issue or something in the search engine?- public_reply · Connor Webb 19 Mar 2025 14:56Hi Tariq, thanks for reporting this. Duplicate results in search are usually a data quality issue at index time, but let me dig into it. Can you give me an example postcode that's reproducing the problem? I'll pull the records and see if there's a mapping or deduplication issue.
- public_reply · Tariq Saleem 21 Mar 2025 09:34Cheers. Tried it with a few but the clearest example is from that batch we onboarded last week from the branch import. When I search one of those postcodes, I see the same company pulled three times with slightly different formatting in the name field. The underlying account IDs are the same though, so it's not duplicate records in the CRM, it's how the brain is handling them.
- internal_note · Connor Webb 24 Mar 2025 11:45Found it. Branch imported accounts from a spreadsheet a few days ago with trailing spaces in the name field. Xero sync then pulled fresh data on the same accounts without the spaces. Index is matching both as separate records on the dedupe hash because the name field is part of the hash. Samir needs to rebuild the dedupe logic to strip whitespace before hashing. High priority, this will hit all customers with messy data imports.
- public_reply · Connor Webb 24 Mar 2025 15:25We've found the root cause. Your branch team imported accounts with some formatting inconsistencies, and the brain's search index didn't deduplicate them properly. We're working on a fix to handle this better. In the meantime, I can manually run a cleanup on that batch if you'd like, but the permanent fix will take a few days. Would tomorrow be soon enough for the manual cleanup?
- status_change · 11 Apr 2025 23:59