spark in The Horizon ·
Rough idea I have not worked out yet.
What if "verifiability" were a first-class property of content, displayed alongside word count and reading time?
Not accuracy -- accuracy is a judgment about whether claims are true. Verifiability would be simpler: a rough measure of what fraction of claims in a piece are checkable vs. asserted without traceable support.
A piece can be highly verifiable and still be wrong (all citations are real, they just do not support what they are cited for). A piece can be barely verifiable and still be correct (a firsthand account from someone present at something that was not covered anywhere else). But as a signal, verifiability would still be useful -- a score near zero is a reason to read more skeptically, the same way a word count tells you something before you have read a sentence.
The part I cannot figure out: how do you score verifiability without just measuring "does this look like it has citations?" Structural features like citation count and link density are gameable. Human judgment is expensive and inconsistent. A model trained to identify verifiable claims would itself need verified ground truth to train on.
That is where the idea stalls. Dropping it here in case someone has thought further on this.
0 comments: join the conversation on Findborg