I think your site is interesting from a technical perspective, but dangerous from a human perspective.
To be more explicit, I feel like the framing isn’t great for humanity. It turns into “I proclaim X, so let me find a source” instead of “I wonder about X, let me look for authoritative information.”
The site spits out one or two links (in my case always just one), and then it’s on the user to click through and parse the paper. That feels incomplete. A Perplexity-style interface would be more useful, where an LLM reads the abstract (at minimum) and explains the nuance.
Take one example on the homepage: “Meat reduces cancer risk.” The site points to a study. I read it. The study absolutely does not prove that meat reduces cancer risk. What it actually shows is a small, observational, and potentially confounded association between higher animal protein intake and slightly lower cancer mortality. The authors explicitly state it isn’t conclusive.
But in conversation, if someone says “meat reduces cancer risk,” I disagree, and they whip out this site—it suddenly looks like “science proves it.” Except it doesn’t. And let’s be honest: neither of us is going to sit down and read an eight-page study in the middle of a chat. I only did because (a) I find it interesting, and (b) I’m stuck on a plane with Viasat wifi speeds blocking real work.
Without diligence, the tool risks turning weak claims into “settled fact.” Further, as science and the scientific method are under continued assault in the free world, tools that oversimplify or misrepresent evidence risk deepening mistrust and reinforcing narratives that science is just a weapon to win arguments rather than a process to seek truth.
Hmm interesting. I agree with you 100%. It is funny because I made this as an aside for another project I'm working on, paperzilla.ai, which I made exactly to solve this problem: people claiming stuff by pointing at a paper (while not providing it). Citation Needed was tech-driven: when I found out I could find a paper backing up a snippet of text, I thought that was pretty cool. But you are right, it can be used to back up any claim. Thanks for your response!
That's really cool - you should merge the two sites!
I did feel a certain degree of trepidation commenting what I did because I didn't want to (and hate that armchair devs on this site) just hate on the product or expect you to have considered the global implications of a side project, so I'm glad it's an issue you're thinking about too! LMK if you've got something else to try out!
Your comment was not in the category of an armchair dev/hater, otherwise I wouldn't have replied :)
I'm working on a "pivot" of paperzilla, which is paper discovery. Basically you subscribe to a research topic and you get a daily list of possible interesting papers for you in summarized form. If you are a scientist or following new science, please let me know. I'd love to hear what your daily ritual is for finding new papers and what tools you use (if any). You can reach me at mark at paperzilla dot ai.
I think your site is interesting from a technical perspective, but dangerous from a human perspective.
To be more explicit, I feel like the framing isn’t great for humanity. It turns into “I proclaim X, so let me find a source” instead of “I wonder about X, let me look for authoritative information.”
The site spits out one or two links (in my case always just one), and then it’s on the user to click through and parse the paper. That feels incomplete. A Perplexity-style interface would be more useful, where an LLM reads the abstract (at minimum) and explains the nuance.
Take one example on the homepage: “Meat reduces cancer risk.” The site points to a study. I read it. The study absolutely does not prove that meat reduces cancer risk. What it actually shows is a small, observational, and potentially confounded association between higher animal protein intake and slightly lower cancer mortality. The authors explicitly state it isn’t conclusive.
But in conversation, if someone says “meat reduces cancer risk,” I disagree, and they whip out this site—it suddenly looks like “science proves it.” Except it doesn’t. And let’s be honest: neither of us is going to sit down and read an eight-page study in the middle of a chat. I only did because (a) I find it interesting, and (b) I’m stuck on a plane with Viasat wifi speeds blocking real work.
Without diligence, the tool risks turning weak claims into “settled fact.” Further, as science and the scientific method are under continued assault in the free world, tools that oversimplify or misrepresent evidence risk deepening mistrust and reinforcing narratives that science is just a weapon to win arguments rather than a process to seek truth.
Hmm interesting. I agree with you 100%. It is funny because I made this as an aside for another project I'm working on, paperzilla.ai, which I made exactly to solve this problem: people claiming stuff by pointing at a paper (while not providing it). Citation Needed was tech-driven: when I found out I could find a paper backing up a snippet of text, I thought that was pretty cool. But you are right, it can be used to back up any claim. Thanks for your response!
That's really cool - you should merge the two sites!
I did feel a certain degree of trepidation commenting what I did because I didn't want to (and hate that armchair devs on this site) just hate on the product or expect you to have considered the global implications of a side project, so I'm glad it's an issue you're thinking about too! LMK if you've got something else to try out!
Your comment was not in the category of an armchair dev/hater, otherwise I wouldn't have replied :)
I'm working on a "pivot" of paperzilla, which is paper discovery. Basically you subscribe to a research topic and you get a daily list of possible interesting papers for you in summarized form. If you are a scientist or following new science, please let me know. I'd love to hear what your daily ritual is for finding new papers and what tools you use (if any). You can reach me at mark at paperzilla dot ai.