Today we're releasing RoastMyPost, a new experimental application for blog post evaluation using LLMs. Try it Here.

TLDR

RoastMyPost is a new QURI application that uses LLMs and code to evaluate blog posts and research documents.
It uses a variety of LLM evaluators. Most are narrow checks: Fact Check, Spell Check, Fallacy Check, Math Check, Link Check, Forecast Check, and others.
Optimized for EA & Rationalist content with direct import from EA Forum and LessWrong URLs. Other links use standard web fetching.
Works best for 200 - ~10,000 word documents with factual assertions and simple formatting. It can also do basic reviewing of Squiggle models. Longer documents and documents in LaTeX will experience slowdowns and errors.
Open source, free for reasonable use[1]. Public examples are here.
Experimentation encouraged! We're all figuring out how to best use these tools.
Overall, we're most interested in using RoastMyPost as an experiment for potential LLM document workflows. The tech is early now, but it's at a good point for experimentation.

A representative illustration

How It Works

Import a document. Submit markdown text or provide the URL of a publicly accessible post.
Select evaluators to run. A few are system-recommended. Others are [...]