"LLM Council"

November 23, 2025 @ 6:20 am · Filed by under Artificial intelligence

« previous post | next post »

Yesterday afternoon, Andrej Karpathy posted llm-council on github:

His description:

The idea of this repo is that instead of asking a question to your favorite LLM provider (e.g. OpenAI GPT 5.1, Google Gemini 3.0 Pro, Anthropic Claude Sonnet 4.5, xAI Grok 4, eg.c), you can group them into your "LLM Council". This repo is a simple, local web app that essentially looks like ChatGPT except it uses OpenRouter to send your query to multiple LLMs, it then asks them to review and rank each other's work, and finally a Chairman LLM produces the final response.

Though I have too many other things to do, I'm tempted to install it and see what the Council thinks of Grok's weird recent re-programming:

Incredible headline.

[image or embed]

— Tim Onion (@bencollins.bsky.social) November 20, 2025 at 5:01 PM

(…which apparently is real?…)

I may succumb to temptation, if someone else doesn't do it first…

Update — I still haven't had time to try this out, but there's now an LLM Council Community on X…

November 23, 2025 @ 6:20 am · Filed by under Artificial intelligence


4 Comments

  1. Philip Taylor said,

    November 23, 2025 @ 7:22 am

    Tried installing under Windows (11), all went well until

    cd frontend
    npm install
    cd ..

    whereupon was told :

    PS J:\llm-council-master> cd frontend
    PS J:\llm-council-master\frontend> npm install
    npm : The term 'npm' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the
    spelling of the name, or if a path was included, verify that the path is correct and try again.
    At line:1 char:1
    + npm install
    + ~~~
    + CategoryInfo : ObjectNotFound: (npm:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

    [(myl) Google will send you to "Downloading and installing Node.js and npm" …]

  2. Philip Taylor said,

    November 23, 2025 @ 9:08 am

    Thank you Mark. I was seemingly misled by the Setup instructions :

    1. Install Dependencies

    The project uses uv for project management.

    Backend:

    uv sync

    Frontend:

    cd frontend
    npm install
    cd ..

    If Node.js and npm are (as would seem to be the case) required, then they could usefully have been listed under Dependencies. Anyhow, that is neither your concern nor your problem, so I will follow the link that you kindly supplied and report back in the fullness of time …

  3. ycx said,

    November 24, 2025 @ 9:40 pm

    Bear Grylls in shambles

  4. xap said,

    December 5, 2025 @ 7:29 am

    PewDiePie already did this

RSS feed for comments on this post