it runs a server in your terminal, and its ui is your modern web browser (Safari, Chrome, Firefox, glad to look into fixing bugs on others if you find and report them). currently, it’s supported for MacOS on Apple Silicon only because that’s what i test it on.
to get the most out of the offline models, you will need a relatively powerful computer for the time being.
i develop and test enough on a 96GB + M3 Ultra Mac Studio, which handles all four offline models with ease, as well as a 16GB + M2 MacBook Pro, which also produces meaningful outputs.
over time, we’ll add Linux and maybe Windows support, while the local inference power in more affordable machines will also increase.
the openrouter option can be used on a wider variety of Apple configurations, but that approach requires sacrificing some privacy as well as some money to cover provider API costs & openrouter fees. that said, electricity and powerful computers cost money too, and openrouter is very reasonably priced.
so at launch … perhaps not optimally, or not for “free.” the good news is, it’s a marathon.