{"id":89716,"date":"2025-07-23T09:00:00","date_gmt":"2025-07-23T16:00:00","guid":{"rendered":"https:\/\/github.blog\/?p=89716"},"modified":"2025-08-01T15:52:04","modified_gmt":"2025-08-01T22:52:04","slug":"solving-the-inference-problem-for-open-source-ai-projects-with-github-models","status":"publish","type":"post","link":"https:\/\/github.blog\/ai-and-ml\/llms\/solving-the-inference-problem-for-open-source-ai-projects-with-github-models\/","title":{"rendered":"Solving the inference problem for open source AI projects with GitHub Models"},"content":{"rendered":"\n

AI features can make an open source project shine. At least, until setup asks for a paid inference API key.  Requiring contributors or even casual users to bring their own large language model (LLM) key stops adoption in its tracks:<\/p>\n\n\n

\n
$ my-cool-ai-tool\nError: OPENAI_API_KEY not found<\/code><\/pre>\n<\/path><\/path><\/svg><\/path><\/svg><\/clipboard-copy><\/div>\n\n\n
Developers may not want to buy a paid plan just to try out your tool, and self hosting a model can be too heavy for laptops or GitHub Actions runners. <\/p>\n\n\n\n
GitHub Models solves that friction with a free, OpenAI-compatible inference API that every GitHub account can use with no new keys, consoles, or SDKs required. In this article, we’ll show you how to drop it into your project, run it in CI\/CD, and scale when your community takes off.<\/p>\n\n\n\n
Let’s jump in.<\/p>\n\n\n\n
The hidden cost of “just add AI”<\/h2>\n\n\n\n
AI features feel ubiquitous today, but getting them running locally is still a challenge for a few reasons:<\/p>\n\n\n\n