Kronk

[!IMPORTANT] The Kronk provider for Fantasy is considered experimental! The API and behavior can change until we consider it stable.

Kronk is a Go package by ArdanLabs that uses yzma under the hood to provide hardware accelerated local inference with llama.cpp directly integrated into your applications. Kronk provides a high-level API that feels similar to using an OpenAI compatible API.

When using the Kronk provider in Fantasy, you only need to specify the model you want to use and it'll be automatically downloaded in your machine and used for inference.

To see which models are available for you, see the Kronk Catalog.

Examples on how to use it are available in examples/kronk.