sushi

<p align="center"><img src="docs/assets/sushi-logo.png" alt="sushi" width="256"></p>

# SUSHI

A local AI server for low-bit, very high quality Sushi quants, using custom Sushi Trellis Quant (STQ) or EXL3-Affine hybrid converter. It is a detached fork of [ddalcu's mlx-serve](https://github.com/ddalcu/mlx-serve). Sushi runs stand-alone and also as a guest engine inside mlx-serve, Apple Silicon M1+ and macOS 26.2 or later are required.

## Model support list

Usage see [recommended launch commands and coding agent usage](docs/usage.md).
Which pack suits your Mac, and running a bigger one from the SSD: [Choosing a pack and streaming](docs/usage-streaming.md).

* [DeepSeek-V4.1-Flash-1017-Sushi-STQ-1.75bpw](https://huggingface.co/beamster/DeepSeek-V4.1-Flash-1017-Sushi-STQ-1.75bpw) (pack revision 2.0, requires 96 GB+ and sushi 1.5.0 or later, experts [streamed](#streaming), text and image input; on M5 Macs it generates on Sushi Cooler, sushi's own engine, and `--ds41-mlx-engine` or the model's settings page go back to MLX)
* [Qwen3.8-Flash-Next-Sushi-2bpw](https://huggingface.co/beamster/Qwen3.8-Flash-Next-Sushi-2bpw) (requires 48 GB+, or 32 GB [streamed](#streaming))
* [Qwen3.8-Flash-Next-Sushi-STQ-2.75bpw](https://huggingface.co/beamster/Qwen3.8-Flash-Next-Sushi-STQ-2.75bpw) (pack revision 2.0, requires 64 GB+ and sushi 1.5.1 or later)
* [Qwen3.8-Flash-Next-Sushi-4bpw](https://huggingface.co/beamster/Qwen3.8-Flash-Next-Sushi-4bpw) (requires 96 GB+, or 64 GB [streamed](#streaming))
* [GLM-5.3-Flash-Sushi-2bpw](https://huggingface.co/beamster/GLM-5.3-Flash-Sushi-2bpw) (requires 96 GB+, text, image and video input)
* [GLM-5.3-Flash-Sushi-2.4bpw](https://huggingface.co/beamster/GLM-5.3-Flash-Sushi-2.4bpw) (requires 128 GB, text, image and video input)

**Unsupported:** [MiMo-V2.6-Flash-Sushi-2.3bpw](https://huggingface.co/beamster/MiMo-V2.6-Flash-Sushi-2.3bpw) still
loads, but from 1.5.1 releases do not test, benchmark or tune it. 1.5.0 is the last release that supports it.

## Quality

<p align="center"><img src="docs/assets/dsv41-1.75bpw-kld-v1-to-v2.svg" alt="DeepSeek-V4.1-Flash KLD by category, pack revision 1.0 to 2.0, against oQ2.5e" width="100%"></p>

<p align="center"><img src="docs/assets/kld-chart.png" alt="KLD vs size" width="100%"></p>

* DeepSeek-V4.1-Flash-1017-Sushi-STQ-1.75bpw, pack revision 2.0, scores KLD 0.1576 (top-1 agreement 89.13%) on the
  eight-category reading, down from 0.1843 in revision 1.0 and below oQ2.5e's 0.1717 at 2.96 bits per weight
* MiMo-V2.6-Flash-Sushi-2.3bpw scores KLD 0.0860 (top-1 agreement 91.95%)
* GLM-5.3-Flash-Sushi-2.4bpw scores KLD 0.0742 (top-1 agreement 90.33%)
* GLM-5.3-Flash-Sushi-2bpw scores KLD 0.1037 (top-1 agreement 88.18%)

## Install

```bash
brew install beamivalice/tap/sushi
```

Update with `brew upgrade sushi`.

`sushibar` puts the server in the menu bar: the generation speed in tok/s as its title, and a menu with a live status card, Start/Stop and Load/Unload buttons (Start brings the serv