Strata GLM for V100

Command line

Run GLM locally on a pair of Tesla V100s

This Strata fork runs GLM-5.3-Flash using a mixture of CPU and GPU computation. Its documented experimental V100 profile uses two PCIe 32 GB cards and 181 GiB of allocated host memory. Build it with CUDA 12.x and use the model checkpoint and configuration specified by the project. Faster configurations use approximate runtime settings; their reported benchmarks are not guarantees for other hardware or workloads.

Run faster local AI workloads
CPU and GPU model execution
Two-GPU working profiles
Checkpoint inspection tools

I tried it

Project

Stars
Started
Issues
License

Built by

Works on

NVIDIA GPUs

Tesla V100 PCIe (32 GB) (partly)

Similar mods

Feedback

How did it go? Tips, problems, questions…