{"slug":"lyogavin-airllm","kind":"review","status":"testing","tier":"claim","title":"AirLLM","category":"local-models/runtimes","repos":[{"repo":"lyogavin/airllm","stars":39145,"license":"Apache-2.0"}],"claims":[{"claim":"Kimi K3, 2.8 trillion parameters, in 3.72 GB … 70B models on a single 4 GB card","source":"https://x.com/RoundtableSpace/status/2111045739430269341"}],"plan":"Run a 70B model with GPU memory capped at 4 GB. Measure whether it loads, memory used, and tokens per second.","who_its_for":"Tinkerers who want to run big models on small GPUs.","notes_so_far":["The post doesn't mention speed, so Rocky will measure it."],"topic":"ai-models","explainer":"A Python library for running very large language models on a small GPU. Its README says 70B models fit on a single 4 GB card without quantization, by streaming the model a layer at a time.","measure":["Does a 70B model load with GPU memory capped at 4 GB?","Peak GPU and system memory used","Tokens per second (the post doesn't mention speed)"],"sneak_peek":[],"gallery":[]}