Article · MarkTechPost · Saturday 7 November

Buoyant Releases Kinference: An Open Source Inference Control Plane for Kubernetes and vLLM

Buoyant has released Kinference, an Apache 2.0 inference control plane for vLLM on Kubernetes. It adds declarative routing policies, KV-cache-aware load balancing and per-tenant metrics, and cut median TTFT 95% in Buoyant’s own benchmark.

Read at marktechpost.com The opening of the article at marktechpost.com.

In this story

A tool or model opens in the Heidelberg AI Curriculum: what it does, what it costs, how to start. A repo opens on GitHub.

AINews covers three AI developments: Mistral Large 4 ('Le Chonk') shipped as an open-weights release built for agentic coding and cybersecurity, with 1.05T total and 52B active parameters, 1M context, text+image input, and pricing near $0.70/$2.10 per 1M input/output tokens; OpenAI now monitors all frontier RL runs, evaluations, and internal Codex use using activation probes, reviewer models over reasoning and tool use, and investigator agents; and Anthropic says Claude now produces a preliminary misuse assessment immediately after each response, which it says has surfaced misuse its classifiers were not designed to catch.

30 Oct 2026

ArticleMarkTechPost

Zyphra ZYFA: FP8 FlashAttention for AMD MI355X, Up to 1.76x Faster

Zyphra released ZYFA, open-source FP8 and BF16 prefill attention kernels for AMD Instinct MI355X. On real model activations, its FP8 kernels run 1.16-1.76x faster than AMD’s AITER, with 62-78% lower error at head dimension 128.

1 day ago

ArticleMarkTechPost2 sources

Thinking Machines Releases Inkling 2 Small, a 594B MoE With 17B Active

Thinking Machines Lab released Inkling 2 Small, a 594B-parameter MoE with 17B active parameters. It scores 31 on the Artificial Analysis Intelligence Index and runs on Tinker, with Apache 2.0 weights planned.

Repothinking-machines-lab/tinker-cookbook

2 days ago

ArticleMarkTechPost

Together AI Releases Together Compiler: Agents Make Inference Up to 2.6x Faster

Together AI released Together Compiler, an agent-driven loop that keeps re-optimizing the inference forward pass as traffic shifts. It beat the best InferenceX configurations by up to 2.6x TPS in under 24 hours per model.