WEKA's Val Bercovici: KV Cache, DeepSeek V4, HBF, SLC vs QLC NAND, CXL, NVLink, Tokenomics

AI inference infrastructure is shifting from training-centric models to memory-bound architectures as agent swarms drive explosive growth in context window requirements. While KV cache compression techniques like sliding window attention and quantization offer significant unit-level reductions, the net demand for memory continues to surge due to longer context windows and increased concurrency. High-performance networks now allow storage to operate at bandwidths exceeding traditional DRAM, enabling a tiered memory hierarchy where NAND flash serves as a critical component. Val Bercovici, Chief AI Officer at Weka, highlights that the future of software profitability lies in vertical integration, where SaaS companies may increasingly acquire "NeoCloud" infrastructure to manage the rising operational costs of token generation. This shift underscores a broader trend toward domain-specific, efficient inference solutions that bypass the limitations of general-purpose hardware.