Intel D270 GPU AI Accelerator–Up to 480 GB LPDDR5X(intel.com)
293 points by hurricanepootis | 168 comments
Intel product page for its data center GPU D-series AI inference accelerators, including the D270 with up to 480 GB LPDDR5X memory.
#inference on Hacker News
10 stories tagged inference, newest first.
Turning GLM-5.3-Flash into a Jev-like decision model(privatemode.ai)
116 points by flxflx | 50 comments
Privatemode AI blog post describing how to prompt GLM-5.3-Flash so its first output token gives a decision in a single forward pass, benchmarked against Jev and Laya.
OpenAI Jalapeño: Better than Nvidia Blackwell(newsletter.semianalysis.com)
555 points by bmulholland | 351 comments
SemiAnalysis article analyzing OpenAI's self-designed Jalapeño inference ASIC, comparing its TCO and throughput per MW against Nvidia Blackwell and Rubin.
AMD acquires Taalas to boost inference performance by etching models in silicon(theregister.com)
902 points by itvision | 681 comments
The Register reports AMD has acquired AI chip startup Taalas, which hardwires model weights into silicon for fast inference, reaching up to 17,000 tokens per second.
The session you cannot take with you(earendil.com)
708 points by apitman | 203 comments
An Earendil engineering post arguing that inference API sessions are becoming non-portable, so users can no longer own or move their conversation transcripts between models.
OpenAI unveils its first custom chip, built by Broadcom(techcrunch.com)
792 points by jamdesk | 451 comments
TechCrunch article reporting that OpenAI unveiled its first custom AI inference chip, codenamed Jalapeno, built in partnership with Broadcom.
Flux 2 Klein pure C inference(github.com)
421 points by antirez | 135 comments
GitHub repository by antirez for flux2.c, a pure C inference implementation of the FLUX.2 Klein image generation model.
The maths you need to start understanding LLMs(gilesthomas.com)
532 points by gpjt | 106 comments
Blog post by Giles Thomas explaining the high-school-level maths (vectors, matrices, high-dimensional spaces) needed to understand LLM inference.
New LLM optimization technique slashes memory costs(venturebeat.com)
424 points by hochmartinez | 176 comments
VentureBeat article by Ben Dickson on a new LLM optimization technique that reduces memory costs by up to 75%.
Llama 3 implemented in pure NumPy(docs.likejazz.com)
404 points by orixilus | 48 comments
GitHub repository llama3.np, a pure NumPy implementation of the Llama 3 language model for inference.