Zengxiao (Henry) He

LLM Inference Optimization · Stanford M.S. EE · GPU Systems & Serving

Open to LLM inference & AI-infra roles

About

I’m an AI infrastructure engineer focused on LLM inference — making large models fast, cheap, and reliable to serve. I care about the systems layer where latency, throughput, and GPU cost are actually decided: serving engines, kernels, batching, and quantization.

My work spans LLM serving platforms, GPU kernel programming (Triton/CUDA), and performance benchmarking for modern inference workloads. At Alibaba Cloud I built backend components for an LLM inference platform; as a Stanford researcher I worked on efficient, hardware-inspired AI. Lately I’ve been writing FlashAttention-style fused kernels and building a from-scratch distillation-and-serving stack with paged-KV continuous batching.

I do my best work close to the metal — profiling, measuring, and squeezing tokens/sec out of real hardware. Outside of work, I’m usually playing fingerstyle guitar, on a tennis court, or learning performance driving.

Experience

Software Engineer Intern — LLM Inference Infrastructure

Alibaba Cloud

Jun 2023 — Sep 2023

Worked on an LLM inference platform for model serving and deployment. Built and maintained backend components for inference service orchestration and request routing in C++/Python/Java.

Software Engineer Intern — AI Platform

Oracle

Jun 2025 — Sep 2025

Built LLM-assisted automation for enterprise backport workflows on Oracle's supply-chain platform: LLM-based triage and root-cause generation over BugDB changes, workflow orchestration, and a self-hosted LLM assistant on OCI — cutting backport processing time by ~40%.

Research Assistant — Efficient AI

Stanford University — Prof. Tom Lee

Dec 2024 — Apr 2026

Research on efficient AI and hardware-inspired neural computation. Built Python/PyTorch simulation and evaluation pipelines for sparse architectures and their inference behavior.

Teaching Assistant (CS 295: Software Engineering)

Stanford University

Dec 2025 — Mar 2026

Teaching and mentoring students on building robust software systems alongside Prof. Sara Achour. Designing assignment workflows, reviewing LLM-generated code for correctness, and providing project feedback to students.

Co-Founder & Full-Stack Engineer

Glowia AI

Nov 2025 — Mar 2026

Built a conversational AI agent for MedSpa customer engagement — Instagram DM intake, appointment booking, deposits, and operator monitoring — over configurable LLM inference pipelines tuned for cost and latency.

Projects

Tessera — From-Scratch LLM Distillation & Serving

A from-scratch LLM systems stack built around teacher–student distillation and efficient serving — paged-KV continuous batching, speculative decoding, int8/AWQ/FP8 quantization, custom Triton/CUDA kernels, and a Rust (tokio/axum) gateway. 500+ stars on GitHub.

Triton LLM Inference Kernel Lab

A compact GPU kernel lab for LLM inference primitives in Triton and CUDA: row-wise softmax, tiled FP16 GEMM with FP32 accumulation, and a FlashAttention-style fused attention kernel — each validated against a PyTorch reference. Measured on H100: fused softmax at 2.8 TB/s, ~83% of HBM3 peak bandwidth.

Skills

PythonC++RustCUDATritonPyTorchFlashAttentionKV cacheQuantizationvLLMLiteLLMLLM servingBenchmarking

Writing

A Fused Softmax at 83% of H100 HBM Bandwidth: A Roofline Walkthrough

Jul 1, 2026

How a one-program-per-row Triton softmax gets within 17% of the memory roofline on H100 — the bandwidth math, the measured numbers, and the shapes where it honestly loses to torch.softmax.

Paged-KV + Continuous Batching From Scratch: What Building Tessera Taught Me About vLLM's Design

Jun 10, 2026

I rebuilt vLLM's two core ideas — a block-paged KV cache and iteration-level scheduling — in a from-scratch engine. Here's the design logic, the ownership contract that prevents KV-block leaks, and the preemption policy that fell out of it.

All writing →

Education

Stanford University

M.S. in Electrical Engineering — GenAI Inference Optimization

Sep 2024 — Jun 2026

GPA: 3.8/4.0. Teaching Assistant for CS 295 Software Engineering.

Central South University

B.Eng. in Software Engineering

Sep 2020 — Jun 2024

GPA: 3.9/4.0. Core Member, National Innovation & Entrepreneurship Base.