Modal Blog

How Snorkel runs realistic agent simulations with Modal

Modal powers inference for the open-weight models that Snorkel is evaluating as well as GPU and CPU sandboxes for millions of simulations per month.

Launching a trillion-parameter model at Sakana AI

Introducing our Training API

Spindle is an open-source API for training and fine-tuning open models — deploy a server with a single command and run any Tinker-compatible training code against it.

Get started with LLM training on Modal

Reinforcement learning for LLMs is hard, so we built Modal Dojo — a library that gives you the building blocks, tuned defaults, and observability to get started with production-grade LLM training.

Serverless routing made sticky

Voice agents, multiplayer games, and other workloads can now build on Sticky Sessions. Achieve strong affinity without losing autoscaling. Here’s how we built this on Modal Servers.

NVIDIA Vera CPUs double throughput for Modal Sandboxes

We benchmarked NVIDIA's new Arm-based Vera CPU on agentic Sandbox workloads. Under heavy load, it completes far more tasks than the x86 and Graviton alternatives, without sacrificing latency.

How Legora runs 2,000 parallel agent evals on Modal Sandboxes

To test a legal agent that works for hours across thousands of documents, Legora boots its entire application, pinned to a specific commit, inside a separate Modal Sandbox for every eval run.

Building fast filesystems for VMs

How we made VM Sandbox filesystems fast by speaking in blocks instead of files.

Muse Spark Open is now available on Modal

Muse Spark Open, a coding-optimized multimodal model from Meta, is now available on Modal Auto Endpoints.

Modal crosses $1B in Annualized Run Rate and raises $750M Series D

Modal has crossed $1B in annualized run rate and raised a $750M Series D led by Accel to build new infrastructure for AI applications. We can't think of a better time to be building software.

Toward RL on mixed-precision, mixed-accelerator samplers

We trained a model on agentic coding tasks using fully asynchronous RL and mixed-precision, mixed-accelerator samplers. With off-policy correction, training stayed stable through convergence, on par with homogeneous samplers, at a lower cost.

Endpoint Candidates: evaluate and promote inference endpoints

Test and promote inference endpoints to evaluate quality, latency, throughput, and cost.

TypeSafe AI scales Jev to trillions of tokens in three days on Modal

TypeSafe AI launched Jev, a new class of AI model built for automation, and scaled to trillions of tokens over a weekend on Modal.

Modal Clusters are generally available

Multi-node GPU clusters with RDMA, gang scheduled from Modal's shared capacity pool and billed by the second, behind a single decorator.

VM Sandboxes: Full computers for agents

VM Sandboxes are built for those who need to give their agents the power of a full computer.

Runtime Roundup: VM Sandboxes, Multi-node clusters, and more

Everything announced live at Modal's inaugural conference, Runtime.

Sidecars: A low-latency trust boundary for Sandboxes

Sidecars are isolated containers that run alongside your main Sandbox on the same host.

Quail: Speeding up AI-SQL by jointly optimizing query planner and inference engine

Maximizing perf on AI-SQL queries with the KV-optimal left-deep join

How to serve trillions of tokens for trillion-parameter coding agents

Learn how we optimized performance and efficiency serving the workload that is changing software engineering forever — and you can too.

Product updates: Sandbox Sidecars, new models, a refreshed dashboard, and more

Recent product updates from Modal and news from around the community.

One compute platform for end-to-end training at Markov Robotics

Markov Robotics is teaching robots to tackle complex tasks with fewer than ten hours of robot data, using Modal to power video processing, world model training, and real-time inference—and run ten times more experiments.

Modal is expanding in Europe with our new London office

Modal is expanding, and hiring on all fronts across Europe.

How Botika runs full-stack generative AI on Modal

Botika is building the next generation of tooling for agentic e-commerce brands, backed by a fleet of custom models trained and served on Modal.

Qwen3.8-2.4T-A95B now available on Modal

Qwen3.8-2.4T-A95B by Alibaba, with a 1M token context window, is now available via Modal Auto Endpoints.

Bringing serverless functions closer to the speed of wire

Modal’s Function Call data path is now >50ms faster. Our new routing layer is geographically distributed, so you can further reduce your network overhead.

A note on the Hugging Face agent incident

Hugging Face published a technical timeline of a recent agent intrusion. Modal's platform and isolation were not compromised in this incident.

Kimi K3 by Moonshot now available on Modal

Kimi K3, a 2.8 trillion parameter multimodal model by Moonshot, along with a custom-trained DFlash speculator, is now available on Modal.

Devin Outposts on Modal

Devin, built by Cognition, is an AI software engineer: it plans, writes, tests, and ships code semi-autonomously. With Outposts, Devin can now run its work in Modal sandboxes.

Scaling to 1 million concurrent sandboxes in seconds

How (and why) we built a scheduling system that can scale to 1 million concurrent sandboxes (per workspace) in seconds.

Inkling by Thinking Machines now available on Modal

Inkling, a general-purpose multimodal model by Thinking Machines, along with a custom trained DFlash speculator, is now available on Modal.

Modal is a computer

What is Modal? A machine that runs programs of arithmetic & logic operations on information.

How to price serverless GPUs

To compare rates for serverless and reserved GPUs, look at your application's peak-to-average ratio.

Multi-token Residual Prediction

One tiny module, two ways to win on Diffusion LMs

Anthropic integration with Modal brings scalable compute to Claude Science

Announcing our integration with Claude Science, bringing Modal's elastic compute to researchers when they need it.

Modal's serverless Servers

A deep dive inside our new ultra-low-latency primitive.

Achieve state-of-the-art inference latencies with speculative decoding

How Modal and Decagon worked together to cut inference latency - and you can too.

Introducing Modal Auto Endpoints: Optimized inference you actually own

LLM inference at SotA speeds and Modal quality, now available to everyone.

Unpacking sandbox startup latency: why started ≠ ready

Building performant sandbox systems goes way beyond the initial container boot. Here, we unpack what that means, and discuss some tools to help you manage the entire lifecycle.

Speculation Is All You Need

Why we're all-in on speculative decoding.

Product updates: VM Sandboxes, Lower latency routing, RBAC, and more

Recent product updates and news from around the community.

Making FlashAttention-4 faster for inference

What part of "dtype = 'fp8', num_splits = 0, pack_gqa = True, q_stage = 1, page_size = 1" do you not understand?

Reinforcement learning is an infrastructure problem

What we've seen helping teams run Reinforcement Learning at scale on Modal. Plus an open-source library to skip the scaffolding.

Role-Based Access Control for humans and agents

Introducing Role-Based Access Control for humans and agents, now available for all users on Teams and Enterprise plans.

Modal's Series C: Raising $355M at a $4.65B valuation

We've raised $355M at a $4.65B valuation to continue building the production cloud for AI.

Scaling reinforcement learning at Applied Compute

How Applied Compute trains custom agents with Reinforcement Learning for enterprises like DoorDash, Cognition, and Mercor on Modal.

Introducing Claude Managed Agents with Modal Sandboxes

How we achieved truly serverless GPUs

A deep dive on Modal's deep tech for fast boots.

Boosting multimodal inference performance by >10% with a single Python dictionary

If we've said it once, we've said it once per millisecond: never block the GPU.

Building an RL theorem-proving workflow on Modal

Learn how AE Studio used evolutionary algorithms on Modal to efficiently improve Lean proof generation.

Building with Modal and the OpenAI Agents SDK

Modal is an official sandbox provider for the OpenAI Agents SDK.

Autoscaling Autoresearch: Give your agents elastic GPUs on Modal

Autoresearch automates AI research. Modal automates AI infrastructure.

Butter is joining Modal

Butter, a San Francisco-based AI sandbox technology, is joining Modal.

Real-time inference for robots at Physical Intelligence

How Physical Intelligence runs remote, real-time, robotic inference on Modal.

Product Updates: RTX Pro 6000 Blackwell, Command K, Sandbox FS API and more

Product updates, community highlights, and upcoming events.

Runway chooses Modal to power real-time inference for Runway Characters

Modal is proud to power real-time inference for Runway Characters.

How Doppel eliminated ML infrastructure tax with Modal

How Modal helped the ML team at Doppel parallelize experimentation and scale inference.

Product Updates: Directory Snapshots, GLM-5, billing updates and more

A roundup of everything we shipped in February: Directory Snapshots for Sandboxes, a free GLM-5 endpoint, new billing API, and more.

Accelerating AI research that accelerates AI research

Learn why researchers at Scaling Intelligence, Hazy Research, and other top labs are choosing Modal.

Directory Snapshots: Resumable project state for Sandboxes

Introducing Directory Snapshots, a programatic way to snapshot a specific directory within a running Sandbox and mount it into another Sandbox later, independently of the base image and the rest of the filesystem.

How Ramp built a full context background coding agent on Modal

Try GLM-5.1, the new frontier of open intelligence, on Modal

GLM-5.1 establishes a new SotA for open models. Try it free today.

Product Updates: Modal in AWS & GCP marketplaces, Sandbox improvements, and more

All the latest news across product, content, community, and events this month.

Seamless computational bio at Chai Discovery

Learn how Chai Discovery moves seamlessly from ML experimentation to production antibody pipelines with Modal.

Keeping 20,000 GPUs healthy

How we do active and passive monitoring on hyperscalers and neoclouds.

Modal + Mistral 3: 10x faster cold starts with GPU snapshotting

We've partnered with Mistral to bring you Day 0 support for Mistral 3, with GPU-snapshot-optimized performance.

Agents need good developer experience too

Turns out, good devex for agents looks a lot like good devex for humans.

How Reducto improved enterprise-scale document processing latency by 3x

Learn how Reducto used GPU memory snapshotting and flexible autoscaling to build fast multi-model pipelines.

Host overhead is killing your inference efficiency

Never block the GPU.

How Decagon shipped real-time voice AI on Modal

How Decagon and Modal made real-time voice AI possible, combining fine-tuned small models with a re-engineered inference runtime for sub-second latency.

One-second voice-to-voice latency with Modal, Pipecat, and open models

How we built a real-time voice bot on Modal's distributed serverless platform.

Product updates: Updates to Volumes, JS and Go SDKs, and more

Welcome to another round of Modal Product Updates! Here's what's new this month.

Modal + Datalab: Deploy high-throughput document intelligence in <5 minutes

We've collaborated with Datalab, the creators of Marker and Surya, to make it faster than ever to deploy document intelligence workflows.

Announcing our $87M Series B

We’re excited to announce that we have raised more than $80M in a Series B round, led by Lux Capital. Our post-money valuation is $1.1B.

We reverse-engineered Flash Attention 4

Asynchrony, fast approximate exponents, and 10x more efficient softmax.

Justin Dignelli joins Modal as VP of Sales

We're excited to welcome Justin Dignelli to Modal. As VP of Sales, he will be leading our GTM efforts.

Build an AI coding platform that scales to millions of monthly sessions

Modal Sandboxes: impeccable vibes meet incredible scale.

Product updates: Modal Notebooks, sandbox idle timeouts, and more

Welcome to another round of Modal Product Updates! Here's what's new this month.

Inside Modal Notebooks: How we built a cloud GPU notebook that boots in seconds

Exploring the internals of our new product, a modern Jupyter notebook built for fast startup and real-time collaboration.

Introducing Notebooks

A collaborative environment for high-performance interactive computing on GPUs.

How Zencastr transcribed hundreds of years worth of audio in just a few days

Zencastr scaled up to 1,500 concurrent GPUs on Modal to process hundreds of years of podcast audio in just a few days. Today they run transcription, speaker detection, and audio enrichment for millions of podcast episodes on Modal, giving them cost efficiency, fast iteration, and zero DevOps overhead.

Product updates: GPU memory snapshots, notebooks, service tokens, and more

Welcome to another round of Modal Product Updates! Here's what's new this month.

GPU Memory Snapshots: Supercharging sub-second startup

Using GPU snapshots to enable sub-second container startup times.

What is an AI code sandbox?

AI code sandboxes are seeing an explosion of adoption as the volume of LLM-generated code in the world grows.

Transcribe speech 100x faster and 100x cheaper with open models

How we transcribed one week of audio in one minute for under one dollar.

Dollars per token considered harmful

Engineers of language model applications should think about requests, not tokens.

Product updates: Multi-node training clusters, B200 and H200s, and Client 1.0 release

Welcome to another round of Modal Product Updates! Here's what's new this month.

Jamsocket is joining Modal

Jamsocket, a backend platform for building sync engines, is joining Modal.

How Modal powered 250,000 Lovable app creations in a weekend

During a single weekend event, Lovable users built 250,000 new applications, all running in isolated development environments. Lovable used Modal to generate 1 million code sandboxes—with 20,000 running concurrently at peak—over just 48 hours.

How we used evals and inference-time compute scaling to generate beautiful QR codes that actually work

There's only one playbook for improving generative applications. Read about it here.

How Quora uses Modal to run thousands of Python sandboxes simultaneously

Quora is building Poe, a platform where anyone can deploy a public AI chatbot. Quora uses Modal Sandboxes at scale to safely run LLM-generated code in the context of user chats.

Run FLUX.1-dev three times faster

Price, performance, and control: pick three.

Introducing: Modal 1.0

We've released v1.0 of the Modal client, marking a new milestone of maturity and stability for our platform.

Introducing: B200s and H200s on Modal

We’re excited to be making Nvidia B200 and H200 GPUs available on Modal starting today!

Twirl is joining Modal

Twirl, a Stockholm-based data orchestration platform, is joining Modal.

Introducing Modal Batch: Process 1 million jobs with 1 line of code

Modal Batch is a new interface backed by a new durable queue system built specifically to make job processing easy, scalable, and fault-tolerant.

Modal's serverless KV store gets its limit raised to infinity

We've supercharged our Dicts to support new caching and locking workflows—oh, and unlimited items.

Linear programming for fun and profit

How we use an eighty-year-old algorithm to find arbitrages in the cloud market.

Modal SDKs for JavaScript and Go (alpha)

Today we're releasing lightweight client libraries for JavaScript and Go, making it easier to start sandboxes and call serverless functions — no Python required.

How Lemon Slice built real-time generative video with Modal and Daily

Modal + Daily + Pipecat is the best-in-class infra stack for real-time inference pipelines.

How sync. uses Modal to lipsync 100 hours of video a day

sync. is a research lab training foundational models to understand and manipulate humans in video. After outgrowing Google Colab, they partnered with Modal for efficient deployment, allowing rapid iteration and scaling to process over 100 hours of video daily.

Product updates: Running batch jobs with 1M inputs, ephemeral apps, and a new TensorRT-LLM example

Welcome to another round of Modal Product Updates! Here's what's new this month.

Our first brand campaign

Behind the scenes of updating our visual identity and launching our first-ever out-of-home campaign in San Francisco.

Product updates: Price drops, brand refresh, and image-to-video example

Welcome to another round of Modal Product Updates! Here's what's new this month.

'I paid for the whole GPU, I am going to use the whole GPU': A high-level guide to GPU utilization

A guide to maximizing the utilization of GPUs, from cloud allocations to FLOP/s.

We open sourced the GPU Glossary

GPU documentation for the people, now by the people.

Product updates: Logs v2, live container profiling, and region selection on all plans

Welcome to another round of Modal Product Updates! Here's what's new this month.

Memory snapshots: Checkpoint/restore for sub-second startup

Serializing container state to disk for aggressive cold start optimization.

Product updates: L40Ss, proxy auth tokens, and sandbox disk snapshotting

Welcome to another round of Modal Product Updates! Here's what's new this month.

Modal Sandboxes are generally available

Sandboxes are a new way to run code in Modal, with a focus on security and isolation.

Modal is SOC 2 Type II compliant

Modal is excited to announce its SOC 2 Type II certification.

Product updates: memory snapshotting, OIDC, async job queues & more

Welcome to another round of Modal Product Updates! Here's what's new this month.

Introducing: L40S GPUs on Modal

NVIDIA L40S GPUs available on Modal now!

What is LLM fine-tuning?

An intro to fine-tuning large language models in 2025

Dogfooding Modal: What we learned at our internal hackathon

Highlights from our 2024 internal hackathon, showcasing innovative projects built using Modal.

How a top tier European soccer team sped up their data processing and reduced costs by 50%

Find out how a top tier European soccer team uses Modal for computer vision on game data

WireGuard at Modal: Static IPs for serverless containers

Now your Modal containers around the world can have static outbound IPs. Featuring WireGuard, policy-based routing, and NAT.

Press release: Modal signs strategic collaboration agreement with AWS to deliver accelerated generative AI solutions

Announcing Modal's newest cloud partnership.

How OpenArt scaled their Gen AI art platform on hundreds of GPUs

Find out how OpenArt uses Modal to deploy highly customized ComfyUI workflows used by millions of customers

Product updates: Static IP proxies, Slack integration, live usage dashboard & more

Welcome to another round of Modal Product Updates! Here's what's new this month.

Tidbyt is joining Modal

Tidbyt, a NYC-based hardware manufacturer, is joining Modal

The future of AI needs more flexible GPU capacity

Why Modal is obsessed with serverless AI infrastructure

Hybrid search over California embeddings with Modal, MongoDB, and Clay

Build intelligent applications with Modal's serverless infrastructure and MongoDB Atlas's data platform.

How Contextual AI automated CI with Modal GPUs

Learn how Contextual AI accelerated their developer iteration speed by using Modal to run tests on GPUs.

Boost your throughput with dynamic batching

Learn how we used our new dynamic batching feature to improve throughput and reduce inference costs for the Whisper model with a single line of code!

Building a cost-effective analytics stack with Modal, dlt, and dbt

A step-by-step guide to building a scalable analytics stack using Modal, dlt, and dbt for efficient data loading, transformation, and deployment.

Product updates: Rollbacks, batching, sandbox tunnels & more

Welcome to another round of Modal Product Updates! Here's what's new this month.

Modal supports HIPAA compliance

You can now enter BAAs with Modal to run HIPAA-compliant workloads.

Inside the Modal code playground

How we built an in-browser code playground using Modal Sandboxes.

GPU prices are falling...

...and we're passing the savings to you. 15-30% price cuts on GPUs and CPUs.

Beat GPT-4o at Python by searching with 100 dumb LLaMAs

Scale up smaller open models with search and evaluation to match frontier capabilities.

Product updates: Datadog integration, lower function latency & more

Welcome to another round of Modal Product Updates! Here's what's new this month.

Competitive prompt engineering

Learn how Basis partnered with Modal to bring the spirit of competitive programming to prompt engineering.

Run GPU jobs from Airflow with Modal

Isolate your tasks with Modal containers while using Airflow for orchestration.

How to catch crypto miners using syscall signatures

See how Modal combats cryptomining abuse with syscall-based program analysis, to secure GPUs for legitimate users.

How Hunch supercharged AI workflows with Modal Sandboxes

Find out how Hunch uses Modal to run AI code even its users don't trust.

Create an infinite icon library by fine-tuning Stable Diffusion

How we fine-tuned a Stable Diffusion model on the Heroicons library to generate all the icons we could dream of.

Why Substack moved their AI and ML pipelines to Modal

Learn how Substack sped up their developer iteration cycles by moving ML training and deployment to Modal from AWS SageMaker.

Introducing: Region selection

You can now specify which cloud region you would like to run your Functions in.

Product updates: Cloud buckets, Okta SSO & more

Welcome to another round of Modal Product Updates! Here's what's new this month.

Beating proprietary models with a quick fine-tune

Fine-tune on just a few hundred examples and kick off your very own data flywheel.

Why you should move your ETL stack to Modal

Easily develop and deploy custom ETL jobs while saving 99% on sync costs.

Modal named to 2024 Enterprise Tech 30 list by Wing

Celebrating the best in enterprise tech.

How Ramp automated receipt processing with fine-tuned LLMs

Find out how Ramp uses Modal to customize open source LLMs to automate receipt processing.

Lambda on hard mode: Inside Modal's web infrastructure

In this post, we'll talk about how Modal handles real-time HTTP requests and WebSockets in serverless functions.

Introducing: WebSockets on Modal

Modal now supports WebSocket connections, enabling real-time, bidirectional data transfer between client and server.

How Suno shaved 4 months off their launch timeline with Modal

Find out how Suno uses Modal to scale inference and batch pre-processing to thousands of GPUs.

Product updates: WebSocket support, interactive commands & more

We've been busy in 2024 so far, bringing you WebSockets, interactive commands, H100s and more. Learn about what's new at Modal.

Introducing: H100s on Modal

We’re excited to be making Nvidia H100 GPUs available on Modal starting today!

Embedding English Wikipedia in under 15 minutes

Leverage Modal’s parallel batch jobs and in-house storage features to quickly generate embeddings for billions of tokens.

How to fine-tune an LLM on Modal

An operational guide to fine-tuning an LLM on any dataset in minutes (ft. CodeLlama, Llama 2, Mistral, and more)

Modal is now generally available

Modal offically launches today with no waitlist. And we also raised a Series A!

Press release: Modal Labs announces Series A financing round

Modal Labs Announces Series A Financing Round, Securing $16 Million Investment to Launch Cloud-Based Infrastructure Platform, Build Towards End-to-End Enterprise Data Stack

Modal is SOC2 compliant

Modal is excited to announce that it has successfully completed a System and Organization Controls (SOC) 2 Type 1 audit.