Skip to main content
BlogNewsletterRSS
Back to changelog

Stream long AI Gateway answers without a cutoff

Shipped by Brenden Ehlers

A slow reasoning model can think for a long time before it writes a word, and then write a lot of words. Streams now keep going for up to 15 minutes.

The per-request timeout applies only until the first byte arrives. After that, a stream stops if the provider goes quiet for 30 seconds or the total timeout runs out. If you set your own total timeout, it still applies.

Get started

  1. Send a request with stream: true through https://gateway.ngrok.ai.
  2. To change the limits, see streaming responses in the account settings guide.