How Can We Stream AI Chat Messages Like ChatGPT?

Note: This post is a translated version of an article originally published on my personal blog. You can read the original Korean post here. How Can We Stream AI Chat Messages Like ChatGPT? Can We Do It With Traditional HTTP? When using services like ChatGPT, Claude, or Gemini, you'll notice that the AI's response is printed out on the screen bit by bit. How exactly is this implemented? Fundamentally, web services operate on the HTTP protocol. This protocol works as unidirectional communication: the client sends a request to the server, and the server sends back a single response. However, what we want is for the server to send AI message tokens down to the client as soon as they are ready. The traditional request-response pair we are familiar with isn't quite cut out for this. Can We Use WebSockets? As an alternative, we could use WebSockets, but this isn't a great approach either. Here's why: Streaming messages doesn't actually require a bidirectional channel. We only need the server