Project 7~60 minutesGuided Practice

AI Chat

Interview Scenario: "We want to add an AI assistant to our product. Build a simple chat interface that calls OpenAI's API and streams the response."

What You'll Learn

Integrating with OpenAI's Chat Completions API
Streaming responses for real-time typewriter effect
Managing environment variables securely
Reading streams with JavaScript's Fetch API
Maintaining conversation history for context

You Need an OpenAI API Key

This project requires an OpenAI API key. If you don't have one, sign up at platform.openai.com. New accounts get free credits to experiment with. Never commit your API key to git!

Step 1: Environment Setup

First, install the OpenAI Python SDK and python-dotenv for environment variable management:

Terminal

1pip install openai python-dotenv

Create a .env file in your backend directory to store your API key:

backend/.env

1OPENAI_API_KEY=sk-your-api-key-here

Make sure to add .env to your .gitignore file:

.gitignore

1# Environment variables
2.env
3.env.local
4.env*.local
5
6# Never commit API keys!

Environment Variables are Essential

Interviewers will look for proper secret management. Never hardcode API keys! Using environment variables shows you understand security basics and can work with production deployments where secrets come from secure vaults.

Step 2: Streaming Backend with FastAPI

Now let's create the FastAPI backend that streams responses from OpenAI. The key is using StreamingResponse with an async generator:

backend/main.py

1from fastapi import FastAPI
2from fastapi.middleware.cors import CORSMiddleware
3from fastapi.responses import StreamingResponse
4from pydantic import BaseModel
5from openai import OpenAI
6from dotenv import load_dotenv
7import os
8
9load_dotenv()
10
11app = FastAPI()
12
13app.add_middleware(
14    CORSMiddleware,
15    allow_origins=["http://localhost:3000"],
16    allow_credentials=True,
17    allow_methods=["*"],
18    allow_headers=["*"],
19)
20
21client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))
22
23class Message(BaseModel):
24    role: str  # "user" or "assistant"
25    content: str
26
27class ChatRequest(BaseModel):
28    messages: list[Message]
29
30async def generate_stream(messages: list[dict]):
31    """Async generator that yields chunks from OpenAI"""
32    stream = client.chat.completions.create(
33        model="gpt-4o-mini",  # Fast and cheap for development
34        messages=messages,
35        stream=True,
36    )
37
38    for chunk in stream:
39        # Each chunk contains a delta with the next piece of text
40        if chunk.choices[0].delta.content is not None:
41            yield chunk.choices[0].delta.content
42
43@app.post("/api/chat")
44async def chat(request: ChatRequest):
45    """Stream AI response back to the client"""
46    # Convert Pydantic models to dicts for OpenAI
47    messages = [{"role": m.role, "content": m.content} for m in request.messages]
48
49    return StreamingResponse(
50        generate_stream(messages),
51        media_type="text/plain",
52    )

Why Stream?

Without streaming, users stare at a blank screen for 5-10 seconds waiting for the full response. Streaming shows text appearing word-by-word, making the AI feel responsive and alive. This is the standard UX pattern used by ChatGPT, Claude, and every modern AI interface.

Generator Functions Explained

The yield keyword makes generate_stream a generator function. Instead of returning all data at once, it "yields" one piece at a time. FastAPI's StreamingResponse sends each yielded chunk immediately to the client, rather than buffering the entire response.

Run the backend:

Terminal

1cd backend
2uvicorn main:app --reload --port 8000
3
4# Test with curl:
5curl -X POST http://localhost:8000/api/chat \
6  -H "Content-Type: application/json" \
7  -d '{"messages": [{"role": "user", "content": "Say hello!"}]}'

Step 3: React Chat Interface with Streaming

Now for the frontend. We need to read the stream using response.body.getReader()and update the UI as chunks arrive:

frontend/app/chat/page.tsx

1"use client"
2
3import { useState, useRef, useEffect } from "react"
4
5interface Message {
6  role: "user" | "assistant"
7  content: string
8}
9
10export default function AIChat() {
11  const [messages, setMessages] = useState<Message[]>([])
12  const [input, setInput] = useState("")
13  const [isStreaming, setIsStreaming] = useState(false)
14  const messagesEndRef = useRef<HTMLDivElement>(null)
15
16  // Auto-scroll to bottom when messages change
17  useEffect(() => {
18    messagesEndRef.current?.scrollIntoView({ behavior: "smooth" })
19  }, [messages])
20
21  const handleSubmit = async (e: React.FormEvent) => {
22    e.preventDefault()
23    if (!input.trim() || isStreaming) return
24
25    const userMessage: Message = { role: "user", content: input }
26    const newMessages = [...messages, userMessage]
27
28    // Add user message and clear input
29    setMessages(newMessages)
30    setInput("")
31    setIsStreaming(true)
32
33    // Add empty assistant message that we'll fill in
34    setMessages([...newMessages, { role: "assistant", content: "" }])
35
36    try {
37      const response = await fetch("http://localhost:8000/api/chat", {
38        method: "POST",
39        headers: { "Content-Type": "application/json" },
40        body: JSON.stringify({ messages: newMessages }),
41      })
42
43      if (!response.body) throw new Error("No response body")
44
45      // Read the stream
46      const reader = response.body.getReader()
47      const decoder = new TextDecoder()
48
49      while (true) {
50        const { done, value } = await reader.read()
51        if (done) break
52
53        // Decode the chunk and append to the last message
54        const chunk = decoder.decode(value)
55
56        setMessages((prev) => {
57          const updated = [...prev]
58          const lastMessage = updated[updated.length - 1]
59          lastMessage.content += chunk
60          return updated
61        })
62      }
63    } catch (error) {
64      console.error("Error:", error)
65      setMessages((prev) => {
66        const updated = [...prev]
67        updated[updated.length - 1].content = "Sorry, something went wrong."
68        return updated
69      })
70    } finally {
71      setIsStreaming(false)
72    }
73  }
74
75  return (
76    <main className="flex flex-col h-screen max-w-3xl mx-auto">
77      {/* Messages */}
78      <div className="flex-1 overflow-y-auto p-4 space-y-4">
79        {messages.length === 0 && (
80          <div className="text-center text-gray-500 mt-20">
81            <p className="text-xl mb-2">Welcome to AI Chat</p>
82            <p>Send a message to start the conversation.</p>
83          </div>
84        )}
85
86        {messages.map((message, index) => (
87          <div
88            key={index}
89            className={`flex ${message.role === "user" ? "justify-end" : "justify-start"}`}
90          >
91            <div
92              className={`max-w-[80%] p-4 rounded-2xl ${
93                message.role === "user"
94                  ? "bg-blue-500 text-white rounded-br-md"
95                  : "bg-gray-100 text-gray-800 rounded-bl-md"
96              }`}
97            >
98              <p className="whitespace-pre-wrap">
99                {message.content}
100                {/* Blinking cursor while streaming */}
101                {isStreaming &&
102                 index === messages.length - 1 &&
103                 message.role === "assistant" && (
104                  <span className="inline-block w-2 h-4 ml-1 bg-gray-400 animate-pulse" />
105                )}
106              </p>
107            </div>
108          </div>
109        ))}
110
111        <div ref={messagesEndRef} />
112      </div>
113
114      {/* Input Form */}
115      <form onSubmit={handleSubmit} className="p-4 border-t">
116        <div className="flex gap-2">
117          <input
118            type="text"
119            value={input}
120            onChange={(e) => setInput(e.target.value)}
121            placeholder="Type your message..."
122            disabled={isStreaming}
123            className="flex-1 p-3 border rounded-full focus:outline-none focus:ring-2
124                       focus:ring-blue-500 disabled:bg-gray-100"
125          />
126          <button
127            type="submit"
128            disabled={isStreaming || !input.trim()}
129            className="px-6 py-3 bg-blue-500 text-white rounded-full font-medium
130                       hover:bg-blue-600 disabled:bg-gray-300 disabled:cursor-not-allowed
131                       transition-colors"
132          >
133            {isStreaming ? "..." : "Send"}
134          </button>
135        </div>
136      </form>
137    </main>
138  )
139}

Reading Streams in JavaScript

The response.body.getReader() API gives you a ReadableStreamDefaultReader. You call reader.read() in a loop, which returns {done, value}. The value is a Uint8Array that you decode with TextDecoder. When done is true, the stream is complete.

Cursor Animation

The blinking cursor (animate-pulse) only shows on the last message while streaming. This visual cue tells users the AI is still "typing." It's a small detail that makes a big difference in perceived quality.

Conversation History Matters

Notice we send the full messages array to the backend, not just the latest message. LLMs are stateless - they don't remember previous turns. We must send the entire conversation history so the AI has context. This is how ChatGPT "remembers" what you talked about.

Exercises

What You Learned

LLM Integration

OpenAI API setup with SDK
Environment variable management
Streaming responses with generators
Token and context management

React Streaming UI

Reading fetch streams with getReader()
Progressive UI updates during streaming
Cursor animation for typing effect
Auto-scroll to latest content

Ready for more?

In the next project, you'll build a real-time chat application with WebSockets, where multiple users can chat together instantly.

Project 8: Real-time Chat