HTTP: fetch and a small server

Most programs talk to other programs over HTTP: a client sends a request made of a method (GET, POST, …), a path, headers and sometimes a body; a server answers with a status code, headers and a body. This lesson shows both sides in Node. To keep every example runnable and deterministic, the clients will talk to a small server that the same script starts on your own machine. No example touches the internet.

A first request and a first server

Node’s built-in node:http module creates servers. A server is an event emitter, and the function you give to createServer is a listener for its "request" event, so it runs for each request. It receives the request (a Readable stream) and the response (a Writable stream). res.writeHead(status, headers) sets the status and headers, and res.end(body) sends the body and finishes.

server.listen(0, "127.0.0.1") asks the operating system for any free port, on the local machine only. The port number is only known afterwards, from server.address().port, and the example never prints it because it changes on every run.

import { createServer } from "node:http";

const server = createServer((req, res) => {
  res.writeHead(200, { "content-type": "text/plain" });
  res.end("hello from the server");
});
await new Promise((resolve) => server.listen(0, "127.0.0.1", resolve));
const url = `http://127.0.0.1:${server.address().port}/`;

fetch is a global function, based on the web standard and stable in Node since version 21. It returns a promise of a Response. Reading the body is a second asynchronous step, because the body may still be arriving: response.text(), response.json() and a few others return promises too.

const response = await fetch(url);
console.log(response.status, response.ok, response.statusText);
console.log(response.headers.get("content-type"));
console.log(await response.text());

server.close();
server.closeAllConnections();

200 true OK
text/plain
hello from the server

status is the number, ok is true for any status from 200 to 299, and headers has the Headers interface: get is not case-sensitive. At the end we close the server. fetch keeps connections open for reuse, so closeAllConnections() is needed, or the program would stay alive waiting for them. (The cancellation lesson used the same ending.)

A helper for the rest of the lesson

Starting and stopping a server takes the same few lines every time, so here they are as a reusable helper. It returns the base address and a stop function. Save it as hf_helper.mjs; the rest of the examples import it.

import { createServer } from "node:http";

export async function startServer(handler) {
  const server = createServer(handler);
  await new Promise((resolve) => server.listen(0, "127.0.0.1", resolve));
  const url = `http://127.0.0.1:${server.address().port}`;
  const stop = () =>
    new Promise((resolve) => {
      server.close(resolve);
      server.closeAllConnections();
    });
  return { url, stop };
}

server.close(callback) stops accepting new connections and calls back when all existing ones are finished, so closeAllConnections() follows to end the idle ones at once.

Routing and JSON

A server decides what to do from the request’s method and URL. req.url holds only the path and query (/users/1?x=2), so to take it apart, give it to the URL class with a made-up base. APIs usually speak JSON: set the content-type header to application/json and send JSON.stringify(data).

import { startServer } from "./hf_helper.mjs";

const users = new Map([
  ["1", { id: 1, name: "Ada" }],
  ["2", { id: 2, name: "Alan" }],
]);

function send(res, status, data) {
  res.writeHead(status, { "content-type": "application/json" });
  res.end(JSON.stringify(data));
}

const { url, stop } = await startServer((req, res) => {
  const { pathname } = new URL(req.url, "http://localhost");
  const match = pathname.match(/^\/users\/(\d+)$/);
  if (req.method === "GET" && match && users.has(match[1])) {
    return send(res, 200, users.get(match[1]));
  }
  send(res, 404, { error: "not found" });
});

The first rule of the handler is “answer every request”: the last line sends a 404 for anything that did not match. Now the client side.

const found = await fetch(`${url}/users/1`);
console.log(found.status, await found.json());

const missing = await fetch(`${url}/users/99`);
console.log(missing.status, missing.ok, await missing.json());
await stop();

200 { id: 1, name: 'Ada' }
404 false { error: 'not found' }

Look at the second request. The server answered 404, and fetch did not throw. This is the most common surprise: a promise from fetch is rejected only when no response arrived at all (no network, refused connection, a malformed URL). Any response, even a 500, counts as success for fetch. You must look at response.ok or response.status yourself. A small wrapper gives the “throw on HTTP errors” behaviour that most programs want.

import { startServer } from "./hf_helper.mjs";

async function getJson(url) {
  const response = await fetch(url);
  if (!response.ok) {
    throw new Error(`HTTP ${response.status} ${response.statusText}`);
  }
  return response.json();
}

const live = await startServer((req, res) => {
  res.writeHead(500);
  res.end("oops");
});
try {
  await getJson(`${live.url}/data`);
} catch (error) {
  console.log(error.message);
}
await live.stop();

HTTP 500 Internal Server Error

Real network failures are a different kind of error. The next example starts a server and stops it at once, so the address is certain to be closed, and then calls fetch.

const dead = await startServer(() => {});
await dead.stop();
try {
  await fetch(dead.url);
} catch (error) {
  console.log(error.name, error.message);
  console.log(error.cause.code);
}

TypeError fetch failed
ECONNREFUSED

The error is always a TypeError with the message fetch failed. The useful detail is in error.cause, which holds the system error code, here ECONNREFUSED (nothing is listening). This is the cause property you met in the async errors lesson. For timeouts and cancellation, pass signal: AbortSignal.timeout(ms) as in the cancellation lesson.

Sending data: POST

For a request with a body, give fetch an options object: the method, the headers and the body. A JSON body needs JSON.stringify and the matching content-type header. On the server side, the request is a Readable stream; the body arrives in chunks, which we collect with for await. setEncoding("utf8") makes the chunks strings and avoids cutting a multi-byte character in two.

The server below accepts POST /todos with a JSON body { "title": "..." }. It answers 201 Created with the new item, or 400 Bad Request if the body is not valid JSON or has no title.

import { startServer } from "./hf_helper.mjs";

async function readBody(req) {
  req.setEncoding("utf8");
  let text = "";
  for await (const chunk of req) text += chunk;
  return text;
}

The handler parses the body, validates it, and always answers.

const todos = [];
const { url, stop } = await startServer(async (req, res) => {
  const reply = (status, data) => {
    res.writeHead(status, { "content-type": "application/json" });
    res.end(JSON.stringify(data));
  };
  try {
    const body = JSON.parse(await readBody(req));
    if (typeof body.title !== "string") {
      return reply(400, { error: "title required" });
    }
    todos.push({ id: todos.length + 1, title: body.title });
    reply(201, todos.at(-1));
  } catch {
    reply(400, { error: "invalid JSON" });
  }
});

Now two requests: one good and one bad.

async function post(body) {
  const response = await fetch(`${url}/todos`, {
    method: "POST",
    headers: { "content-type": "application/json" },
    body,
  });
  console.log(response.status, await response.json());
}

await post(JSON.stringify({ title: "write lesson" }));
await post(JSON.stringify({ name: "no title" }));
await post("{ not json");
await stop();

201 { id: 1, title: 'write lesson' }
400 { error: 'title required' }
400 { error: 'invalid JSON' }

Three outcomes, three status codes. Choosing the right code is part of an API’s design:

• 200 OK, and 201 Created after a successful creation.
• 400 Bad Request when the client sent something invalid, 404 Not Found for an unknown path, 405 Method Not Allowed for a known path with the wrong method.
• 500 Internal Server Error when the fault is the server’s own. A handler that throws should not leave the request hanging, so answer with a 500.

The client should treat 4xx (the request was wrong, do not just repeat it) and 5xx (the server failed, a retry may help) differently, as the third exercise shows.

Streaming a response

A response body does not have to be read in one piece. response.body is a web stream (see the streams lesson), which is async iterable. This matters for large downloads and for servers that send data over time. res.write sends a chunk without ending the response.

import { startServer } from "./hf_helper.mjs";

const { url, stop } = await startServer((req, res) => {
  res.writeHead(200, { "content-type": "text/plain" });
  let n = 0;
  const timer = setInterval(() => {
    res.write(`line ${++n}\n`);
    if (n === 3) {
      clearInterval(timer);
      res.end();
    }
  }, 10);
});

The client decodes the bytes to text as they arrive.

const response = await fetch(url);
let text = "";
for await (const part of response.body.pipeThrough(new TextDecoderStream())) {
  text += part;
}
console.log(JSON.stringify(text));
await stop();

"line 1\nline 2\nline 3\n"

The server sent three chunks 10 ms apart, and the client could have acted on each one as it came. TextDecoderStream turns the bytes into text, taking care of characters that are split across chunks. We join the pieces and print the result, because how the network groups the bytes into chunks can vary.

Shutting down cleanly

Ctrl+C sends the signal SIGINT, and a container or process manager sends SIGTERM. With no listener, Node dies on the spot, possibly in the middle of a request. A listener on process (an emitter, too) can stop the server first. The example sends the signal to itself with process.kill, which works on Linux and macOS; Windows does not support signals and only emulates some of this.

import { startServer } from "./hf_helper.mjs";

const { url, stop } = await startServer((req, res) => res.end("hi"));
console.log(await (await fetch(url)).text());

process.once("SIGINT", async () => {
  await stop();
  console.log("server closed");
});
process.kill(process.pid, "SIGINT");

hi
server closed

Once the server is closed nothing keeps the event loop alive, so the program ends by itself.

WebSocket

HTTP is request and response: the client always asks first. When a server must push messages as they happen (a chat, a live score) the web uses a WebSocket, a long-lived two-way connection. Node has the standard WebSocket client as a global, stable since Node 22.4: new WebSocket("wss://..."), with message, open and close events and a send method. Node has no built-in WebSocket server; real programs use a package for that. For this reason there is no runnable WebSocket example here.

Common pitfalls

• Assuming fetch rejects on 404 or 500. It does not. Check response.ok.
• Not reading the body. Unread bodies keep connections busy. Call .text(), .json() or cancel the stream.
• Reading the body twice. response.json() after response.text() fails because the body is a stream that has been consumed. Use response.clone() if you need both.
• A handler that sometimes does not answer. Every code path must end with res.end, or the client waits forever.
• Forgetting the content-type header on a JSON request or response.
• Not closing the server. The script stays alive. Use close() and closeAllConnections().
• Hard-coding a port in examples and tests. Use listen(0) and read the port afterwards.
• No timeout. A fetch without a signal can wait for a very long time.

Exercises

Oslo: 3C
Cairo: 31C
no data for Paris

import { startServer } from "./hf_helper.mjs";

const temps = { Oslo: 3, Cairo: 31 };
const { url, stop } = await startServer((req, res) => {
  const city = new URL(req.url, "http://localhost").searchParams.get("city");
  if (!(city in temps)) {
    res.writeHead(404);
    return res.end();
  }
  res.writeHead(200, { "content-type": "application/json" });
  res.end(JSON.stringify({ city, tempC: temps[city] }));
});

async function describeWeather(base, city) {
  const target = new URL("/weather", base);
  target.searchParams.set("city", city);
  const response = await fetch(target);
  if (response.status === 404) return `no data for ${city}`;
  const { tempC } = await response.json();
  return `${city}: ${tempC}C`;
}

for (const city of ["Oslo", "Cairo", "Paris"]) {
  console.log(await describeWeather(url, city));
}
await stop();

Oslo: 3C
Cairo: 31C
no data for Paris

200 { sum: 5 }
400 { error: 'numbers required' }

import { startServer } from "./hf_helper.mjs";

const { url, stop } = await startServer(async (req, res) => {
  req.setEncoding("utf8");
  let text = "";
  for await (const chunk of req) text += chunk;
  const { a, b } = JSON.parse(text);
  const valid = typeof a === "number" && typeof b === "number";
  res.writeHead(valid ? 200 : 400, { "content-type": "application/json" });
  const answer = valid ? { sum: a + b } : { error: "numbers required" };
  res.end(JSON.stringify(answer));
});

for (const body of [{ a: 2, b: 3 }, { a: "2", b: 3 }]) {
  const response = await fetch(`${url}/sum`, {
    method: "POST",
    headers: { "content-type": "application/json" },
    body: JSON.stringify(body),
  });
  console.log(response.status, await response.json());
}
await stop();

200 { sum: 5 }
400 { error: 'numbers required' }

ready after 3 requests
giving up
client error 404

import { startServer } from "./hf_helper.mjs";

async function fetchRetry(url, attempts) {
  for (let attempt = 1; attempt <= attempts; attempt++) {
    const response = await fetch(url);
    if (response.ok) return response.text();
    if (response.status < 500) {
      throw new Error(`client error ${response.status}`);
    }
  }
  throw new Error("giving up");
}

function flaky(failures) {
  let requests = 0;
  return startServer((req, res) => {
    requests++;
    res.writeHead(requests <= failures ? 503 : 200);
    res.end("ready");
  }).then((server) => ({ ...server, count: () => requests }));
}

const first = await flaky(2);
console.log(await fetchRetry(first.url, 5), "after", first.count(), "requests");
await first.stop();

const second = await flaky(2);
await fetchRetry(second.url, 2).catch((e) => console.log(e.message));
await second.stop();

const gone = await startServer((req, res) => {
  res.writeHead(404);
  res.end();
});
await fetchRetry(gone.url, 3).catch((e) => console.log(e.message));
await gone.stop();

ready after 3 requests
giving up
client error 404

Summary

• fetch(url, options) returns a promise of a Response. Read the body with text() or json(), once.
• fetch rejects only when no response arrived (TypeError: fetch failed, with the system code in error.cause). Always check response.ok for HTTP errors.
• A POST needs method, headers (with content-type) and a body, usually JSON.stringify(...).
• node:http createServer runs a function for every request. Route by req.method and the path of new URL(req.url, base), read the request body with for await, and always finish with res.end.
• Use meaningful status codes: 200, 201, 400, 404, 500. Retry on 5xx, not on 4xx.
• response.body is a web stream. listen(0) picks a free port. Node has a WebSocket client, stable since 22.4, and no built-in WebSocket server.

Next: managing the packages your project depends on, with npm.