Creating a HTTP Server in Java
Java has a massive ecosystem of HTTP servers and frameworks which sit on top.
Spring is a popular framework that lets you program against it’s API and easily configure the underlying servlet container used ( Tomcat, Jetty, Undertow ).
Other frameworks such as Vertx and Play! are built on top of Netty — a low level asynchronous event-driven networking framework.
There is also the Java EE landscape which includes the likes of the JBoss, Wildfly and Weblogic application servers.
When you build a RESTFul service with one of these frameworks you work with the API and don’t think too much about what is going on under the hood.
In this blog post we’ll explore how you could go about building a functional API on top of a basic HTTP server by simply using Sockets in Java.
Specifically we’ll:
- use the Sockets API to handle HTTP requests
- implement a very basic example of decoding a HTTP GET Request
- expose a functional API to allow an end user to define routes and embed the server in their application
Hopefully this will help you develop a better understanding of what could be going on behind the scenes.
The goal here is not to implement the full specification of a HTTP server with a fully fledged API on top — I’m not even sure the Socket API is low level enough to implement the full spec, e.g. we’ll just look at a GET request as a learning exercise.
The API produced:
If we have a Java Gradle application with a build.gradle.kts file we can simply include a dependency on the new server project we’re going to create called ‘functional-server-library’.
implementation(project(":functional-server-library"))Our main application can then use this library as it will expose a Server class.
The server class will:
- take in the port number it should listen for incoming requests on
- allows routes to be added using the addRoute method
- start listening for requests when the start method is called
Therefore the end app will look something like:
public class App {
public static void main(String[] args) throws IOException {
Server myServer = new Server(8080);
myServer.addRoute(GET, "/testOne",
(req) -> new HttpResponse.Builder()
.setStatusCode(200)
.addHeader("Content-Type", "text/html")
.setEntity("<HTML> <P> Hello There... </P> </HTML>")
.build());
myServer.start();
}
}HTTP Protocol
Before we dive in, let’s take a quick sec and remind ourselves of the HTTP protocol message format.
The format of a HTTP request is text based and can be broken into the following parts:
The first line consists of the HTTP method, URI and Protocol version:
This is followed by:
- multiple lines of headers
- 1 empty line
- optional request body for POST / PUT requests
E.g.
A HTTP GET request message:
GET /testOne HTTP/1.1
Host: 127.0.0.1:8080
User-Agent: curl/7.84.0
Accept: */*A HTTP POST request message:
POST /testOne HTTP/1.1
Host: localhost:8080
User-Agent: insomnia/2022.6.0
Content-Type: text/plain
Accept: */*
Content-Length: 11
hello thereA HTTP Response is very similar:
The first line of the request
Overall:
STATUS LINE
HEADER
HEADER
...
BODYE.g.
HTTP/1.1 200 OK
Content-Type: text/plain; charset=UTF-8
Connection: Keep-Alive
Hello there...Note on HTTP Versions:
One feature HTTP 1.1 introduced was persistent connections which allows 1 connection to handle multiple http requests. This is more efficient as it eliminates the need for multiple TCP handshakes.
This functionality can be enabled by setting the HTTP Header Connection to the value keep-alive.
We’re going to skip this functionality below ( as well as many other parts relevant to the HTTP spec ).
Rough Sequence Diagram of server
The server can be broken down into the following events:
- Server is started and listens for connections on a certain port
- When a connection is received, use HttpHandler to handle connection ( input and output streams passed )
- Decode message using HttpDecoder
- Route message to correct endpoint
- Write response using HttpWriter
With that out of the way let’s get building!
1. Server class
- Define port
- Add endpoints / routes
- Handle lifecycle of Request / Response
public class Server {
private final Map<String, RequestRunner> routes;
private final ServerSocket socket;
private final Executor threadPool;
private HttpHandler handler;
public Server(int port) throws IOException {
routes = new HashMap<>();
threadPool = Executors.newFixedThreadPool(100);
socket = new ServerSocket(port);
}The threadPool created is hard coded at 100 so this will limit the number of requests that would be allowed to be handled concurrently ( I arbitrarily picked 100, ideally this would be configurable by the end user ).
Adding Routes:
Let’s add a method to add Routes / Endpoints to the application. To do this we will store the route in a Map<String, RequestRunner>.
public void addRoute(HttpMethod opCode, String route, RequestRunner runner) {
routes.put(opCode.name().concat(route), runner);
}The Map key will be a combination of the Opcode of the Request ( in our case GET ) and the URI.
The RequestRunner is a Java interface which only has 1 method which takes a HttpResponse as an argument and returns a HttpRequest object.
This allows it to be a functional interface where we can supply this using a Lambda.
E.g.
public interface RequestRunner {
HttpResponse run(HttpRequest request);
}HttpMethod is a simple enum:
public enum HttpMethod {
GET,
PUT,
POST,
PATCH
}Starting the server:
We’ll now add a start method which will wait for incoming requests and handle them:
public void start() throws IOException {
handler = new HttpHandler(routes);
while (true) {
Socket clientConnection = socket.accept();
handleConnection(clientConnection);
}
}socket.accept() is blocking so handleConnection will only be called when a client connects to the port defined.
We’ll use the HttpHandler Object created to handle the connection.
private void handleConnection(Socket clientConnection) {
try {
handler.handleConnection(clientConnection.getInputStream(), clientConnection.getOutputStream());
} catch (IOException ignored) {
}
}However, we don’t want this 1 request to block all other requests from executing, so we will wrap this functionality into a Runnable.
The request / response lifecycle of each request will therefore be handled by 1 thread ( a synchronous server ).
If we wanted to be able to handle 1 request across multiple threads then we would need to use the Java NIO lower level library ( which stands for new IO ).
The above therefore becomes:
/*
* Capture each Request / Response lifecycle in a thread
* executed on the threadPool.
*/
private void handleConnection(Socket clientConnection) {
Runnable httpRequestRunner = () -> {
try {
handler.handleConnection(clientConnection.getInputStream(), clientConnection.getOutputStream());
} catch (IOException ignored) {
}
};
threadPool.execute(httpRequestRunner);
}2. HttpHandler
- Decode HTTP Request
- Route request to the correct RequestRunner
- Write Response to output stream
/**
* Handle HTTP Request Response lifecycle.
*/
public class HttpHandler {
private final Map<String, RequestRunner> routes;
public HttpHandler(final Map<String, RequestRunner> routes) {
this.routes = routes;
}Handling a Request:
public void handleConnection(final InputStream inputStream, final OutputStream outputStream) throws IOException {
final BufferedWriter bufferedWriter = new BufferedWriter(new OutputStreamWriter(outputStream));
Optional<HttpRequest> request = HttpDecoder.decode(inputStream);
request.ifPresentOrElse((r) -> handleRequest(r, bufferedWriter), () -> handleInvalidRequest(bufferedWriter));
bufferedWriter.close();
inputStream.close();
}We decorate our OutputStream to a BufferedWriter which let’s us write text to a character-output stream.
Steps:
- Decode message into a HttpRequest object
- If present, handle
- Else, handle invalid request
- Close output and input streams
Invalid Request scenario:
Build a HTTP Response object with status code 400 and a message of “Bad Request”.
private void handleInvalidRequest(final BufferedWriter bufferedWriter) {
HttpResponse notFoundResponse = new HttpResponse.Builder().setStatusCode(400).setEntity("Bad Request...").build();
ResponseWriter.writeResponse(bufferedWriter, notFoundResponse);
}Valid Request scenario:
Get the route key from HttpRequest object ( uri path ), if present we extract the RequestRunner from the Map of routes, execute and write the response.
Otherwise we create a not found Response with a status code of 404.
private void handleRequest(final HttpRequest request, final BufferedWriter bufferedWriter) {
final String routeKey = request.getHttpMethod().name().concat(request.getUri().getRawPath());
if (routes.containsKey(routeKey)) {
ResponseWriter.writeResponse(bufferedWriter, routes.get(routeKey).run(request));
} else {
// Not found
ResponseWriter.writeResponse(bufferedWriter, new HttpResponse.Builder().setStatusCode(404).setEntity("Route Not Found...").build());
}
}3. HttpDecoder
The goal of this class is to decode the input stream into a Http Request object.
The decode method breaks this into two steps:
- Read all lines of text in the InputStream to a List<String>
- Parse the List<String> into a HttpRequest Object ( if valid )
/**
* HttpDecoder:
* InputStreamReader -> bytes to characters ( decoded with certain Charset ( ascii ) )
* BufferedReader -> character stream to text
*/
public class HttpDecoder {
public static Optional<HttpRequest> decode(final InputStream inputStream) {
return readMessage(inputStream).flatMap(HttpDecoder::buildRequest);
}Reading the message:
If no data is in the InputStream we return an empty Optional, otherwise the data is read into a char[] buffer created.
The Scanner object is then used to read the buffer line by line and append each to an Array list that is returned.
If any exception occurs, an empty Optional is returned.
private static Optional<List<String>> readMessage(final InputStream inputStream) {
try {
if (!(inputStream.available() > 0)) {
return Optional.empty();
}
final char[] inBuffer = new char[inputStream.available()];
final InputStreamReader inReader = new InputStreamReader(inputStream);
final int read = inReader.read(inBuffer);
List<String> message = new ArrayList<>();
try (Scanner sc = new Scanner(new String(inBuffer))) {
while (sc.hasNextLine()) {
String line = sc.nextLine();
message.add(line);
}
}
return Optional.of(message);
} catch (Exception ignored) {
return Optional.empty();
}
}Building the Http Request:
This method does some simple parsing to determine if the message is a valid HTTP Request.
If the message is empty or the first line does not contain three words than an empty Optional is returned.
The first line is verified by:
- Using the Enum HttpMethod to extract the HTTP method for the first word
- Using the java.net.URI class to verify the URI component
- Comparing the third word ( Protocol ) to “HTTP/1.1”
If none of these conditions are met ( Exceptions caught ) then an Empty Optional is returned.
The final parsing of request headers ( remember we’re only dealing with a GET Request scenario with no body ) in done within the addRequestHeaders method.
private static Optional<HttpRequest> buildRequest(List<String> message) {
if (message.isEmpty()) {
return Optional.empty();
}
String firstLine = message.get(0);
String[] httpInfo = firstLine.split(" ");
if (httpInfo.length != 3) {
return Optional.empty();
}
String protocolVersion = httpInfo[2];
if (!protocolVersion.equals("HTTP/1.1")) {
return Optional.empty();
}
try {
Builder requestBuilder = new Builder();
requestBuilder.setHttpMethod(HttpMethod.valueOf(httpInfo[0]));
requestBuilder.setUri(new URI(httpInfo[1]));
return Optional.of(addRequestHeaders(message, requestBuilder));
} catch (URISyntaxException | IllegalArgumentException e) {
return Optional.empty();
}
}public enum HttpMethod {
GET,
PUT,
POST,
PATCH
}Parsing the request headers:
We skip the first line in the message as we’ve already read this above, for each remaining line we check for the colon character ‘:’ and simply read what is before as the header name and what comes after as the header value.
At the end of the method we add the new Map<String, List<String>> object created as the request headers using the Builder object passed.
private static HttpRequest addRequestHeaders(final List<String> message, final Builder builder) {
final Map<String, List<String>> requestHeaders = new HashMap<>();
if (message.size() > 1) {
for (int i = 1; i < message.size(); i++) {
String header = message.get(i);
int colonIndex = header.indexOf(':');
if (! (colonIndex > 0 && header.length() > colonIndex + 1)) {
break;
}
String headerName = header.substring(0, colonIndex);
String headerValue = header.substring(colonIndex + 1);
requestHeaders.compute(headerName, (key, values) -> {
if (values != null) {
values.add(headerValue);
} else {
values = new ArrayList<>();
}
return values;
});
}
}
builder.setRequestHeaders(requestHeaders);
return builder.build();
}🎉 we’ve now decoded our input stream into a HttpRequest Object.
4. ResponseWriter
Going back to our HttpHandler we can see both the handleRequest and handleInvalidRequest use the static method writeResponse within the ResponseWriter class.
This is used to write the HttpResponse object to the output stream.
In this example we’re simply writing the body as text/plain but in reality this class could possibly use different implementations for writing the body depending on the type of data being wrote ( JSON / XML etc ).
Writer steps:
- Extract status code
- Extract status code description
- Build header Strings ( E.g. “Content-Length: 50” )
- Write first line ( protocol, status code, status code description )
- Write request headers in following lines
- Write empty line
- Write body ( if present )
/**
* Write a HTTPResponse to an outputstream
* @param outputStream - the outputstream
* @param response - the HTTPResponse
*/
public static void writeResponse(final BufferedWriter outputStream, final HttpResponse response) {
try {
final int statusCode = response.getStatusCode();
final String statusCodeMeaning = HttpStatusCode.STATUS_CODES.get(statusCode);
final List<String> responseHeaders = buildHeaderStrings(response.getResponseHeaders());
outputStream.write("HTTP/1.1 " + statusCode + " " + statusCodeMeaning + "\r\n");
for (String header : responseHeaders) {
outputStream.write(header);
}
final Optional<String> entityString = response.getEntity().flatMap(ResponseWriter::getResponseString);
if (entityString.isPresent()) {
final String encodedString = new String(entityString.get().getBytes(StandardCharsets.UTF_8), StandardCharsets.UTF_8);
outputStream.write("Content-Length: " + encodedString.getBytes().length + "\r\n");
outputStream.write("\r\n");
outputStream.write(encodedString);
} else {
outputStream.write("\r\n");
}
} catch (Exception ignored) {
}
}Extracting header strings:
private static List<String> buildHeaderStrings(final Map<String, List<String>> responseHeaders) {
final List<String> responseHeadersList = new ArrayList<>();
responseHeaders.forEach((name, values) -> {
final StringBuilder valuesCombined = new StringBuilder();
values.forEach(valuesCombined::append);
valuesCombined.append(";");
responseHeadersList.add(name + ": " + valuesCombined + "\r\n");
});
return responseHeadersList;
}Convert response body entity to String:
private static Optional<String> getResponseString(final Object entity) {
// Currently only supporting Strings
if (entity instanceof String) {
try {
return Optional.of(entity.toString());
} catch (Exception ignored) {
}
}
return Optional.empty();
}To finish let’s quickly look at the HttpRequest / HttpResponse Pojos and test our application:
HttpRequest:
package Sockets.pojos;
import Sockets.contract.HttpMethod;
import java.net.URI;
import java.util.List;
import java.util.Map;
public class HttpRequest {
private final HttpMethod httpMethod;
private final URI uri;
private final Map<String, List<String>> requestHeaders;
private HttpRequest(HttpMethod opCode,
URI uri,
Map<String, List<String>> requestHeaders) {
this.httpMethod = opCode;
this.uri = uri;
this.requestHeaders = requestHeaders;
}
public URI getUri() {
return uri;
}
public HttpMethod getHttpMethod() {
return httpMethod;
}
public Map<String, List<String>> getRequestHeaders() {
return requestHeaders;
}
public static class Builder {
private HttpMethod httpMethod;
private URI uri;
private Map<String, List<String>> requestHeaders;
public Builder() {
}
public void setHttpMethod(HttpMethod httpMethod) {
this.httpMethod = httpMethod;
}
public void setUri(URI uri) {
this.uri = uri;
}
public void setRequestHeaders(Map<String, List<String>> requestHeaders) {
this.requestHeaders = requestHeaders;
}
public HttpRequest build() {
return new HttpRequest(httpMethod, uri, requestHeaders);
}
}
}HttpResponse:
package Sockets.pojos;
import java.time.ZoneOffset;
import java.time.ZonedDateTime;
import java.time.format.DateTimeFormatter;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Optional;
public class HttpResponse {
private final Map<String, List<String>> responseHeaders;
private final int statusCode;
private final Optional<Object> entity;
/**
* Headers should contain the following:
* Date: < date >
* Server: < my server >
* Content-Type: text/plain, application/json etc...
* Content-Length: size of payload
*/
private HttpResponse(final Map<String, List<String>> responseHeaders, final int statusCode, final Optional<Object> entity) {
this.responseHeaders = responseHeaders;
this.statusCode = statusCode;
this.entity = entity;
}
public Map<String, List<String>> getResponseHeaders() {
return responseHeaders;
}
public int getStatusCode() {
return statusCode;
}
public Optional<Object> getEntity() {
return entity;
}
public static class Builder {
private final Map<String, List<String>> responseHeaders;
private int statusCode;
private Optional<Object> entity;
public Builder() {
// Create default headers - server etc
responseHeaders = new HashMap<>();
responseHeaders.put("Server", List.of("MyServer"));
responseHeaders.put("Date", List.of(DateTimeFormatter.RFC_1123_DATE_TIME.format(ZonedDateTime.now(ZoneOffset.UTC))));
entity = Optional.empty();
}
public Builder setStatusCode(final int statusCode) {
this.statusCode = statusCode;
return this;
}
public Builder addHeader(final String name, final String value) {
responseHeaders.put(name, List.of(value));
return this;
}
public Builder setEntity(final Object entity) {
if (entity != null) {
this.entity = Optional.of(entity);
}
return this;
}
public HttpResponse build() {
return new HttpResponse(responseHeaders, statusCode, entity);
}
}Testing!
If we boot up our App and generate a HTTP GET Request to the URI /testOne we should see the following response:
➜ sockets git:(master) ✗ curl http://127.0.0.1:8080/testOne
<HTML> <P> Hello There... </P> </HTML>% to a URI not defined:
➜ sockets git:(master) ✗ curl http://127.0.0.1:8080/testTwo
Route Not Found...% and finally an invalid HTTP Request:
➜ sockets git:(master) ✗ echo hi | nc 127.0.0.1 8080
HTTP/1.1 400 BAD_REQUEST
Server: MyServer;
Date: Mon, 9 Jan 2023 22:17:20 GMT;
Content-Length: 18
Invalid Request...% This has only scratched the surface of building a HTTP server but hopefully as a learning exercise it has been useful to gain a better understanding of some of the possible steps going on behind the scenes.
All code referenced can be found here: https://github.com/rjlfinn/java-http-server
