Although all our coding so far has been in JavaScript or its close cousin typescript, not all websites are coded in this language. If you know other popular coding languages it will boost your capabilities as a dev. The benefits of languages over one another have much to do with situation and preference. Proponents of .NET will tell you that in comparison to javascript, the language was designed with much more intention, and while javascript devs are constantly installing new libraries to do anything .NET has in its built-in library most of the functionality needed to be a developer. They will also tell you that when you get to the point of maximizing the speed of your code .NET will make things easier. Many people are also fans of object-oriented programming but coming from functional programming it's unlikely you will initially feel that way.

Setting up .Net

Dot net projects can be made in VSCode just like JavaScript. In order to get started with this you will need to first install the dot net SDK https://dotnet.microsoft.com/en-us/download and the C# Dev kit https://marketplace.visualstudio.com/items?itemName=ms-dotnettools.csdevkit

Setting up project

Start by creating a new project and cd-ing into it.

You can use the following command to view a list of templates available for your project:

But what we want to build is a WebApi app. The command to create that template is this:

Personalizing your project

Open the new directory and navigate to the program file. This file is the entry point for execution and where servers and middleware can be configured. When you create a new project Microsoft helpfully includes some code to tell you the weather in this project. We can delete this. After removing the weather stuff the file should look like

Before we leave the program file to make an API controller we want to add 3 more lines to the program file.

1. builder.Services.AddControllers(); so the file knows we are making a controller

2. app.UseAuthorization(); which is middleware that is required to run on a route before it accesses the endpoint

3. app.MapControllers(); which attaches endpoints to our controllers.

the final file should look like this

Creating a controller

Inside your MyWebApi directory make a new directory called controllers. Inside this directory make a new file called MyController.cs

within the file add this code

you now have a very simple API built in .NET

Using Swagger to test controllers

Make sure your terminal is inside the API directory then use

You should get a link to visit but there is nothing there. The trick is to add /swagger onto the end of your URL. If everything has worked properly you should be taken to a web page where you can click around and test any routes from your controller give it a go. It should look something like this.

As a final note before you commit any of this work to GitHub .NET projects generate small dev files at a scary rate so you should make sure you set up a .gitignore a shortcut for making one that ignores all the files you will only need locally is

Applications often store information specific to individual users. It is important to only allow access to the correct people, to avoid potentially malicious actors stealing or deleting private data. The term authentication refers to the process of verifying the identity of a specific user; authorization refers to verifying that this user is allowed to do something.

Verifying users with passwords

The simplest way to verify a user is by asking them for a secret when they create an account. They can then provide this secret when they want access—since only they should know it you can trust that they are who they say they are.

Early versions of password protection were simple:

1. Ask user for a password on account creation

2. Store the password in database

3. (later) Ask user for a password again on login

Unfortunately storing passwords in "plaintext" like this is problematic. Anybody with access to the DB can see every user's password—this includes employees of the business and any hackers who manage to connect to the DB.

Leaking passwords like this is especially bad because most users re-use the same password for many sites. You may not be too worried about the security of your dog-rating app, but some of your users probably gave you their bank password.

Ideally we need a way to verify passwords without actually storing the password.

Hashing passwords

Hashing is when you use a mathematical process (algorithm) to convert a string into a different one. Hashes are:

• One-way: it should be impossible to reverse the process.

• Deterministic: hashing the same string always gives the same result.

• Unique: hashing a different string should never create the same result.

Here's a quick way to create a hash using Node:

There is no way to turn the "f52fb..." hash back into "hunter2", so it's safe to store in our DB. A different password will never create the same hash, so there won't be conflicts.

Here's the flow from above updated to use hashing:

1. Ask user for a password on account creation

2. Hash the password, then store the hash in database

3. (later) Ask user for a password again on login

Salting password hashes

There is still a security problem here: hashing the same password always creates the same output. This lets a hacker who has stolen your DB use something called a "rainbow table" to make finding passwords easier.

A rainbow table is a big list of pre-computed hashes for common passwords. This lets them quickly match pre-computed hashes to those in your DB to find out what password was used to create the hash. This is much faster than hashing each password before comparing, since good hashing algorithms are deliberately slow to prevent this type of brute-force attack.

We can avoid this problem by ensuring each hash is truly unique. If we add a random string to the password before hashing then the result will always be different. We must then store the salt in the DB alongside the password, as we will need it to recreate the hash when we verify a user.

1. Ask user for a password on account creation

2. Generate a random salt

3. Hash the password + salt, then store the hash and salt in database

It is now impossible for any of these hashes to be pre-computed, since the hacker would need to know the salt in advance..

Using BCrypt to hash passwords

Our sign up/log in process has gotten quite complex. There are many moving parts to get right, and any flaw could be exploited by a hacker. It is a good idea to rely on a popular battle-tested library to implement these features for us instead of trying to write them ourselves.

The npm package provides simple methods for hashing a password (with salt) and comparing a password to a hash. Let's see how we'd use it to hash a password:

The .hash method is designed to be slow, to make brute-force attacks harder. You control how slow it should be using the second argument (12 is a good compromise between speed and security). It returns a promise—you need to wait for this to resolve before you can access the hash.

The .compare method lets us compare the hash to passwords to see if they match:

This method returns a promise that resolves with a boolean telling you whether the password matches the hash.

Keeping users logged in

We can verify that a user's submitted password matches the one they previously signed up with, but this isn't enough for an authentication system. Users do not want to type their password in every time they load a new page or take an action. We need a way to persist our verification across many requests.

HTTP is a "stateless" protocol. This means each new request to your server is totally independent of any other. There is no way by default for a request to contain information from previous requests.

Cookies were introduced in 1994 as a way for web browsers to store information on behalf of the server. The response to one request can contain a cookie; the browser will store this cookie and automatically include it on all future requests to the same domain.

How cookies work

A cookie is just a standard HTTP header. They can be set by a server including the set-cookie header in a response. Here's an example HTTP response:

That set-cookie header tells the browser to store a cookie with a name of "userid" and a value of "1234". This cookie is then sent on all future requests to this domain via the cookie request header. Here's an example HTTP request:

The server can read the cookie header to retrieve the info it previously stored.

Cookie attributes

Cookies also support extra attributes to customise their behaviour. These can be set after the cookie value itself, like this:

Expiry

By default a cookie only lasts as long as the user is browsing. As soon as they close their tabs the cookie will be deleted by the browser. The server can specify an expiry time for the cookie. There are two ways to control this: Expires and Max-Age. Expires lets you set a specific date it should expire on; Max-Age lets you specify how many seconds from now the cookie should last.

Security

These options help make your cookies more secure:

• HttpOnly stops client-side JavaScript from accessing cookies. This can prevent malicious JS code from reading your cookies ("Cross-site Scripting" or XSS).

• Same-Site=Lax stops the cookie from being sent on requests made from other domains. Without this other sites can fake requests from a logged in user ("Cross-site Request Forgery" or CSRF).

Cookies in Node

Since cookies are just HTTP headers you can set them and read them in a Node server:

Working with the raw headers like this is awkward though. Everything is just a big string, so it's hard to create a cookie with the right options and even harder to extract the cookie value from the header. There could also be multiple cookies within one header.

Cookies in Express

Express provides helper methods for setting cookies. The response.cookie method lets you set a cookie by providing the name, value and any options:

You can also delete a cookie using the response.clearCookie method. This takes the name of the cookie to delete and sets the right header to remove it:

Express does not have a built-in way to read cookies. You need to install the cookie-parser middleware from npm for this. It works just like the built-in body-parsing middleware—it grabs the header, parses the string into an object, then attaches it to the request object for you.

Using cookies for authentication

Now we know how to use cookies to store info in the browser we can keep a user logged in. Once we've verified their password we can store a cookie that effectively says "this is oli". On subsequent requests the server would read the cookie to find out which user it came from. E.g. the cookie might look like this: userid=1.

Unfortunately this wouldn't be very secure. Anyone can send any HTTP headers they like to your server. You could use dev tools to edit your request to send userid=2 to pretend to be a totally different user.

We need a way to ensure our cookie hasn't been tampered with. Luckily cryptography has a solution for us: . This means combining a value with a secret in a one-way mathematical operation. This is called a "signature", and it can only be reproduced by someone with both the value and the secret.

If we send both the value and the signature in the cookie we can recalculate the signature on the server (using the secret) to make sure the cookie wasn't tampered with. E.g. if someone changed userid=1 to userid=2 it would break the signature.

Express' cookie-parser middleware supports signing cookies automatically. You just have to pass a secret in when you initialise the middleware, then set the signed option when you create a cookie:

Now when you read cookies they'll be available at request.signedCookies:

If the signature didn't match the cookie will not be present in the object.

Stateless authentication flow

Our current auth flow looks like this:

1. Ask user for a password on login

2. Read the hash from the DB

3. Check the hashes are the same with bcrypt.compare

This is known as "stateless" auth, because we store all the info we might need for authentication in the cookie. The server doesn't need to check the DB to know whether a request is authenticated.

This is convenient but has some downsides:

1. Cookies have a 4kb size limit, so you can't fit too much info in them.

2. The server cannot log users out. Any device with a valid cookie is "logged in".

3. The server cannot enforce expiry time, since users can edit their cookies in the browser.

Ideally we need a system that lets us keep track of who is logged in on the server, so we can revoke that access when necessary.

Session authentication flow

Instead of storing all the user info in the cookie, we can just store a random "session ID" that corresponds to info stored in our DB. Now the cookie itself has no power, the server decides whether the user is logged in based on the session info.

Here's a modified auth flow:

1. Ask user for a password on login

2. Read the hash from the DB

3. Check the hashes are the same with bcrypt.compare

This avoids all the problems with stateless auth:

1. We only store an ID in the cookie, so the 4kb limit doesn't matter

2. The server can log users out by deleting the corresponding row from the DB

3. The server can store session expiry dates in the DB to ensure an old cookie cannot be reused

Here's roughly how this might be implemented. First we need a table to store our session in:

For now we're just storing the user_id, which we can use to look up other user info later. We'll also store the expiry time so we can ensure users aren't logged in forever.

Then we need a function to generate a random session ID and insert the session:

We'll also need function to retrieve a session:

Now handlers can read the session cookie to find out which user (if any) made the request:

Since this logic will be repeated in any request that needs to check the session it would be worth abstracting into an Express middleware.

Authentication summary

Things got a little complicated, so let's recap our final authentication flow to make it clear:

Signing up

1. User submits their email and password

2. Hash the password with BCrypt

3. Insert a new user into the DB to store email and hash

Logging in

1. User submits their email and password

2. Retrieve the stored user where email matches

3. Compare submitted password with stored hash

Checking auth

1. Read the session ID from the signed cookie

2. Retrieve the stored session from DB

3. (optional) remove the session & cookie if expired

Why Dependency Injections

It is quite common to have an object that depends on another object to do its function, especially in an object-oriented environment like .NET. For instance, your constructor depends on your data context object. However, it can cause problems to always have to build dependencies first and then the objects that depend on them next. Also what if two classes need to access different versions of the dependency for example we might want to use the data context class both in a testing class and in the actual project but when using it in the testing context, we might want to set it up in such a way that we don't actually affect the database we could do this by adjusting the controller class but it would be easier if we could make that class initially more flexible.

The solution to these problems is dependency injections where rather than creating a dependency that a class depends on we give the class an interface to tell it what type of object to expect and then create a service that allows us to create dependencies as needed and inject them into the class.

Set up

Open the simple hello world project you made last week. Within MyWebApi (or whatever you named the project) add a models directory and create a Book.cs file within it that looks like this

We now need to install Microsoft EntityFrameworkCore to help set up a quick in-memory database. Like so

Then we make a data folder and inside make a BookContext.cs file that will look like so

This is moving towards a simple in-memory database containing two books with randomly generated IDs. First, we will need to update our Program.cs file to use them.

Start by installing the following package

Then update the Program.cs file to look like what follows.

This should give our BookContext the power to create in-memory databases and ensure they are created on app build.

Now let's update our controller to be a book controller that returns a list of books.

Give it a quick swagger test, and then let's look at a part of this code in more detail.

The constructor for this class (line 4) requires BookContext to function. This means these two objects are now tightly linked and we can't easily change one without changing the other.

To allow us more flexibility we are going to create something called an interface that we can feed in instead. An interface lets the constructor know what to expect but will not be as absolute as actually giving it the class.

The first step to do that is to go into our data directory and make a new file called IBookContext that looks like this.

This creates an interface and tells the class passed this that at run time it can expect something that contains a DbSet of books rather than the specific class passed to it before. Now we just need to tell all the other parts of our program to look for the interface rather than the object.

Let's start with the BookContext file itself.

If we make this change the BookContext can now inherit from both DbContext and IBookContext.

Now we want to update our Program.cs file so that when the context is built it knows it can take anything that matches the interface requirements

Finally, we do what we wanted to all along which is to make our book controller independent from the BookContext class by putting our interface in the constructor.

Besides making your work easier by having these two things looser it will also give you the ability to if you created an xUnit testing environment (like the one in the challenge repo), write code like this that substitutes fake data for the book context.

Note this part which would not be possible if the Book controller relied on a BookContext

Most applications need to persist data—that is keep it around for future use. This means it will be available even if your server process restarts. A database is a program designed for efficiently and robustly storing information, and making it accessible to your app.

SQLite is a relational database that is quite simple to get started with. It runs in the same process as your server, unlike other popular databases like PostgreSQL or MySQL. These run as a separate server that needs to be managed separately from your app. As we will see SQLite stores data in a single file, which makes it convenient to work with for simple apps.

Using SQLite

If you have the SQLite command-line program you can create and manipulate databases in your terminal. However since we're building a Node app we'll use a library to do this in JS. There are several to choose from, but the simplest is better-sqlite3.

Let's start a new Node project and set it up to use a database. First create a new directory and a package.json:

Then install better-sqlite3 from npm:

This will be added to the dependencies object in your package.json. Now you can use it to initialise a SQLite database. You'll need a JS file to do this—since there will be a few database related files create a new database directory for them all. Then create a database/db.js file where you can initialise the DB:

The better-sqlite3 library exports a constructor function that creates a new SQLite database and returns a with methods for talking to that DB. Try running your JS file now and you should see this object logged:

By default if we don't pass any arguments to new Database() it'll create an "in-memory" DB. This means the data won't be persisted—this is useful for testing as you can't permanently break anything. If you want to persist data you can pass the name of the file you want to use.

If you run this code it'll create the file if it doesn't exist, or re-use an existing one if it does. However hard-coding isn't the best idea—there are situations where we might want to use a different DB, like running tests. Instead we can use an environment variable to set the filename.

Now we can choose what DB file to use without changing the code. For example if you run this file with:

you should see a new file called db.sqlite appear at the root of your project.

Using prepared statements

Accessing data is a two-step process with better-sqlite3. For performance reasons all queries must first be "prepared" before they can be run. This way the library can re-use the same query over and over. You create a statement with the db.prepare method. This takes a SQL string and returns a object:

The statement object has several methods for running a query based on what the expected result is. Use .run when you don't need a result (e.g. for deleting a row), .get when you expect a single row, and .all when you want to get all rows matching the query.

In this case we're expecting a single result (the date), so you can use the .get method:

Run this again in your terminal and you should see an object logged with the current date:

Setting up your schema

Relational databases need a defined schema to tell them how to organise your data. This helps them structure your data effectively. A simple schema is a collection of CREATE TABLE statements that you run against your database to create the tables and columns it needs. You could write all this inside strings in a .js file, but it's nicer to use a separate .sql file.

Let's create a simple schema that will let us store tasks for a to-do list. Create a new file database/schema.sql:

We're creating a new table called tasks with three columns: id will be an automatically incrementing integer generated by the DB, content will be the text content of each task, and created_at will be an auto-generated timestamp.

Note the IF NOT EXISTS; a good schema should be —you should be able to run it against your DB multiple times without changing the result. Running our schema a second time would result in a "table already exists" error if we didn't have this.

Let's run this schema against our DB in JS. We need to read the .sql file contents, then pass them into the db.exec method, which is designed to run one-off queries containing multiple statements like this. Edit database/db.js:

You can check this worked by selecting from the built-in sqlite_schema table (which lists everything else in the DB):

You should see your new tasks table in the array:

Your database is now ready to use. We just need to make it available to other parts of our application by exporting the db object. Here's the full database/db.js file you need, with explanatory comments:

Now we can import the db object in our app, which will run this file, ensuring the database is created and the schema properly set-up.

Using our database

Let's implement some features for this app that use our new database. It can be helpful to write your data access as separate functions to your server routes, so you can focus on doing one thing at a time.

Create a new model folder, with a new tasks.js file inside. This is where we'll write functions that read and write tasks from our DB. Let's import our db object and write a function to insert a new task:

We only need to specify the content value, since the id and created_at columns will be generated by the DB. We are using a parameterised query to pass in the dynamic content value. The ? in the query will be replaced by the first argument passed in when we execute the prepared statement.

Let's use this function to insert a new query. Temporarily add some code that calls your new function, then query the DB to see what is in the tasks table:

You should see an array with one task logged:

Returning generated data

Our createTask function currently doesn't return anything. Since the id and created_at columns are generated by the DB it would be nice if we returned the newly created task. We can do this using the RETURNING SQL statement. Amend your query and function:

Note we must now use the .get method, since we expect the statement to return a single value. If you call this function you should receive the inserted task object:

Seeding example data

It's a bit awkward to have to manually call functions to insert data when we want to check our DB is working. It would be nice if we had a script to run that could "seed" the DB with some pre-defined example data.

Delete your existing db.sqlite file, so we don't have to worry about any data already inserted.

Let's write some SQL to insert example tasks. Create a database/seed.sql file:

We use a transaction to ensure all the inserts succeed, similar to in our schema. There's one new addition: the ON CONFLICT ensures that we can run this script multiple times without getting duplicate ID errors.

Now we need a JS script that reads this file and runs it against the DB. This will be very similar to schema.js. Create a database/seed.js file:

Now you can run this script with DB_FILE=db.sqlite node database/seed.js to insert example data. It's worth adding an npm script to your package.json to make this reusable for other team members in your project:

Now anyone cloning the project can run npm install then npm run seed to have everything up and running quickly. If you ever want to start over you can delete your DB file and re-run the seed script.

Amending the schema

A task app needs to track whether each task is completed. To do this our tasks table will need a complete column. There are two approaches to amending the schema. We won't be using the first, but it's here for completeness if you're curious.

The simpler way is just to delete our database, change the schema.sql, then regenerate the DB. This is fine while we're still working on a feature, but if we tried this on our production DB we'd lose all our users' data.

So you can just delete your db.sqlite file, then amend your schema.sql file:

You'll also need to amend the seed.sql file before running it again. It needs the complete column, otherwise you'll get an error like table tasks has 4 columns but 3 values were supplied.

Named SQL parameters

Now our schema has changed we need to update the createTask function, since we may want to create a new task that is already completed. This means the function needs to set both columns. We could use multiple parameters like this:

However this is error-prone as it relies on the order we pass the arguments. A safer way is to use named parameters by passing a JS object. This also makes the SQL query more readable:

When passed an object like createTask({ content: "stuff", complete: 1 }) the statement will map each key to the corresponding $param in the query.

Listing rows

Our app is going to need a way to read all the tasks from the DB. We need to write a new model function that selects all rows from the tasks table:

Run this function to check that it shows all the tasks you have inserted so far.

Formatting columns

Our created_at column is a full timestamp, with a date and time, which is not particularly human-readable. Let's imagine our app is only for tracking tasks on the same day, so we only care about the time part. We can amend our query to use the TIME

If you log the result of this query you should see the task objects change:

However the column name has changed to represent this. Ideally we would keep the same name (created_at). We can rename columns using the AS SQL operator:

If you run this again you should see the object key is now created_at like before.

Deleting a row

Let's add another model function that can delete a task from our tasks table. It should take an ID parameter and delete just the row that matches.

Test this by calling removeTask(1) to delete the first task. You can call listTasks to check that it has been removed.

Testing the model

So far we've been manually testing things by calling our model functions. It would be better to write some automated tests to make this reproducible and reusable. That way we can catch reversions if the code breaks later on.

Let's write a test to verify several of our model functions. Create a new file test/tasks.test.js:

Ideally our tests shouldn't mess with our dev environment (where we may have changes to the DB we don't want to overwrite). We can run the tests with a separate DB file by specifying a different value for the env var:

We should make sure to add test.sqlite to our .gitgnore too, since we don't want random DBs floating around on GitHub.

It's not necessary in this case, but if we wanted to seed the test DB (so we can assert against the example data) we can tell Node to require seed.js before running the test:

Updating a row

It's likely we'll need to be able to change the content of a task if a user wants to edit it. We need to write a function that can take a task object and update the row with a matching ID:

We return the edited task for convenience.

Let's write a quick test in test/tasks.test.js to make sure this works:

Toggling a boolean

When a user marks a task as complete (or incomplete) we need to update that row to match. We can easily toggle our "fake boolean" integer column using NOT:

This function will flip the complete column from 0 to 1, or from 1 to 0.

Let's add a test for this function too:

Integrating a UI

We now have all the functionality for our app contained in the model. All that's left is to build a UI that calls these functions, so that a user can interact with them.

Let's create a Node server that uses our new database. First install express:

Then we'll create the boilerplate we need to get an HTTP server going. First we need a server.js file:

and an index.js to start the server:

Check this is working by running node index.js in your terminal.

Submitting new tasks

Now we need a form to add new tasks to our database. Edit server.js:

Then add a / POST handler to receive the submission a insert the new task:

If we want our app to persist data we need to start it with the DB_FILE env var set:

Since we'll want to run this a lot we should add an npm script to save on typing:

Rendering tasks

Now that we can insert tasks we need to show them on the page. Edit the GET / handler to get the list and render them:

Updating tasks

We need to be able to toggle and delete each task. This will mean each task <li> needs to contain a <form> that can send a POST telling the server to either toggle or remove the task.

Let's write a POST /update handler first. This is what will carry out the changes when a task is toggled or removed. Writing this first will help us know what our form should include.

Our handler is going to need to know two things: which action should it take (toggle or remove), and which task should it update. Both will need to be submitted by the form as part of the request body:

Since all our "business logic" is contained in the model our route handler ends up fairly small.

Now we need to update the list render to add a form to each task. It'll have two submit buttons—one for each action—and a hidden input with the ID. Since it will get a bit long to embed inline we'll create a separate function to render this HTML:

Now the request body sent by the form will look like this if the toggle button was clicked:

or like this if the remove button was clicked:

The POST /update handler will call the right model method depending on the action.

Client-side validation is important for a good user experience—you can quickly give the user feedback when they need to change a value they've entered. For example if passwords must be a certain length you can tell them immediately, rather than waiting for the form to submit to the server and receive an invalid response.

Communicating requirements

It's important to tell the user what you expect them to do. As always you need to present information visually and programmatically, so user's of assistive technologies like screen readers can access it. At a bare minimum this means each form field need an associated label.

If the field will be validated you also need to communicate those requirements to the user ahead of time. There's nothing more frustrating than having a submission rejected for an unknown reason.

Required fields

If the user must provide a value the common convention is to put a * character after the label. You could also use the word "required".

It's important not to duplicate information for assistive technology users. For example if your field is already programmatically marked as required (e.g. via the required attribute), then hearing the * character read out is at best superfluous and at worst confusing. It's a good idea to hide this symbol from non-sighted users in this case:

More specific instructions

A field can have stricter validation than just "required"—for example a new password field might check the length and complexity of the value. In these cases you will need to provide the requirements after the label. To make sure this is available to assistive tech users you must also associate the element with the field. You can do this using the attribute on the field—this should be set to the ID of the element containing the instructions:

This description will be available to assistive tech users when they focus the input; screen readers will usually read it out after the label.

HTML5 validation

Now we've communicated our requirements to the user we need to actually enforce them. We need a way to check the values the user entered match our expectations, and prevent the form from submitting if they don't. Luckily browsers natively support lots of different types of validation via different HTML attributes.

If a form containing invalid values is submitted the browser will prevent the request from being sent, and instead will show a message for each invalid field telling the user what they did wrong.

Requiring values

The required attribute will stop the user submitting the form if they haven't entered this value yet.

Types of values

Browsers will validate certain input types to make sure the value looks correct. For example:

Some browsers (especially on smartphones) will even change their input method to match. For example the keyboard may show the @ key for an "email" input.

Matching a pattern

We can specify a regular expression the value must match using the pattern attribute. For example this input will be invalid if it contains whitespace characters:

Other validation

There are several other , which work for different kinds of inputs.

Enhancing with JavaScript

It's great that we can get a base level of client-side validation working with just HTML—if our JS fails to load (or breaks) the user gets basic validation. This makes it quick and simple to provide a helpful experience to users. However it has a few downsides:

• We cannot style the message bubbles that the browser shows for invalid fields.

• The messages are .

• Required inputs are marked invalid as soon as the page loads (since they are empty).

We can improve this user experience by enhancing our validation using JavaScript.

Disabling default form behaviour

First we need to tell our form not to do its own validation, since we're going to trigger this ourselves using JS. We can do this by setting the novalidate attribute:

Trigger validation from JS

To recreate the default behaviour we need to listen for the form's submit event, then prevent submission if there are any invalid fields. We can check all fields using the form element's .checkValidity() method. This returns true if all fields are valid, otherwise it returns false.

Marking invalid fields

Our "enhancement" is currently worse than the default, since it prevents submission without telling the user which fields are invalid. We need to provide feedback to the user so they can fix their mistakes.

First we need to tell the browser/assistive tech whether the field is valid or not. We can use the aria-invalid attribute for this. Each field should have aria-invalid="false" set at first, since it can't be invalid until we check it.

Now we need to know when the field fails validation, so we can update this attribute to "true". Luckily calling checkValidity() causes invalid fields to fire an "invalid" event that we can listen for:

Providing feedback

Marking fields as invalid isn't enough. We also need to provide the validation message that the browser previously showed. First we need to add this element to the DOM every field, and associate them using aria-describedby again. We want the DOM to end up like this before any validation runs:

Then when a field is invalid we need to grab the default message from the field's property and display it:

We have now replaced the default HTML experience, with all the problems we listed fixed.

Even more enhancement

Since we're validating using JS we can add more features if they make sense. Right now our form only validates on submission. This means users will not get feedback as they fill in the form, and fields will not get re-validated until the user submits again.

Re-validation

It would be nice to clear the invalid state when the user edits a field. We can do this by listening for the "input" event on the field and reversing the steps from before:

Validating more often

On longer forms it might be helpful for the user to see validation as they fill in fields, rather than waiting until they submit at the end. There is a balance here though—many apps validate on every key press, which often leads to fields being marked as invalid while the user is halfway through typing a valid value.

It's usually less annoying to validate when the user's focus leaves the field. We can do this by listening for the "blur" event, then triggering the validation using the field's method:

Styling

We have a functional, accessible solution now, but it could be improved with some styling. It's common to style validation messages with a "danger" colour like red. Relying on colour alone will not work for all users, so you should also mark invalid inputs with a visual change like a different coloured border or an icon.

You can target elements in CSS , which is helpful for targetting invalid inputs:

React makes dealing with the DOM in JavaScript more like writing HTML. It helps package up elements into "components" so you can divide your UI up into reusable pieces. You can try out the examples online by creating a new React playground.

Quick summary

A lot of React concepts are explained in detail below. If you just want to get started quickly here's a code sample with the most important features:

React elements

Interacting with the DOM can be awkward when you just want to render an element:

This is frustrating because there is a simpler, more declarative way to describe elements—HTML:

Unfortunately we can't use HTML inside JavaScript files. HTML is a static markup language—it can't create elements dynamically as a user interacts with our app. This is where React comes in:

This variable is a React element. It's created using a special syntax called that lets us write HTML-like elements within our JavaScript.

The example above will be transformed into a JS function call that returns an object:

Templating dynamic values

JSX supports inserting dynamic values into your elements. It uses a similar syntax to JS template literals: anything inside curly brackets will be evaluated as a JS expression:

You can do all kinds of JS stuff inside the curly brackets, like referencing other variables, or conditional expressions.

Note on expressions

You can put any valid JS expression inside the curly brackets. An expression is code that resolves to a value. I.e. you can assign it to a variable. These are all valid expressions:

This is not a valid expression:

if blocks are statements, not expressions. The main impact of this is that you have to use ternaries instead of if statements inside JSX.

React components

React elements aren't very useful on their own, since they're just static objects. To build an interface we need something reusable and dynamic, like functions.

A React component is a function that returns a React element.

Valid elements

Your components don't have to return JSX. A React element can be JSX, or a string, number, boolean, or array of elements. Returning null, undefined, false or "" will cause your component to render nothing.

Returning an array is especially useful for from data:

Array items in JSX must have a special unique so React can keep track of the order if the data changes.

Composing components

Components are useful because JSX allows us to compose them together just like HTML elements. We can use our Title component as JSX within another component:

When we use a component in JSX (<Title />) React will find the corresponding Title function, call it, and use whatever element it returns.

Customising components

A component where everything is hard-coded isn't very useful. Functions are most useful when they take arguments. Passing different arguments lets us change what the function returns each time we call it.

JSX supports passing arguments to your components. It does this using the same syntax as HTML:

Most people name this object "props" in their component function:

We can use these props within your components to customise them. For example we can insert them into our JSX:

Now we can re-use our Title component to render different DOM elements:

Typing our props

We need to define the type of the props object so TypeScript can check we're using the component correctly. We can do this in the same way we would type any object:

If you have several props it can be more readable to extract the type to an alias:

The React docs have a page on , which can be helpful.

Non-string props

Since JSX is JavaScript it supports passing any valid JS expression to your components, not just strings. To pass JS values as props you use curly brackets, just like interpolating expressions inside tags.

Children

It would be nice if we could nest our components just like HTML. Right now this won't work, since we hard-coded the text inside our <h1>:

JSX supports a special prop to achieve this: children. Whatever value you put between JSX tags will be passed to the component function as a prop named children.

You can then access and use it exactly like any other prop.

Now this JSX will work as we expect:

This is quite powerful, as you can now nest your components to build up more complex DOM elements.

Typing the children prop

Since the children of an element can be almost anything React has built-in type for it: React.ReactNode.

Rendering to the page

You may be wondering how we get these components to actually show up on the page. React manages the DOM for you, so you don't need to use document.createElement/.appendChild.

React consists of two libraries—the main React library and a specific ReactDOM library for rendering to the DOM. We can use ReactDOM to render a component to the DOM.

It's common practice to have a single top-level App component that contains all the rest of the UI.

Event listeners

JSX makes adding event listeners simple—you add them inline on the element you want to target. They are always formatted as "on" followed by the camelCased event name (onClick, onKeyDown etc):

React state

An app can't do much with static DOM elements—we need a way to create values that can change and trigger updates to the UI.

React provides a special "hook" function called useState to create a stateful value. When you update the value React will automatically re-render the component to ensure the UI stays up-to-date.

When this button is clicked we want the count to go up one:

We need to use the useState hook. It takes the initial state value as an argument, and returns an array. This array contains the state value itself, and a function that lets you update the state value.

It's common to use destructuring to shorten this:

If we call setCount(1) React will re-run our Counter component, but this time the count variable will be 1 instead of 0. This is how React keeps your UI in sync with the state.

Lifting state up

React components encapsulate their state—it lives inside that function and can't be accessed elsewhere. Sometimes however you need several components to read the same value. In these cases you should to a shared parent component:

Here FancyButton and FancyText both need access to the state, so we move it up to Counter and pass it down via props. That way both components can read/update the same state value.

Typing state

TS can mostly infer state types from the initial value you provide. In the Counter example above it will infer count to be of type number, since the initial value is 0. We can explicitly provide a type by passing a generic to useState:

That's not useful here, but can be necessary if for example you wish to constrain the type:

We need to define a type for the state setter function when we pass it down to another component as a prop (like the FancyButton example above). Since updating state does some React magic behind the scenes we can't just write a normal function type: we must use React's built-in types:

The React.Dispatch<React.SetStateAction<number>> is a little wild because of the triple-nested generic, but the only part you ever need to change is the final generic (<number> here). This should be the type of the actual state value.

Updates based on previous state

Sometimes your update depends on the previous state value. For example updating the count inside an interval. In these cases you can to the state updater. React will call this function with the previous state, and whatever you return will be set as the new state.

We cannot just reference count, since this is 0 when the interval is created. It would just do 0 + 1 over and over (so the count would be stuck at 1).

Form fields

React apps still use the DOM, so forms work the same way:

Typing DOM events

TS can infer the type of the event parameter in inline handlers. For example:

Here event will be inferred as React.MouseEvent<HTMLButtonElement>.

When you define event handlers as separate function (like the updateName example above) you will need to manually provide this type. React —you just need to pass in the type of DOM element it will be triggered for. For example:

Often the easiest way to find the type is to write the handler inline first, then copy the type that is inferred.

Controlled components

React tries to normalise the different form fields, so behaviour is consistent across e.g. <input> and <select>. If you need to keep track of values as they update you can add an onChange listener and value prop.

This pattern is often known as "controlled components".

Side effects

So far we've seen how React keeps your UI in sync with your data. Your components describe the UI using JSX and React updates the DOM as required. However apps sometimes need to sync with something else, like fetching from an API or setting up a timer.

These are known as "side effects", and they can't be represented with JSX. This means we need a different way to ensure our side effects stay in sync just like our UI.

Using effects

React provides another "hook" like useState() for running side-effects after your component renders. It's called useEffect(). It takes a function as an argument, which will be run after every render by default.

Here's our counter, with an effect to sync the document title with the count:

Calling setCount will trigger a re-render, which will cause the Effect to re-run, so the title will stay in sync with our state.

Skipping effects

By default all the Effects in a component will re-run after every render of that component. This ensures the Effect always has the correct state values. However what if we had multiple state values? Updating unrelated state would re-run the Effect even if count hadn't changed.

useEffect() takes a second argument: an array of dependencies for the Effect. Any variable used inside your Effect function should go into this array:

Now the Effect will only re-run if the value of count has changed.

Effects with no dependencies

Sometimes your Effect will not be dependent on any props or state. In this case you can pass an empty array, to signify that the Effect has no dependencies and shouldn't need to be re-run.

Here we want to show what key the user pressed, so we need an event listener on the window. This listener only needs to be added once:

Without the empty dependency array we would end up adding a new event listener every time the Effect re-ran. This could cause performance problems.

Cleaning up effects

It's important your Effects can clean up after themselves. Otherwise they might leave their side-effects around when the component is "unmounted" (e.g. if the user navigates to another page).

Our previous example needs to make sure the event listener is removed from the window. We can tell React to do this by returning a function from the Effect. React will call this function whenever it needs to clean up: both when the component is unmounted and before re-running the Effect.

React helps you remember to do this by (even if you pass an empty dependency array). This is designed to help you catch places where you forgot to clean up your Effect.

This workshop is also available as a video.

TypeScript is JavaScript with some additional syntax for describing what types things are. Here's an example:

What are types?

A type is something that tells the language what a piece of data is and how it's intended to be used. For example JavaScript has 7 "primitive" types:

1. Null

2. Undefined

3. Boolean

4. Number

5. BigInt

6. String

7. Symbol

Everything else is the Object type. Arrays are objects with number keys, and functions are objects that are also callable.

You don't really need to understand JS types in detail to use TypeScript, but if you're interested there's .

What is static typing?

JavaScript is a dynamically typed language. This means the language figures out what a piece of data should be when your program runs.

There are benefits to this: the language doesn't have to be compiled before you can run your code, and it's friendlier to beginners because you don't have to think about types yourself.

There are also downsides: it's easy to mix up your types and try to do something like access an undefined value, which often results in strange bugs. You also don't get lots of helpful editor features as you write your code, like smart autocompletion and error highlighting when you try to do things you shouldn't.

Statically typed languages on the other hand try to understand what types things are before the code even runs, either by inferring obvious types or having the author explicitly write them.

Although this is more work upfront it often results in code that works the first time you run it, since the language has ensure everything is correct in advance.

How does TypeScript work?

TypeScript lets you write normal JS with a bit of extra stuff to annotate what types things are (in places where TS can't figure them out automatically).

JS engines cannot execute TS code directly. Instead TS must be compiled to JavaScript before being run. This adds an additional step of complexity to any TS project, although certain runtimes like Deno and Bun make this simpler by automatically compiling TS code before executing it.

Let's see how we can set up a basic Node project to compile TypeScript for us.

Setting up a TypeScript project from scratch

Initial setup

First we need a new directory to put our project in:

Then we need to initialise a JS project using npm:

Now we can use npm to install dependencies—3rd party code that our project relies on to work. First of all we will definitely need the TypeScript compiler. This is what will ensure our code works, and turn it into JS we can run.

Note that we install it with the -D flag to mark it as a development dependency. That's because our final code won't use the typescript compiler—we just need it during development to compile TS to JS.

Our first TypeScript

Let's try compiling some TS and running it. Open the project in your editor of choice. If you're using VS Code you should be able to do this with:

Create a file named index.ts and add some TS code to check everything is working:

You can try running this code:

Node will attempt to run it, since it doesn't actually care what extensions your files have, but you should get a syntax error:

That's because Node's V8 engine doesn't understand TypeScript syntax. We need to use the typescript library we installed to compile this TS to JS.

If the compiler succeeds you should get nothing logged to your terminal. Instead you should now have an index.js file next to your index.ts. This will contain your compiled JS code:

The compiler has stripped out the special TS syntax, which means we can now run this code with Node:

You should see "It's working!" logged to your terminal.

Configuring the compiler

Since TypeScript needs to produce code for many different JS runtimes the compiler has lots of options. We can configure it directly from the command line. For example to change where our compiled JS code will go:

This should create a build/ directory and put index.js in there.

It can get cumbersome to do this as your project starts to need more and more configuration, so it's common to use a tsconfig.json file to contain all the options you need. Create a new file named tsconfig.json and add these options:

This tells our compiler 3 things:

1. We want our code to be compiled into the build/ directory

2. We are targeting the modern Node module system

3. We want TS to be as as possible with our code

Now we can just run the compiler with no arguments. It will autodetect the config and any TS files.

You should see one change in build/index.js: it now has "use strict;" at the top. This is because we enabled the strict option, so TS is ensuring our code opts in to JS's .

Watching for changes

Right now we have an awkward 3-step process for making changes. We have to save changes to a file, compile TS to JS, then run the JS with Node. Ideally we want a smoother development experience while we're working on code.

Both Node and TypeScript have a "watch mode", where they will automatically restart when you save changes to a file. There is a slight complication in that we need to run both at once: tsc to watch for changes to .ts files and Node to watch for changes to build/index.js. The simplest way is to open two panes/tabs in your terminal and run one command in each:

Now whenever you edit a .ts file tsc will re-compile it into build/. This will cause Node to re-run your .js code automatically.

Building an app with TypeScript

We're going to build a simple task management app—this will let us see how various TypeScript features help us write more robust code. We'll focus on just the business logic, without worrying about how the user interacts with the app for now.

We've already got our project set up compiling TypeScript, so we can start working in index.ts. You can see TS errors in two places:

1. Your terminal after you run tsc

2. Highlighted in VS Code as a red underline. Hover the underlined code to see the error in a popup

Creating tasks

The first feature our app needs is task creation. We'll write a function that takes the task content and returns an object with the properties describing the task. We won't worry about where we're storing tasks yet.

You should see a type error highlighting the content parameter:

Function parameter types

TypeScript is telling us that it cannot infer the type of content. Since we enabled the strict option this is an error that will stop our code from compiling. We want TS to always know what types our function parameters are so it can catch any mistakes we make inside the function.

We can fix the error by using a type annotation to tell TS what type content is:

Type annotations

Type annotations can also be used for normal variable definitions. We could add one to the index variable:

However this is not necessary here as TS can infer the type, since we initialised it as a number. Generally you only need to annotate a variable if you aren't going to give it a value yet.

Function return types

If you hover the function name in VS Code you can see the type signature of create:

Currently TS is inferring the return type, based on what we actually returned. This is handy, since we'll get autocompletion and mistake checking when we use it:

It is often useful to explicitly annotate function return types. You can do that with an annotation after the parameters:

For simple functions like this it can feel repetitive, but it can be helpful to be clear with your intentions before you write a function. That way TS will catch mistakes if you accidentally return something different.

Literal types

Our status type is broader than it needs to be. Currently it's typed as string, when it can technically only ever be "incomplete". It's usually a good idea for your types to be as strict as possible to prevent errors. You can broaden types as needed later on.

TS supports "literal types", which are literally a value instead of a primitive. Let's update our function's return type:

Completing tasks

The next feature our app needs is the ability to complete a task. Let's write a function that takes a task object and changes the status property:

TS will error here just like before, since we have not told it what type the task parameter is.

We can fix this by adding a type annotation to the parameter:

You may have noticed we are repeating ourselves here. Let's see how we can avoid this.

Type aliases

We've got two identical copies of the type representing a task object. In JS we would probably abstract this repeated value to a variable—in TS we can do something similar using "type aliases". These let you define a named type for re-use using the type keyword.

Let's define a new type called Task that we can use for both functions:

Type aliases are usually capitalised to distinguish them from normal JS variables. They can also be imported and exported just like JS values.

Type unions

There is another type error in our code. The assignment to task.status is not allowed:

This is because we defined status as a literal type. We need to update the definition to allow another possible value. TS supports something called "union types" to achieve this. You can specify multiple types separated by a | character (similar to a JS "logical or" operator). TS will check that values match one of the listed types.

Let's update our Task definition to allow another status type and fix our error:

Note that type aliases don't have to be object types. For example we could define our status property as its own type if we felt it was too complicated to write inline:

Types are just like normal variables, so you can compose them together in this way if it's helpful.

Storing tasks

Right now our app doesn't really work, since we can only create single tasks. We need a place to store a list of tasks. Let's use a global array variable for this. We'll need to push tasks into it when we create them:

We'll see a new type error here highlighting the return value:

More complex TS errors like this can be a little confusing, so don't worry if they feel overwhelming. This one is complaining that the type of the status property in the object we're returning is not compatible with the status property of the Task type that we've explicitly said our function will return.

If you hover the task variable you will see that TS infers the type as:

The error is saying that our task's status can be any string, rather than restricted to just "incomplete" or "complete". This might seem weird considering we've set status to "incomplete", but remember that JS objects are mutable. They can be changed as any time, so TS cannot rely on the initial value of a property. For example this would cause problems:

We can fix this by explicitly annotating the variable's type:

Marking complete by ID

Now that we have a list of tasks our complete function could be made more useful if it could mark tasks as complete with just the id.

Interestingly this will create 2 new type errors. The tasks.find call doesn't work because:

And let tasks = [] is now also an error:

This wasn't a problem before because we never actually accessed anything inside the tasks array. Now that we are TS is asking us to explicitly say what type tasks is, since it cannot infer the type.

Collection types

We can define types for collections using a new piece of TS syntax:

The angle brackets are a feature called . These are like types with parameters—the array type can't know what type of thing will be stored inside it, so it requires you to pass this type in, almost like a function argument.

Other JS collection types work in similar ways. For example: Set<number> or Map<string, number>.

Array types have a shorthand syntax that you will sometimes see: Task[]. We'll stick to using the long form though, since it is consistent with other types and can be less confusing for longer array types.

Function parameter type inference

It's worth noting here that we did not have to write type annotations for the inline function we passed to tasks.find.

TS can infer the type of task here, since it knows we are iterating over an array of Task objects. We get autocompletion and error checking for free.

Type narrowing

We have another type error. TS isn't happy that we're assigning to task.status:

If you hover the task variable returned by tasks.find you'll see that its type is Task | undefined. That's because the find method on arrays is not guaranteed to find anything. If complete was passed an invalid id we'd end up with an undefined task. Trying to assign to the status property would cause a JS runtime error, which would crash our program.

All the compiler errors can seem annoying, but TS is really trying to save us from ourselves!

We can fix this by checking that we really did find a task before we attempt to assign to it:

This is called —the TS compiler is smart enough to change types inside of conditionals. Inside of this if it is impossible for task to be undefined, so our assignment is allowed.

Task removal

Let's write a function to remove tasks from our list. It should receive an id and remove the matching Task object from the tasks array. There are several ways to remove items from arrays, but we'll stick with splice:

There is a bug in this code that TS has not caught: findIndex returns -1 when it cannot find a match. Since -1 is still a valid number the return type is just number. This means that unlike with find, which can return undefined, TS cannot force us to handle the "not found" case here.

If we passed this function an invalid id the index would be -1, so we would call tasks.splice(-1, 1). A negative index causes splice to work backwards from the end, incorrectly removing the final item from the array. We need to write a conditional to prevent this behaviour:

This emphasises that TypeScript cannot catch every bug in your code. You still need to understand what you're writing and remain vigilant!

New kinds of tasks

Let's imagine our users want to record birthdays in the task app. We'd need a way to distinguish a birthday from a normal task.

Optional properties

The simplest way would be to add an optional birthday property to the Task object:

The after a property tells TS that it is optional—it can be missing or set to undefined.

This new type might work for us. We can repurpose the content property as the birthday person's name. However birthdays don't really have "statuses"—they can't be completed. We should probably mark status as optional using a ? as well.

We also need a createBirthday function to insert birthdays into the list:

This works for now, but we'll soon see some issues caused by this structure.

Listing tasks

Now we have two kinds of tasks we should write a function that prints a nicely formatted list:

Unfortunately we've got an error:

Since we made the birthday property optional we need to handle the case where a task doesn't have this property. We also probably want to log different formats for tasks vs birthdays.

Unfortunately the way we've structured our data has made this kind of difficult—tasks and birthdays aren't really that similar, so it would make more sense for them to be separate types. That would avoid the mess of having to check for optional properties all over the place.

Let's extract a new type alias to represent birthday objects:

We can then update the tasks array to contain either Task object or a Birthday object (using a union):

We'll also update the Task definition to make status a required property again.

We've also added a new property to both types: kind. This will allow us to check what type we're working with, and is generally a good way to structure type unions (it's known as ).

Finally we'll update our other functions to fit the new structure:

Our list function is now easier to write, since we can split the logic for each kind of task:

TypeScript can narrow the type of task inside each branch of the if so that it knows what properties are available.

Styling text

Our logs to the terminal are looking a little boring. Luckily as of Node version 20 we can apply styles to this text using the built-in .

Unfortunately this will immediately cause an error:

Third party types

Node is not written in TypeScript, and so does not include built-in type definitions. TS needs us to provide these definitions, otherwise it has no idea what node:util exports.

When a library you want to use does not include its own types you can usually find them in the community project . You can install types from this project using npm in your terminal:

This will fix our error and let us use any Node built-ins without any problems.

Now we can use styleText to make our logs prettier:

Bonus TypeScript features

There are some useful things that we didn't need to build this app. They're a little more advanced, but included here for reference as you may need them when building more complex things.

Type assertions

Sometimes TS has no way to know what type a value is. For example when data is coming from outside the program (like the response to a fetch request). In these cases you may need to override the type system and just enforce a particular type using the as keyword.

For example when working with the DOM:

TS has no way of knowing whether an element with this ID exists in the DOM, so it will force you to check to make sure the variable is not null. There are two ways to handle this. If you are sure there is an element with that ID you can using as:

Or you can actually check your assumption is correct using code to narrow the type:

Unknown types

Sometime you just cannot know the type of something. In these cases you have two options: any and unknown. The any type effectively turns off type-checking, which makes it very dangerous. You're telling TS: "any type is valid here, so don't both checking it". This can be a big source of bugs, so it's generally not advised to rely on any. If a type is TS will force you to narrow it before you can do anything with it. For example:

Intersection types

We can use the to combined multiple types into one. This is like the inverse of a union, which makes a type more permissive. Instead it makes a type stricter. For example:

Generics

Sometimes you might want to write dynamic types that depend on some later type. For example the built-in array type we saw earlier needs to know what type of thing it is going to hold (e.g. Array<string>). TS uses for this—you can think of them like parameters for your types.

For example if we wanted to represent the return type of functions that can fail:

Generics are an advanced topic so you probably won't need to write your own very often yet, but it's good to know what's happening when you see pointy brackets.

Node is often used to create HTTP servers for the web. It's a bit fiddly to do this with just the built-in modules, so we're going to use the Express library to help create our server.

HTTP recap

HyperText Transfer Protocol (HTTP) is a way for computers to exchange messages over the internet. The "client" computer will send a "request" (often via a web browser). E.g. if you visit https://google.com your browser sends a request like this:

A "server" computer receives this request and sends a "response". E.g. Google's server would send a response like this:

We're going to learn how to use Node to create an HTTP server that can respond to requests.

Getting started

1. Create a new directory

2. Move into that directory

3. Create an empty package.json

Follow along with each example in your own editor.

CommonJS modules

Before we start creating a server we need to learn how Node manages code in different files. Modules are used to isolate code. In the browser by default all JS has access to the same global scope, even if loaded via separate script tags.

Although browsers do now have Node was created before they were added to JavaScript. This means it has its own system (called "CommonJS").

Confusingly Node has recently added support for these standard JS modules (ESM), however most server code has not yet been updated, so we must learn the older system.

Exporting code

By default all files are self-contained in Node. Code in one file cannot access anything in another file. To use something in another file you must "export" it. You can do so by assigning the value to module.exports:

Importing code

To access code that is exported from another file you must "import" it. You can do so by calling the require function with the path to the file:

Note: the file extension is optional—if you leave it off Node will assume it's a .js file. This is quite common in the Node ecosystem.

Multiple exports

You can export multiple values by assigning an object to module.exports:

Built-in & 3rd party modules

Node provides some built-in modules. You can import these just like your own modules, only without the path (just using their name). Recent versions of Node allow you to add the node: prefix to make it more explicit that this is a built-in. For example to use the built-in "fs" (filesystem) module:

The same applies to modules you installed via npm. Node will check in your node_modules directory to find any imports that aren't relative file paths. For example to import the Express library:

Now we know how to manage our modules let's take a look at using Express to build a server.

Creating a server

We can create a new server object using the express library:

You can run this file from your terminal with:

Unfortunately nothing much will happen yet, since we haven't told our server to do anything.

Starting the server

Our server isn't currently listening for requests. Servers need to connect to the network and listen for incoming HTTP requests via a "port".

It's a good idea to start the server in a different file: this will help with testing later on. We first need to export the server object we just created:

Then we create a new file named index.js file. This is just the "entrypoint" for our app—its job is to start the server. All the actual server logic will stay in server.js.

This tells the server to listen for any requests sent to port 3000. Now we can run the program in our terminal again:

The server will start, but you won't see anything happen. Your terminal will be "stuck" as the Node process is still running (the server will keep listening indefinitely). This means you can't run any more commands.

It would be nice see a log so we know our server started correctly. Luckily the .listen method takes a second argument—a function to run once the server is listening.

We can use this to log a message:

Stop the previous process, re-run your file, and you should see "Listening on http://localhost:3000" logged.

Trying out our server

Since your server is now running you can send some HTTP requests to it to see what happens. The easiest way is to visit http://localhost:3000/ in your browser.

You can also use the curl CLI program to send requests right from your terminal. Open a new tab and run this command to send a GET request:

You should see the response logged. Whether your used your browser or curl you should get a 404 Not found response from your server.

This is Express' default "missing" page, which it will return when you haven't defined a response for a particular request. Let's fix that by defining our first route.

Handling requests

Our server currently does nothing. We need to add a "route". This is a function that will be run whenever the server receives a request for a specific path.

The server object has methods representing all the HTTP verbs (GET, POST etc). These methods take two arguments: the path to match and a handler function.

Here we tell the server to call our function whenever it receives a HTTP GET request to the home path. This is similar to an event listener in the DOM.

The handler function will be passed two arguments: an object representing the incoming request, and an object representing the response that will eventually be sent. Here we've named them request and response respectively. These are often abbreviated to req and res.

We can use the send method of the response object to tell Express to send the response. Whatever argument we pass will be sent as the response body.

Open http://localhost:3000/ in your browser. This will send a GET request to your server. You should see the "hello" response on the page. It's helpful to open the network tab of the dev tools so you can see all the details of the request and response.

Testing our server

Making requests manually in the browser or terminal is fine for quick checks, but to be responsible developers we should write automated tests to verify our server is working correctly.

Testing with Node

As of version 18 Node has a built-in test runner. It works in a similar way to the popular Tape testing library (most testing libraries are quite similar).

Let's try an example test. Create a new directory called test/, and then a new file inside called server.test.js. We can create a simple test like this:

Since this is a normal Node JS file we can run it from our terminal like any other JS file:

You should see some logs showing the status of your test.

Node also has a special way to run all the tests in your project:

It will find and run any files in a folder named test, and also any file ending in .test.js. This is handy when you want to divide your tests up into different files but run them all in one go.

Testing our server

Let's write a test that verifies the route we just added. Our test should:

1. Start the server

2. Send a request to the server

3. Verify the request was successful

Note that this is exactly how we manually checked our server was working before. We just want to automate it.

First we need a test function. We will use an async function here so that Node knows to wait for any promises to resolve:

Start the server

Next we need to start the server. We'll use a different port to before, just so we don't clash with any already running instances in your terminal:

We import the server object directly and start it listening ourselves. This is why splitting up our app into index.js and server.js helps testing: we have control over when the server starts.

Send a request to the server

Now we need to send a request to the server. As of Node 18 we can use fetch to make HTTP requests, just like in the browser. We can use it to send a request to our server:

Here we're sending a request, waiting for the server to respond, closing the server, then checking that the response was successful. We need to close the server before any assertions, since the test will stop executing as soon as an assertion fails. If that happened before we closed it the server would never stop listening.

Check the response body

Finally we need to get the response body and check that it is correct:

Here we're using the .text() method to get the raw text content of the response body, then checking it matches what we expect.

Now that we know how to test our server we can get back to learning a bit more about Express.

Port flexibility

We are currently hard-coding the port our server listens on to 3000. This works fine, but we could make it more flexible. When you deploy this code to a hosting provider like Heroku they will run your code on their computer. This computer may have lots of different programs running that all need to use ports. Ideally the host can tell your server which port to listen to.

This is usually achieved with . These are like global variables that are set before your program runs. For example we can set a variable named TEST in our terminal like this:

Remember if you are using Windows (without WSL) you will have to use the SET command and ampersands between commands like this:

Node makes environment variables available via the global process.env object. So we could read that TEST variable using this JS:

We can tweak our server code in index.js to use a PORT environment variable if it's set, otherwise fallback to 3000:

Now we can control what port our server listens on without editing the code. E.g. to start it using a different port:

When you deploy to a hosting provider like Heroku they will use this to start your server with a random available port.

The response

HTTP responses need a few different things:

1. A status code (e.g. 200 for success or 404 for not found)

2. Headers to provide info about the response

3. A body (the response data itself)

Status code

Our home route currently only provides the body, as Express will set the status code to 200 by default when you call response.send(). To set a different code use the response.status method. Let's add a new route that returns a different status code:

You can chain .status together with .send to make it shorter:

Try creating a test for this new route in tests/server.test.js: it should check that the status is 500 and the body is "something went wrong". It will look very similar to the first test we wrote.

Headers

Express will automatically set some HTTP headers describing the response. For example since we called send with a string it will set the content-type to text/html and the content-length to the size of the string.

You can set your own headers using the response.set method. This can take two strings to set a single header:

Or it can take an object of string values to set multiple headers:

HTML body

We aren't limited to plaintext in our body. The browser will parse any HTML tags and render them on the page. Change your home route to return some HTML instead:

Run your tests again and you should see the first one fail, since it is still expecting the response body to be "hello". Amend the test so that it matches the new body.

Visit http://localhost:3000/ again and you should see an h1 rendered.

Since we're rendering HTML using JS strings we can insert dynamic values using template literals. Let's add the current year to the response:

You'll need to amend your test again to keep it passing.

The request

Now let's look at the HTTP request. This object include lots of information that's useful to us.

Search parameters

Incoming request URLs can contain extra information known as "search parameters" (or sometimes "query parameters"). These are key/value pairs listed after a ? character in the URL.

They are usually added to the URL automatically as the result of a form submission. For example this form:

will send a GET request to /search?keyword=what-the-user-typed. This allows your server to receive user-submitted information.

Let's write a test first before we try to implement this. Our route is going to be simple: it will return a message telling the user what keyword they submitted. Our new test should check that a request to /search?keyword=bananas returns a body including "You searched for bananas".

This test will currently fail when you run it, as we now need to implement the new route in server.js.

Express provides the search parameters from the URL on the request.query object. Each key=value pair will be parsed and represented as an object property.

Your test should now pass when you run it. If you visit http://localhost:3000/search?keyword=css you should see "You searched for css" on the page.

Dynamic paths

Sometimes you can't know in advance all the routes you need. For example if you wanted a page for each user profile in your app: /users/oli, /users/dan etc. You can't statically list every possible route here. Instead you can use a placeholder value in the path to indicate that part of it is variable:

We use a colon (:) to indicate to Express that any value can match a part of the path. It will put any matched values on the request.params object so you can use them. For example a request to /users/oli would result in a request.params object like: { name: "oli" }.

If you visit http://localhost:3000/users/oli you should see "Hello oli". If you visit http://localhost:3000/users/knadkmnaf you should see "Hello knadkmnaf".

Missing routes

Try visiting http://localhost:3000/asdfg in your browser. You should see Cannot GET /asdfg. This is Express' default response for when no handler matches a path.

You can customise this by putting a "catch-all" handler after all your other routes. If no other route matches then this will be used (since Express matches them in the order they are defined).

We can use the server.use method to create a handler that will match any method/route:

Reload http://localhost:3000/asdfg and you should now see your custom response. Try writing a test that checks whether requests to different URLs correctly return this 404 response.

Middleware

Express route handlers don't have to send a response. They actually receive a third argument: the next function. Calling this function tells Express to move on to the next handler registered for the route, without sending a response to the browser.

Let's add another handler for the home route. It will just log the request, then move on to the next handler:

If you run this code and refresh the home page you should see GET / logged in your terminal.

The route methods like .get accept multiple handler functions, so you can actually pass several all in one go. I.e. this does the same thing as above:

Express calls handlers that don't send a response "middleware". Our example here isn't that useful, but we could change it to run before all requests. We can do this with server.use:

Now we'll get a helpful log like GET / in our terminal when we load any page. Without middleware we would have to copy this into every route we wrote.

We'll be making more use of middleware in later topics where we have shared logic that must run on many different routes.

Static files

It's common to have some static files that don't change for each request. E.g. CSS, images, maybe some basic HTML pages. For convenience Express includes a built-in middleware for serving a directory of files: express.static.

Create a new directory named public. This is where we'll keep all the files sent to the client. Create a public/style.css file with some example CSS:

Finally configure the middleware to serve this directory:

The server will now handle requests to http://localhost:3000/style.css and respond with the file contents. Note that there is no public in the final URL: Express serves the files from the root of the site.

We can link this CSS from our homepage to apply the styles:

Visit http://localhost:3000/ in your browser again and you should see the styles applied.

Post requests

So far we've only created GET handlers. Let's add a POST handler to see how we'd deal with forms submitting user data to our server:

We can't make a POST request as easily in our browser, since that would require a form. Instead let's write a test to check this worked.

Request body

A POST request that doesn't send any data isn't very useful. Usually a form would be submitting some user input. Let's imagine we have a form that asks a user's name. We want to get the name they submit and send back a message like "thanks for submitting, oli".

Since bodies can be large they are sent in small chunks—this means there's no simple way to read the body. Instead we must use a "body parser" middleware. This will wait for all the chunks to be received, then make the final body available to our route handler via the request object.

For convenience Express includes parsers for different formats. Request bodies can come in different formats (JSON, form submission etc), so we must use the right middleware. We want express.urlencoded, which is what HTML forms submit by default. It will add a request.body property, which is an object containing the submitted data: