Create, and encourage others to create. Because creation is the true purpose of life. It is what living things do, they transform the chaos that surrounds them into order. This is the story of six months spent building a deep tech company from scratch, as part of EF9, the 9th…

Batch normalisation is a technique for improving the performance and stability of neural networks, and also makes more sophisticated deep learning architectures work in practice (like DCGANs). The idea is to normalise the inputs of each layer in such a way that they have a mean output activation of zero…

An Autoencoder is neural network capable of unsupervised feature learning. Neural networks are typically used for supervised learning problems, trying to predict a target vector y from input vectors x. An Autoencoder network, however, tries to predict x from x, without the need for labels. …

A Dynamic Memory Network (DMN) is a neural network architecture optimised for question-answering (QA) problems. Given a training set of input sequences (knowledge) and questions, it can form episodic memories, and use them to generate relevant answers. Whilst classic Encoder-Decoder (Seq2Seq) models can solve QA problems, their performance is limited…

If you’re a software engineer, you quickly come to view errors as bad. Each error represents some misunderstanding, you thought the program would behave in some way, but it didn’t. Now some aspect of its state is inconsistent with how it’s meant to be. …

The Big Idea of Word Embedding is to turn text into numbers. This transformation is necessary because many machine learning algorithms (including deep nets) require their input to be vectors of continuous values; they just won’t work on strings of plain text. So a natural language modelling technique like Word…

The activation of a node in a neural network is determined by the following: output = activation_function(dot_product(weights, inputs) + bias) This means when calculating the output of a node, the inputs are multiplied by weights, and a bias value is added to the result. The bias value allows the activation…

This is a notebook, a place to record what I learn as I explore a new frontier of applied artificial intelligence. My aim is to explain the key concepts in simple language, because complexity doesn’t have to mean complicated. Foundation Concepts Activation Function —why we make outputs non-linear Autoencoder — how to…

The mathematical foundations of neural networks are differentiable graphs — chains of computations whose derivative can be calculated. It is this property that allows the weights between nodes to be optimised through a process called Backpropagation, and what allows a network to learn from its mistakes. Backpropagation is so named…