All Articles



Lyric RNN is a Recurrent Neural Network which generates lyrics. I have trained it on Hip-Hop, so be warned the content following may be explicit. Here is a small song I made (It is explicit)

This post more acts like a redirection to resources and my code for LyricRNN.

What exactly are RNNs

I will assume, that you allready know what a standard feed forward neural net is. So let’s just start by talking about why you would want a RNN instead of a NN.

Recurrent Neural Networks have one minor advantage over standard neural networks. They can work with sequences of data. Guess what, a text is a sequence and lyrics are text. This means we know that we will use a RNN as our model.

However there different types of RNNs. There are Standard RNNs and there are LSTMs. I will not talk about theory, because many others allready have done. I recommend Andrej Karpathys Blog post on RNNs and Christopher Olah’s blog post on LSTMs. With this being said, let’s get to the fun part.

How to generate Lyrics

I have created a simple Jupyter Notebook, you can find on this page, to guide you. I recommend you download the data and then use Google Colab and import the data from you’re machine. This avoids the hassle of setting up and deeplearning enviroment and you can get a GPU for training.