TheanoLM is a recurrent neural network language modeling toolkit implemented using Theano. Theano allows the user to customize and extend the neural network very conveniently, still generating highly efficient code that can utilize multiple GPUs or CPUs for parallel computation. TheanoLM allows the user to specify arbitrary network architecture. New layer types and optimization methods can be easily implemented.