Category Archives: GPGPU


Deep Learning

download article

To dig even deeper into deep learning, please have a look at the technical report I wrote on my findings (PDF document).

I have had the pleasure of diving into the deep waters of deep learning and learned to swim around.

Deep learning is a topic in the field of artificial intelligence (AI) and is a relatively new research area although based on the popular artificial neural networks that supposedly mirror brain function. With the development of the perceptron in the 1950s and 1960s by Frank RosenBlatt, research began on artificial neural networks. To further mimic the architectural depth of the brain, researchers wanted to train a deep multi-layer neural network – this, however, did not happen until Geoffrey Hinton in 2006 introduced Deep Belief Networks.

Recently, the topic of deep learning has gained public interest. Large web companies such as Google and Facebook have a focused research on AI and an ever increasing amount of compute power, which has led to researchers finally being able to produce results that are of interest to the general public. In July 2012 Google trained a deep learning network on YouTube videos with the remarkable result that the network learned to recognize humans as well as cats, and in January this year Google successfully used deep learning on Street View images to automatically recognize house numbers with an accuracy comparable to that of a human operator. In March this year Facebook announced their DeepFace algorithm that is able to match faces in photos with Facebook users almost as accurately as a human can do.

To get some hands-on experience I set up a Deep Belief Network using the python library Theano and by showing it examples of human faces, I managed to teach it the features such that it could generate new and previously unseen samples of human faces.

The ORL Database of Faces contains 400 images of the following kind:


By training with these images, the Deep Belief Network generated these examples of what it believes a face to look like


The variation in the head position and facial expressions of the dataset makes the sampled faces a bit blurry, so I wanted to try out a more uniform dataset.

The Extended Yale Face Database B consists of images like the following


and in the cropped version we have 2414 images that are uniformly cropped to just include the faces.
Training the Deep Belief Network with this dataset, it generated these never before seen images that actually look like human faces. In other words; these images are entirely computer generated, as a result of the Deep Learning algorithm. Based only on the input images the algorithm has learned how to “draw” the human faces below:



Hinnerup Net A/S presentation at NVIDIA GTC 2012

 Photo by NVIDIA (

At the NIVIDIA GPU Technology Conference in San Jose, California 13th to 17th of May 2012, Michael S. Fosgerau from Hinnerup Net A/S and Henrik Høj Madsen from LEGO presented a session about a number of development challenges and learnings from developing a scalable and distributed GPGPU 3D optimization and rendering system (link no longer works – new link here)for LEGO.

The system consists of 16 blade servers of the type NVIDIA Optiplex 2200 S4, each containing four NVIDIA Quadro FX 5800 GPUs, summing up to a total of:

  • 64 Quadro FX 5800 GPUs
  • 256 Gb GPU memory
  • 15360 CUDA cores

We participated the full conference and have returned home with new ideas, inspiration and knowledge for future projects within the field of GPGPU and massive parallel computing.

Graphic cards are not just for games and fancy effects. Vast computational power hides away in many modern GPU cards. During the conference, NVIDIA launched their new flag ship model, the GTX 690 card that contains two Kepler GPUs with a shared computational power of 2 x 2,810.88 GFLOPS (FMA), or approx. 5.6 TFLOPS. This was just recently what could be squeezed out of a supercomputer (around year 2000). You can see the entire key note by NVIDIA’s CEO Jen-Hsun Huang here: Opening day key note, GTC 2012.

A detailed article about the presentation kan be read on NVIDIA’s blog.

Slides from the presentation will be made public on the NIVIDIA website. When this happens we will update this blog post with links to the slides.