Deep Learning for Bug Localization and Program Repair

Abstract

In this thesis, we focus on the problem of program debugging and present novel deep learning based techniques for bug-localization and program repair. Deep learning techniques have been successfully applied to a variety of tasks in natural language processing over the years. Although natural language text and programs are similar to some extent, the latter have procedural interpretation and richer structure. Applying deep learning techniques to programs presents many novel challenges, which arise due to these differences. We address some of these challenges in this thesis. Most of the existing program debugging research is dominated by formal, theory-first approaches. These approaches fail to take advantage of the existing codebases available online in the form of open source software repositories and student assignment submissions to massive open online courses on programming. Recognizing this, researchers have begun to replace expert-designed heuristics with models learned from codebases to improve the performance of the conventional debugging techniques. This thesis shows that it is possible to solve program debugging problems directly from raw programs using deep learning techniques in an end-to-end manner. More specifically, we present three approaches for bug-localization and program repair that are entirely data-driven and learn to perform their task instead of following the steps specified by a domain expert. We first introduce the notion of common programming errors and present a deep neural network based end-to-end technique, called DeepFix, that can fix multiple such errors in a program without relying on any external tool to locate or fix them. At the heart of DeepFix is a multi-layered sequence-to-sequence neural network equipped with an attention mechanism, comprising an encoder recurrent neural network (RNN) to process the input and a decoder RNN that generates the output by attending to the encoded input. The network is trained on a labeled dataset to predict a faulty program location along...