Repairnator can do bug fix easily (The automated Programmer)
The automated programmer, called Repairnator, wrote patches good enough to fool actual human engineers.
As in the words of Benjamin Franklin, Nothing can be said to be certain in this world, except Death and Taxes. He could also say the same about “software bugs”.
Modern computer programs are so complex that bugs inevitably crop up during the development process and writing a full proof computer program is very much important to do any specific type of work. That’s why finding them and writing patches to target bug fix is an ordinary part of any software development schedule. Indeed, there are companies such as Travis that offer bug fix service to developers.
But finding and fixing patches is a
time-consuming business that uses up significant resources.
Various researchers have developed bots that automate bug fix process, but they
tend to be slow or to produce poorly written code that does not pass muster.
Today, their dreams come true thanks to the work of Martin Monperrus and pals at the KTH Royal Institute of Technology in Stockholm, Sweden. These guys have finally built a bot that can compete with human developers in finding bugs and writing high-quality patches.
These guys name their bot Repairnator and have successfully tested it by allowing it to compete against human developers to bug fix. “This is a milestone for human-competitiveness in software engineering research on automatic program repair and bug fix,” they say.
Computer scientists have long known
that it is possible to automate the process of writing patches. But it is not
clear whether bots can do this work as quickly as humans and to the same
quality.
Humans have the tendency to learn new things from their mistakes and that is
called the intelligence, but bots are not yet so sophisticated to learn from
the mistakes and automatically program themselves trace the repeated
mistake. This is called Artificial Intelligence.
So Monperrus and co-tested this by disguising Repairnator as a human developer and allowing it to compete with humans to develop patches on GitHub, a version control website for software developers. “The key idea of Repairnator is to automatically generate patches that repair build failures, then to show them to human developers, to finally see whether those human developers would accept them as valid contributions to the code base,” say Monperrus and co.
The team created a GitHub user called Luc Esape, who appeared to be a software engineer at their research lab. “Luc has a profile picture and looks like a junior developer, eager to make open-source contributions on GitHub,” they say.
But Luc is actually Repairnator in disguise. This deception was necessary because human tend to assess the work of bots differently from that of humans. “This camouflage is required to test our scientific hypothesis of human competitiveness,” say Monperrus and co, who have now informed the humans involved of the ruse.
Two runs were carried out by the team to test Repairnator targeting patch making and bug fix. The first ran from February to December 2017, when the team ran Repairnator on a fixed list of 14,188 GitHub projects looking for errors. “We found that our prototype is capable of performing approximately 30 repair attempts per day,” they say.
During this time, Repairnator analyzed over 11,500 builds with failures. Of these, it was able to reproduce the failure in over 3,000 cases. It then went on to develop a patch in 15 cases.
However, none of these patches were accepted into the build because Repairnator took too long to develop them or wrote low-quality patches that could not be accepted.
The second experimental run was more successful. This time, the team set Luc to work on the Travis continuous integration service from January to June 2018. Although the team did not specify what improvements they made to Repairnator, on January 12 it wrote a patch that a human moderator accepted into a build. “In other words, Repairnator was human-competitive for the first time,” they say.
Over the next six months, Repairnator went on to produce five patches that were accepted.
That impressive work will show a new era of software development. It also raises some interesting questions. Monperrus and co point to a patch Repairnator developed for a GitHub project called “eclipse/ditto” on May 12.
The team then a received the following message from one of the developers: “We can only accept pull-requests which come from users who signed the Eclipse Foundation Contributor License Agreement.”
That raises an issue since a bot cannot physically sign a license agreement. “Who owns the intellectual property and responsibility of a bot contribution: the robot operator, the bot implementer or the repair algorithm designer?” ask Monperrus and co.
This kind of issue will have to be resolved before humans and bots can collaborate in more detail. But Monperrus and co are optimistic. “We believe that Repairnator prefigures a certain future of software development, where bots and humans will smoothly collaborate and even cooperate on software artifacts,” they say.