The Impact GitHub Copilot Will Have on Software Development

The world’s largest host of source code, GitHub, recently announced Copilot - its controversial AI “pair programmer” that translates natural language into code. Using extensive knowledge gathered from billions of lines of open-source code, it tries to understand your syntax in order to predict what you might write next. Its functionality is similar to Gmail’s Smart Compose feature, except instead of autocompleting emails, it can generate individual lines of code or entire functions in dozens of programming languages.

Copilot is powered by Codex - a descendent of the GPT-3 language prediction model created by OpenAI. You might recognize GPT-3 from the hype it generated last year as the natural language model used to write articles, philosophize and make memes in its spare time. While GPT-3 impressed many by being a jack-of-all-trades, Codex was specifically trained to be good at one thing: coding.

I’ve been using the Copilot technical preview for the past few weeks—and while it’s not perfect, it’s one of the most impressive applications of machine learning that I’ve seen. With a few refinements, I believe it has the potential to make a big impact in the software industry in the coming years.

What can it do?

Copilot’s purpose is to read, analyze, and make code recommendations to developers from within their editor as they type. Instead of simply autocompleting things like method and variable names (tools such as IntelliSense already do this), Copilot attempts to understand the context of the file in order to provide even more comprehensive suggestions. It constantly adapts to your personal coding style based on the edits you accept or reject, so it’s always evolving to be more useful.

One of Copilot’s most mind-blowing features is its ability to transform descriptive comments into entire blocks of code. Generating a solution based only on a description of the desired logic feels almost like magic. Sometimes, it’ll even pull in an API to accomplish the task if it finds that’s what users from its training data have done too.

The “copilot” name becomes evident when working with a language or framework you may not be as familiar with. Its suggestions help to guide you along and instill more confidence in the code you’re writing. Although GitHub made it clear that Copilot isn’t meant to be used as a “search engine,” it was useful in suggesting solutions that I’d otherwise need to search for on Google. This helps to reduce time spent poring over documentation, and instead keeps you focused on working within your editor.

From my experience using it so far, Copilot has been a big time-saver. It built out entire React components for me, suggested correct types to use in TypeScript and is much more proficient in using regexes than I am. It can also autofill repetitive code if it senses a pattern, which I’ve found to be especially useful when writing JSON.

Copilot isn’t perfect and does have some notable drawbacks. Aside from occasionally getting basic syntax wrong, the code it generates doesn’t always work correctly without making further adjustments to it. It will often try to invoke functions and variables that were never defined - for example, suggesting variable names in SCSS that make total sense given the context, but didn’t actually exist anywhere in my codebase. This is due to the fact that Copilot only analyzes the context of the current file and does not pull information from other files in the codebase. As a result, Copilot tends to produce code that looks correct but may break during compilation.

Since Copilot was trained on public repositories, it’s only going to be as smart as the average GitHub contributor. Its suggestions tend to be the more popular way to accomplish a task, rather than necessarily being the best way. There’s also a chance that the code examples it was trained on could be dated or contain security vulnerabilities that the original authors may not have noticed at the time, so it’s important to have reasonable expectations for the output it generates.

It’s worth reiterating however that Copilot is still very much in its early preview stage, so I expect it to only get better.

Ethical Implications

The legality of Copilot has been hotly debated ever since its debut. Although it was trained on public repositories, certain licenses still restrict how that code can be used. In Copilot’s FAQs, it’s noted that “about 0.1% of the time, the suggestion may contain some snippets that are verbatim from the training set,” which some argue could still lead to copyright infringement, depending on the license. Even if it’s not producing identical chunks of code as its training data, some argue that it’s using derivative works whose authors should be credited.

Copilot also raises other ethical questions beyond the possible copyright issues. The idea of a software company building a commercial product off of peoples’ work has left some feeling uncomfortable about the project. Most users are likely unaware that their code was being used to train this model in the first place, although GitHub CEO Nat Friedman defended this data collection in a Twitter thread saying that “training ML systems on public data is fair use.” While this may be true, the argument can be made that users might have been more hesitant to agree to participation had they known their contributions would be repurposed.

There’s also the ongoing effort to ensure that machine learning models like Codex are trained using unbiased, non-discriminatory data. In a new paper, OpenAI has warned that there might be an inherent risk of bias and offensive content in Codex due to the nature of the data it was trained with. The team says that filters are in place to help mitigate these risks but that “the efficacy of these mitigations may scale sublinearly as more capable models are developed.” Preventing this kind of abuse is a difficult challenge for all ML models trained on public data who don’t want to end up like Microsoft’s ill-fated Tay chatbot. These models are not unlike children - they learn from what they’re exposed to, and the internet is… not always the kindest place.

Much of this is brand new legal ground that hasn’t yet been tested in court, so it will be interesting to see how policies surrounding this technology evolve in the coming years.

How will this Impact Developers?

While it might take some time for this technology to fully mature, Copilot and its inevitable competitors are bound to have a profound effect on the software industry.

Will Copilot replace developers’ jobs? Probably not, at least not anytime soon. GitHub is marketing Copilot as a productivity tool where “you’re the pilot.” Its purpose is to improve the developer experience by reducing time spent on the tedious parts of coding, while also encouraging writing better, cleaner output. Simply put, it makes writing code more fun. It should be viewed as a tool to supplement developers’ current knowledge, rather than totally replacing all human input.

One thing is for sure: the future of coding is going to involve a whole lot more reading code than writing it. Now is the time for developers who are not used to reviewing code to start getting more comfortable with it. Using Copilot effectively means you’re in a constant state of code review - every line it generates needs to be read and double-checked for errors, especially in its early stages. You’ll still need to have an understanding of what the generated code is (or isn’t) doing because just blindly accepting Copilot’s suggestions will create problems in your code that are difficult to track down.

According to Friedman, the use of AI to aid in coding is a brand-new frontier representing the “third revolution” in software development. He goes on to say “The problems we spend our days solving may change. But there will always be problems for humans to solve.”

It’s just unclear at what point down the line humans will become the copilot.

At Rightpoint, we leverage the latest technologies to offer best-in-class commerce and digital product solutions. For more information, contact us today.