Anyone using Github Copilot?
Shared by ashwinsk · 119d ago · 11 comments

I've recently started testing Copilot in C++ and it seems to handle context pretty well (at least when it actually gives a result). Any thoughts on the actually real-world usability? Cause apart from helping out in generic tasks, it still requires a lot of human intervention.

Would love to get opinions from people using other languages too.

AndyDent-Touchgram · 97d ago

I just got access this week so am going to be trying it out for some C# Xamarin Forms stuff as well as cloud Python over the next few weeks.

ashwinsk · 97d ago

Should be interesting to see how Copilot handles context in a .NET environment. Do let me know of the results!

lukefrauhiger · 111d ago

I'm using it with javascript frontend frameworks (React, Next.js, Svelte, and vanilla JS) and Node/Express.

Seems like my experience is similar to the others - it still takes quite a bit of human intervention. Most of the intervention is me hitting the escape key because the suggestion wasn't very good. There are some times that it generates code good enough to use and modify, but most of the time it ends up being more efficient for me to just write all of it.

However, I do love it for easier, more repetitive things like sorting functions, comparing two lists, working with JS dates, etc. In situations like this, it's super handy to type a comment and have it spit out some code. In these straightforward use cases, it's pretty accurate, too!

The crazy part to me is that it really learns. I started using Supasebase in a recent project and CoPilot had no idea what to do with the syntax. But after a few days of working with Supabase and writing lots of CRUD actions, CoPilot could predict the syntax! It's still not all that accurate but 1/4 of the time it's close enough that I take the suggestion and modify it.

ashwinsk · 109d ago

Wait so Copilot also learns from your own codebase? Damn I need to check this out. ༼ つ ◕_◕ ༽つ

AndyDent-Touchgram · 111d ago

I'd expect it to work well on web stuff, JS and Python snippets. So it may do well for framework syntax and library discovery. Remember it's trained on the snippets from GitHub.

For algorithmic stuff in C++ or Swift, where idioms have changed dramatically in the last couple of years, I've low expectations.

The Wired story was complimentary https://www.wired.com/story/openai-copilot-autocomplete-for-code/ but cautious.

Granted, they also noticed it making errors, ranging from boneheaded to distressingly subtle. ... Karpathy has seen it generate code with subtle bugs that, depending on the context, could be anywhere from trivial to catastrophic. then talks about how it generated code with a fencepost bug. (I suspect that could as well be from being trained on buggy code!)

That distressingly subtle is the snake in the woodpile.

ashwinsk · 109d ago

I recently re-tested Copilot while solving a Codeforces puzzle against my own solution in C++. It just does not understand certain nuances even if I simplify it logically with clear variables. GPT does need a lot more work as its still prefers us to say pretty much every single detail out by which point any competent coder could just spend an extra few mins finishing it themselves.

I'm content for now that this won't take over the industry....just yet.

ikelvinstudios · 118d ago

I think it's awesome, I used it for a project and I barely wrote any logic.
Almost every code was generated by my copilot, I just had to know what I want it to do and direct my copilot with a description in the comment and may add a few adjustments then VOILA

ashwinsk · 118d ago

RIP Traditional Devs.

kendsouza · 119d ago

I tried it once..did not go back to it. It is built on GPT-3..great for copy and generating answers towards a number of things...also maybe pick up code samples from github repositories and present it you as if it generates them..no intelligence there yet:). If I want sample code, I still google it, which most of the time gets me to stackoverflow.
From a usability perspective, I rather have tried and tested code by humans on something like stackoverflow...rather than copilot pick up stock code from repositories based on context.

ashwinsk · 119d ago

StackOverFlow is definitely the holy grail for any developer out. I guess a Copilot spinoff which gets answers/suggestions directly from SOF should be way more usable. Then again, this is just me thinking.

AndyDent-Touchgram · 111d ago

This may be its real strength - it generates first-pass code which can serve as a good suggestion as to what API to use. Then go and read the docs on anything you don't know well.