4 Reasons why you should mistrust ChatGPT

For weeks now, people's mind is blown by ChatGPT. It can answer everything and it's seems to be a sign we won't need knowledge workers in near future. Why would I pay a lawyer, a consultant or a developer when ChatGPT can do what I want for free?

It might not be that simple.

It provides what you already know

ChatGPT relies on the information it gathered from the Internet and books to formulate its answers.

It became popular among developers for boring tasks, like finding the implementation of a well known algorithm in language X or Y. Or analyze a piece of code with regards of best practices. It works because the user has the expertise to validate produced data. Is it a revolution? It saves the time of some search through Google or Stack Overflow. While it's powerful and time saving, it's not a revolution.

It doesn't care about what you need

Can it code your application for you? Surely not. Our expertise as developers is to be able to figure out our client's actual needs out of questions. We are not coding monkeys, we need to have a deep knowledge of the problem before thinking about a solution. If they don't feel the need to contact an expert developer lots of off-the-shelf software packages, CMS or no code tools already exist.

The boring consensus

ChatGPT provides no innovative idea but a consensus. I wonder if the consensus can be attacked, for instance by putting online a large corpus of data that will represent a thesis that some group want to push. The counter measure could be to screen the data that will feed the IA. In either case, the system becomes a propagator of propaganda.

A lot of resource needed for not that much

Remember that there is no magic here. ChatGPT needs a lot of data. These data are gathered and stored on lots of servers. It also should learn how to talk, which mean that real (underpaid) humans are needed to validate the formal quality of the answers (ref).

So to summarize :

• ChatGPT is technically an incredible product, the ability to converse and adapt the formulation of the answer is truly baffling. I can't deny that.
• It's only interesting if you can validate the answer.
• It can only provide a consensus that can be artificially altered (theoretically).

Here is an interesting podcast that discuss the advancements proposed by ChatGPT if you speak French.