Yeah, but Data is actually intelligent and brilliant. ChatGPT is often just giving best averaged answers based on what’s most likely - and still hallucinates even in the latest version.
I haven’t used it, but it happens even in the demo they showed.
It’s great that the two characters discuss this topic, I think it’s in the multi-part TNG episode where Ambassador Spock is creating a dissident movement on Romulus.