Transformer Pre-trained to Be Generative architecture is the foundation of Chat GPT. This simply means that deep learning methods are used by the program to analyze and produce text. The program "understands" the intricacies of human-produced natural language using vast amounts of data from the internet.