ChatGPT attracted 100 million users in two months after its release. The model and ilk are changing the landscape of many applications, raising concerns, and bringing new research topics. These AI tools fall into a category of machine learning models called pretrained language models. In this talk, I will start with introducing how and why such models work by briefly discussing their principles, architectures, and recent advancement, at a high level. We will then discuss some fundamental limitations of such models. No background on machine learning and natural language processing is required for attending the talk.