University of Helsinki
Author: Johanna Salo, Date: 17/11/2023, Finland
Nowadays, it is impossible to go through your week without mentions of AI or this little program called Chat-GPT, unless you are living under a rock on Mars. GPTs and large language learning models (LLM) have taken over the internet in recent years in the form of chatbots and different types of generated content in media. These programs can answer (output) user-presented questions or prompts (input) by relying on their training material or by navigating the internet for an appropriate answer. There are controversies regarding the legal ownership and copyright of said training material, but that topic deserves its own blog post, and we will not be discussing it here. Instead, we will discuss the utilization of GPT’s in science!
As university students, we have all come across that one person who uses Chat-GPT to answer their homework questions and assignments in our study careers. This post is a review of sorts on using Chat-GPT as a form for “tutoring” in your chemistry studies. How reliable are the answers? Do they make any sense? Is AI going to take over the chemical industry??
Behind this post is the research paper conducted by Kan Hatakeyama-Sato, et al on “Prompt engineering of GPT-4 for chemical research: what can/cannot be done?” published in Japan, 2023. The research group inputted different chemistry-related prompts into Chat-GPT and recorded the findings. The level of difficulty varied from basic information on chemical and physical properties of input compounds all the way to designing synthesis mechanisms for block-chain polymerization and predicting properties of the unknown products. The GPT’s outputs were compared to different established algorithms’ outputs on the accuracy and the number of trials when obtaining answers.
Chat-GPT showed university textbook-level knowledge in organic chemistry and other established and well-researched fields of chemistry, but either gave confidently wrong answers or refused to answer at all when prompted with tasks centering around polymer chemistry or kinetics/rate law. Generally, the GPT demonstrated being book smart and connecting simple concepts together without separate instructions from the input but failed to apply the knowledge on a larger scale. Therefore, using Chat-GPT to answer your homework questions and assignments is redundant, as you would be using the same amount of time on correcting the GPT’s answer as you would in studying and answering the assignment on your own, for now.
The main limitations of GPT’s currently are the access to modern research that is behind a paywall and the limit on the amount of information one can include in the prompt. Another issue is the lack of GPT’s understanding of non-verbal information such as pictures and audio, as it is a language learning model. In chemistry, this mainly shows in the form of the GPT not understanding how molecules look like, or the bonding between atoms and the three-dimensional structure of them. In the paper, Chat-GPT adds in and takes away atoms that would be impossible to occur in real life, which is a huge problem if you want to sketch a reaction pathway using a GPT.
But why are we obsessed with GPT’s? The answer is simple: accessibility. Being able to give natural language instructions to a program to solve complex tasks is a huge selling point for the improvement of GPT’s, as this would lower the entry barrier for chemists who might not be code wizards and robotics experts. These aforementioned limitations could be minimized in the short term by creating plug-ins to the already established algorithms, as is already done with the field of mathematics with the existence of calculator plug-ins for Chat-GPT, allowing it to accurately perform more complex computations.
So, for now, we will still have to vigorously study theory and learn to apply concepts from micro to macro, but there could be a day when we can just give prompt requests for AI and reap the results of their labor. But until that day, please do not use Chat-GPT to answer your homework questions for you.
(This was written by a real flesh-and-bone human being. Trust me.)