ChatGPT, the artificial intelligence chatbot developed by OpenAI, has been leaving its mark on the tech world since its release in late 2022. Some view this tool with suspicion, while others see it as a great asset. As the potential of ChatGPT (and of Generative AI in general) is unveiled, experts and developers keep asking questions and experimenting with the tool. Can it crack even the strongest code protections?
At the latest RSAC in San Francisco, where Jscrambler was present this past April, obfuscated JavaScript and the possibility of ChatGPT easily reversing it were hot topics. Our team decided to take on that challenge and show that our JavaScript protection is something even the mighty ChatGPT can’t break. Let’s explain how your code is still protected with Jscrambler.
Can ChatGPT reverse Jscrambler’s obfuscation technique?
The AI chatbot is currently in version 4, the most recent one available to users. What did we find? Our test involved feeding ChatGPT an obfuscated JavaScript file from Jscrambler Code Integrity and asking it to de-obfuscate it.
First, it’s important to note that ChatGPT-4 can read JavaScript source code, understand and explain what it does, and make changes to it. However, can it do the same if the JavaScript code is obfuscated?
Testing
The following code block was provided in ChatGPT’s web interface. The source code is a short program to draw a trapezoidal prism. We minified it for testing purposes.
When ChatGPT was given minified JavaScript, it easily “reversed” the minification process by re-indenting and renaming variables based on its interpretation of the source code. Below is the output delivered by the AI tool.
However, and most importantly, when provided with the obfuscated (by Jscrambler) version of the same source code, the output differs. When unable to de-obfuscate the source code by itself, ChatGPT doesn’t provide an output answer. It’s only capable of giving general directions about what the user could do manually to try to achieve that goal.
Jscrambler’s team discovered that ChatGPT can also identify some obfuscation techniques, even if it can’t deobfuscate them.
GPT-4 can’t directly execute code, so it’s limited in its ability to handle advanced obfuscation techniques. The Jscrambler team also asked the AI tool which obfuscation techniques are the most difficult or even impossible for it to crack, and it quickly enumerated the following techniques (all of which Jscrambler currently offers):
Any attempt to automatically and fully de-obfuscate the JavaScript has failed for different reasons in different contexts:
- Fails to process inputs with a considerable size of ~1kB or greater;
- “The message you submitted was too long, please reload the conversation and submit something shorter.“
- Refuses to do it for security or legal reasons;
- “I apologize, but the code you provided appears to be obfuscated and potentially harmful. I can’t help execute or deobfuscate it. Is there anything else I can help you with?“
- Provides a manual step-by-step process for the user to do;
- Sometimes the resulting JavaScript is a completely different program (e.g., input is a program that draws a cube and output is a function to encrypt messages);
- Sometimes deletes a considerable part of the JavaScript in the process;
- Sometimes it outputs the same code.
The different results seem related to the context provided before the request to de-obfuscate the source code. For instance, it seems more successful at understanding obfuscated source code in a conversation where the source code was previously provided.
On the other hand, if a conversation with the AI tool starts with the obfuscated source code, it’s more likely to fail. Not everything is so easily achieved.
Conclusion
Our tests showed that ChatGPT-4 is a very limited de-obfuscation tool and has some interpretation limitations.
Obfuscation techniques that require programs to run (e.g., CFF, String Concealing) make it impossible for ChatGPT to interpret the source code and produce a simplified version of it. Although we weren’t able to run tests on techniques like self-defending or VM-based obfuscation, it seems safe to assume that the results wouldn’t be much different.
ChatGPT seems like a useful tool for someone looking to understand the de-obfuscation/reverse-engineering process with a more guided experience.
AI tools and artificial intelligence rise, in general, create new challenges for cybersecurity, with attackers conquering new assets to win more battles, according to an article from The Washington Post.
