GPT Analyzer
Current auditing techniques for smart contracts predominantly focus on identifying standard vulnerabilities like reentrancy and integer overflows through the employment of automated analysis tools, such as Slither [1] and Mythril [2]. Yet, a recent study has uncovered that these tools detect a mere 20% of exploitable vulnerabilities in the wild, highlighting a critical deficiency in the automated vulnerability detection capabilities [3].
In the rapidly evolving landscape of software engineering, Large Language Models (LLMs) [4] have begun to play a transformative role by enhancing code generation, comprehension, and rectification processes [5]. Utilizing LLMs to augment the auditing of smart contracts offers a promising avenue for application.
Consequently, we envision the development of LLM-enhanced techniques for detecting vulnerabilities in smart contracts and introduce GPT Analyzer. This innovative tool incorporates the advanced capabilities of GPT, fine-tuned with an extensive dataset of real-world exploitable vulnerabilities, to detect a broad spectrum of sophisticated, polymorphistic logical vulnerabilities. GPT Analyzer provides several distinct advantages over traditional auditing methods:
- Enhanced Generality. While traditional automated tools rely on expertly crafted detectors based on fixed patterns of control or data flows, GPT Analyzer utilized a dataset of over 6000 real-world vulnerabilities, all of high or medium severity, sourced from esteemed auditing platforms like Code4rena [6] and Immunefi [7]. This enables GPT Analyzer to mimic human-like code interpretation and reasoning, thereby identifying a broader range of vulnerabilities, including those not previously categorized, and offering potential code repair solutions.
- Superior Assurance. The manual inspection of logical vulnerabilities often requires complex reasoning, heightening the risk of crucial vulnerabilities going unnoticed in production—a staggering 91.96% of compromised smart contracts had been thoroughly audited [8]. GPT Analyzer, however, is fine-tuned with a comprehensive dataset encompassing all seven categories of logical vulnerabilities identified by Zhang et al. [3], such as price oracle manipulation and erroneous accounting. This extensive knowledge base allows GPT Analyzer to surpass the analytical capabilities of original LLMs. Moreover, by utilizing Chain-of-Thought Prompting [9] to facilitate intermediate reasoning about detected vulnerabilities—highlighting crucial variables and functions—GPT Analyzer not only gains a deep understanding of the underlying causes of these vulnerabilities but also significantly enhances its precision by validating these findings with specific static analysis rules.
[1] Feist, Josselin, Gustavo Grieco, and Alex Groce. "Slither: a static analysis framework for smart contracts." 2019 IEEE/ACM 2nd International Workshop on Emerging Trends in Software Engineering for Blockchain (WETSEB). IEEE, 2019.
[2] Mueller, Bernhard. "Smashing ethereum smart contracts for fun and real profit." HITB SECCONF Amsterdam 9 (2018): 54.
[3] Zhang, Zhuo, et al. "Demystifying exploitable bugs in smart contracts." 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE). IEEE, 2023.
[4] "Large language model." [Online]. Available: https://en.wikipedia.org/wiki/Large_language_model
[5] Hou, Xinyi, et al. "Large Language Models for Software Engineering: A Systematic Literature Review." arXiv e-prints (2023): arXiv-2308.
[6] "Code4rena." [Online]. Available: https://code4rena.com/
[7] "Immunefi." [Online]. Available: https://immunefi.com/
[8] "Smart Contract Audits Have Failed: Can We Solve the $2.8 Billion Smart Contract Security Problem?." [Online]. Available: https://www.anchain.ai/blog/smart-contract-audits-failed
[9] Wei, Jason, et al. "Chain-of-thought prompting elicits reasoning in large language models." Advances in neural information processing systems 35 (2022): 24824-24837.
Updated 3 months ago