DIFFER: A New Tool for Finding Bugs in Transformed Programs
By Michael Brown
Software transformation has become a crucial aspect of securing legacy software. However, these transformation tools often introduce new bugs and vulnerabilities into the modified programs. To address this issue, we have developed a new differential testing tool called DIFFER. In this blog post, we will provide an overview of DIFFER, discuss its application in testing debloated programs, and outline its future potential.
The Need for Post-Transformation Software Validation
Software transformation tools, such as debloaters, aim to minimize a program’s attack surface by removing unnecessary code. However, the process of cutting code can introduce new bugs and vulnerabilities. Traditional testing approaches, such as regression testing and fuzzing, do not naturally support testing transformed programs against their original versions. This limitation can allow subtle and novel bugs to find their way into the modified programs.
Introducing DIFFER: A Differential Testing Tool
DIFFER is a differential testing tool that compares the outputs of an unmodified version of a program with one or more modified variants of the program. It allows users to specify seed inputs that correspond to both unmodified and modified program behaviors and features. DIFFER then runs the original program and the transformed variants with these inputs and compares the outputs. Additionally, DIFFER supports template-based mutation fuzzing of these seed inputs to maximize coverage of the input space.
When executing the programs, DIFFER expects to see the same outputs for the original and variant programs when given inputs that correspond to unmodified features. Conversely, it expects to see different outputs when executing the programs with inputs corresponding to modified features. If DIFFER detects unexpected matching, differing, or crashing outputs, it reports them to the user, helping identify errors in the modified program resulting from the transformation process or its configuration.
Evaluating Debloated Programs with DIFFER
In a research study conducted in collaboration with GrammaTech, we used DIFFER to evaluate debloated programs created by 10 different software debloating tools. These tools were used to remove unnecessary features from 20 different programs. DIFFER discovered that 43% of the debloated variants still had features that the debloating tools failed to remove. Furthermore, 28% of the variants either crashed or produced incorrect outputs in retained features after debloating.
Future Improvements and Applications
DIFFER is an open-source tool, and we invite the security research community to use, extend, and help maintain it. We have several specific improvements planned for DIFFER, including enhancing the comparators for more advanced output checking and expanding its application to other transformation tools such as software hardening, translation, and surrogacy.
Overall, DIFFER fills a critical need in post-transformation software validation. By detecting bugs and soundness violations in transformed programs, it helps ensure the security and reliability of modified software. With the release of DIFFER as open-source software, we look forward to further advancements in the field of software transformation and testing.
This material is based on work supported by the Office of Naval Research (ONR) under Contract No. N00014-21-C-1032. Any opinions, findings and conclusions, or recommendations expressed in this material are those of the author and do not necessarily reflect the views of the ONR.
By Trail of Bits
Posted in Dynamic Analysis, Open Source
Since 2012, Trail of Bits has helped secure some of the world’s most targeted organizations and products. They combine high-end security research with a real-world attacker mentality to reduce risk and fortify code.
Read more at www.trailofbits.com
RSS – Posts
Subscribe now to keep reading and get access to the full archive.
Type your email…
Subscribe
注意
- この記事はAI(gpt-3.5-turbo)によって自動生成されたものです。
- この記事はHackerNewsに掲載された下記の記事を元に作成されています。
Differ: Tool for testing and validating transformed programs - 自動生成された記事の内容に問題があると思われる場合にはコメント欄にてご連絡ください。