Running SoftICE Debugger in an Emulated MS-DOS Environment
Have you ever wanted to experience the nostalgia of using period tooling and debugging software from the past? If so, you may have come across SoftICE, a popular DOS and Windows debugger that was widely used in the late 80s and 90s. While there are several options available for reverse engineering old software today, none can quite capture the essence of using SoftICE for DOS. In this blog post, we will explore how to run SoftICE in an emulated MS-DOS environment, overcoming the limitations of existing emulators like Bochs.
Before we dive into the details, it’s important to note that the instructions provided in this blog post assume a Linux host system. While it’s unclear whether these steps will work on macOS or Windows, Linux users can follow along and enjoy the journey of reliving the past.
Setting up Bochs for SoftICE
SoftICE doesn’t work in popular emulators like DOSBox-X, but it does work in Bochs, an x86 PC emulator that provides full emulation of the CPU. However, setting up Bochs for SoftICE can be a bit tricky, especially when it comes to running it with VGA graphics mode.
When running SoftICE in Bochs while a program using a VGA graphics mode is running, the debugger’s display is not properly restored, making it invisible. To overcome this limitation, one would typically connect a second monitor to a secondary graphics adapter that provides output from an MC6845 display controller, such as an MDA/Hercules card. By issuing the command “ALTSCR ON” in SoftICE, the debugger’s display could be switched to the secondary monitor. However, Bochs does not emulate MDA or a secondary display, making it impossible to use this solution directly.
To address this issue, the author of this blog post found a discussion about adding MDA emulation to Bochs in an old SourceForge thread. Since there wasn’t a complete implementation available, the author created a fork of Bochs with the necessary changes to emulate MDA text mode. The MDA text screen is written to a named pipe as UTF-8 code points, along with ANSI escape sequences for rendering the MC6845’s character attributes. An external program then polls the named pipe and dumps the text screen on a terminal.
By following the instructions in the blog post and cloning the author’s Bochs fork, users can compile Bochs with CPU level 3 to target 386. Creating a hard disk image using bximage and configuring the .bochsrc file are also necessary steps in setting up Bochs for SoftICE. Additionally, users should be aware of a few minor annoyances, such as Bochs going straight into the debugger after picking “Begin simulation,” which can be mitigated by creating a file containing the “c” command and passing it as an argument to Bochs.
Installing MS-DOS and SoftICE
Once Bochs is set up, users can proceed with installing MS-DOS and SoftICE. The blog post provides detailed instructions on creating a floppy image and formatting it in DOS. The Oak CD-ROM driver is also installed in DOS to enable the use of CD images. Users can create an ISO image from the SoftICE 2.80 files and configure Bochs to insert the CD. After adding the SoftICE directory to the PATH environment variable in AUTOEXEC.BAT and loading SoftICE before any other driver in CONFIG.SYS, the installation process is complete.
Enabling MDA Emulation and Debugging
In order to debug games and programs that use VGA graphics mode, users need to enable the forked Bochs’s MDA emulation by creating a named pipe on the host system. The MDA emulation is enabled when Bochs finds an environment variable named MDA_PIPE containing the path to the named pipe. By starting Bochs with the MDA_PIPE option and running a program that consumes the pipe, users can bring up SoftICE by pressing Ctrl+d. The graphics display will disappear, but SoftICE will be displayed in a terminal window where the program consuming the pipe is running.
With this setup, users can now reverse engineer DOS software as if it were 1990, experiencing the joy of using SoftICE in an emulated MS-DOS environment. It’s important to note that all the software used in this process is considered de facto abandonware, as none of it is more recent than 1997. However, for those interested in exploring the history of debugging tools and experiencing the nostalgia of using period tooling, this blog post provides a comprehensive guide to running SoftICE in an emulated MS-DOS environment.
For detailed instructions and the necessary files, including the author’s Bochs fork and the SoftICE 2.80 files, please refer to the links provided in the blog post.
Reverse Engineering DOS Software as If It Were 1990