Chapter 1 - Handy Utilities
tonyi@ibm.net
| DSPACE | ZSCAN |
One of the first thing I want to explore in starting out on this adventure is a few utilities that you'll want to use for examining whatever code and data files you're working with. These are going to help us take a quick cut at locating areas to investigate. These utilities aren't comprehensive by any means, nor are they an end in themselves - they're quick and dirty tools that can help us get to what we're looking for.
The first utility is a program called DSPACE. DSPACE is pretty simple minded, but its going to reveal things about programs and data files that most programmers rarely ever think about. How often do people consider how much space their programs take up in terms of DOS(or Windows95's) disk cluster allocation scheme? Not often.
What does this mean in terms the customers can understand? It means that your programs and data are probably eating more of the customer's hard disk than they really need to. We all grumble and complain how apps these days are gargantuan and glob off anywhere from 50 to 300+ megabytes of disk space at a clip. It used to be that a 500 megabyte disk drive was the wide open spaces and nobody could imagine how they could ever fill it up. Today a 500 megabyte drive gets kind of cramped real quick.
DOS, Windows 95, and anything else that's living on a FAT file system drive today, lives with the limitations of what's known as cluster allocation units. A "cluster" is the smallest unit of disk space that can be allocated for any particular file.
Let's take the machine I started writing this book on as an example. It was a clone Pentium with a 400M drive as the C: drive formatted as FAT, another 400M drive as D: formatted as OS/2's HPFS, and another 400M drive as E:, also formatted as HPFS. I had DOS, Windows, OS/2 Warp, and Windows NT running on it. HPFS isn't afflicted with the cluster limitation, so the D: and E: aren't interesting in that regard, but the C: drive was because it is FAT.
What's the cluster size? To find out, I ran OS/2's CHKDSK utility on the C: drive and got this as a result:
The current hard disk drive is: C: The type of file system for the disk is FAT. The volume label is PCDOS_7. The Volume Serial Number is 1EFF-5CDA. 425902080 bytes total disk space. 368640 bytes in 13 hidden files. 3842048 bytes in 462 directories. 354607104 bytes in 8510 user files. 172032 bytes in extended attributes. 66912256 bytes available on disk. 8192 bytes in each allocation unit. 51990 total allocation units. 8168 available allocation units on disk. |
The number of interest here is the 8192 for the "allocation unit". This number is the cluster size currently being used on the C: drive. It looks like this particular drive was using 8K clusters.
Having an 8K cluster size means that even a one byte file is going to take 8K of my disk space!
What the DSPACE utility does is tell us how close a specified file is to a cluster boundary. DSPACE will calculate for all the possible cluster sizes between 512 bytes and 32K.
Let's see what some sample output from DSPACE looks like. I ran DSPACE on OS/2 Warp's DISKCOMP.COM utility and this is what I got:
File [c:\os2\diskcomp.com] is 40963 bytes long Cutting 8195 bytes saves a 32K cluster Cutting 8195 bytes saves a 16K cluster Cutting 3 bytes saves a 8K cluster Cutting 3 bytes saves a 4K cluster Cutting 3 bytes saves a 2K cluster Cutting 3 bytes saves a 1K cluster Cutting 3 bytes saves a 512 byte cluster ------------------------------------------------------------- |
What a bummer! If OS/2 Warp's DISKCOMP utility were just *3* bytes smaller, I'd have 8K of my disk space back. On a file that's 40,963 bytes, how hard do you suppose it would be to knock off 3 bytes so I (and millions of other users) could get 8K back? The answer of course, is: that its not too hard. Knocking 3 bytes out of any program that's more than a few hundred bytes usually isn't too hard.
Here's another example of DSPACE output. This time I ran it on the plain text DOS.H header file that comes with the Watcom version 10.0 C/C++ compiler:
File [DOS.H] is 6167 bytes long Cutting 2071 bytes saves a 4K cluster Cutting 23 bytes saves a 2K cluster Cutting 23 bytes saves a 1K cluster Cutting 23 bytes saves a 512 byte cluster ------------------------------------------------------------- |
Again a bummer. A 2K cluster was blown because of 23 bytes in a plain text file. Examining the contents of the DOS.H header file revealed several structure definitions that looked like this:
struct BlahBlah
{
unsigned foo;
unsigned bar;
unsigned blat;
unsigned greb;
};
|
The indention of the braces and variable definitions was done using blanks rather than tabs in the Watcom DOS.H. Simply doing the indentations with a single TAB character on each line would have gotten those 23 byte needed to save the 2K cluster.
OK, we've seen what DSPACE can do. Here's the code for DSPACE. DSPACE is a plain ANSI C program. You should be able to build it with any ANSI conforming C or C++ compiler.
/*-------------------------- DSPACE.C ------------------------------------------
This program figures out how big a specified file is, and then reports
on how many bytes would need to be removed from it to save a cluster
for various cluster sizes.
-----------------------------------------------------------------------------------*/
#include <stdio.h>
#include <process.h>
void main(int argc,char *argv[])
{
FILE *hInputFile;
long lFileLen;
/*
Make sure a parameter was specified
*/
if (argc < 2)
{
puts("No input file specified!");
exit(-1);
}
/*
Try to open the specified file
*/
if ((hInputFile = fopen(argv[1],"rb")) == NULL)
{
printf("Can't open file [%s]!\n", argv[1]);
exit(-1);
}
/*
Seek to the end of the specified file
*/
if (fseek(hInputFile,0L,SEEK_END))
{
fclose(hInputFile);
printf("Can't seek to the end of file [%s]!\n", argv[1]);
exit(-1);
}
/*
See how big this file is
*/
if ((lFileLen = ftell(hInputFile)) == -1L)
{
fclose(hInputFile);
printf("Can't get file pointer position for file [%s]!\n", argv[1]);
exit(-1);
}
/*
Show how big the file currently is
*/
printf("File [%s] is %ld bytes long\n", argv[1], lFileLen);
/*
Show how much needs to be removed to shrink the file's disk space
usage by a cluster.
*/
if (lFileLen > 32768L)
printf("Cutting %5ld bytes saves a 32K cluster\n", lFileLen % 32768L);
if (lFileLen > 16384L)
printf("Cutting %5ld bytes saves a 16K cluster\n", lFileLen % 16384L);
if (lFileLen > 8192L)
printf("Cutting %5ld bytes saves a 8K cluster\n", lFileLen % 8192L);
if (lFileLen > 4096L)
printf("Cutting %5ld bytes saves a 4K cluster\n", lFileLen % 4096L);
if (lFileLen > 2048L)
printf("Cutting %5ld bytes saves a 2K cluster\n", lFileLen % 2048L);
/*
These next two are likely to only be encountered on floppy disks.
*/
if (lFileLen > 1024L)
printf("Cutting %5ld bytes saves a 1K cluster\n", lFileLen % 1024L);
if (lFileLen > 512L)
printf("Cutting %5ld bytes saves a 512 byte cluster\n", lFileLen % 512L);
puts("-------------------------------------------------------------\n\n");
}
|
MOV 32_bit_register, 0 MOV 32_bit_register, -1 MOV 32_bit_register, -2 MOV 32_bit_register, 1 MOV 32_bit_register, 2 PUSH dword immediate with 4 byte constant of 0 through 127 and -1 through -128 |
Suffice to say for now, these types of instructions are special and typically lend themselves well to space reduction techniques. There's a chapter on special numbers and how to deal with them later in the book. The utility of ZSCAN will become more apparent after you get through that chapter. For now, here's the code for ZSCAN.
/*-----------------------------------------------------------------------
This program scans a specified file and grovels through it looking
for byte patterns that look like they're a MOV immediate instruction
to a register, where there may be opportunity for space savings with
XOR, OR, AND, or XOR/INC instructions.
-----------------------------------------------------------------------*/
#include <stdio.h>
#include <process.h>
/*#define REGS16BIT /* uncomment this line to process 16 bit code */
/*
These are the opcodes for immediate MOV instructions to the various
registers, and the PUSH immediate opcode.
*/
#define MOV_AX 0xB8
#define MOV_BX 0xBB
#define MOV_CX 0xB9
#define MOV_DX 0xBA
#define MOV_SI 0xBE
#define MOV_DI 0xBF
#define MOV_BP 0xBD
#define MOV_SP 0xBC
#define PUSH_IM 0x68
FILE *hInputFile;
/*
Return a string representing a register
*/
char *reg(int ch)
{
char *rc;
switch (ch)
{
case MOV_AX:
#ifdef REGS16BIT
rc = "AX";
#else
rc = "EAX";
#endif
break;
case MOV_BX:
#ifdef REGS16BIT
rc = "BX";
#else
rc = "EBX";
#endif
break;
case MOV_CX:
#ifdef REGS16BIT
rc = "CX";
#else
rc = "ECX";
#endif
break;
case MOV_DX:
#ifdef REGS16BIT
rc = "DX";
#else
rc = "EDX";
#endif
break;
case MOV_SI:
#ifdef REGS16BIT
rc = "SI";
#else
rc = "ESI";
#endif
break;
case MOV_DI:
#ifdef REGS16BIT
rc = "DI";
#else
rc = "EDI";
#endif
break;
case MOV_BP:
#ifdef REGS16BIT
rc = "BP";
#else
rc = "EBP";
#endif
break;
case MOV_SP:
#ifdef REGS16BIT
rc = "SP";
#else
rc = "ESP";
#endif
break;
default:
#ifdef REGS16BIT
rc = "??";
#else
rc = "???";
#endif
break;
}
return rc;
}
/*
Show the results for a "hit" during a scan.
*/
void Notify(int ch, long lFilePosition, int num)
{
switch (ch)
{
case PUSH_IM:
printf("Suspect 5 byte immediate PUSH dword %2d at %08lX\n",
num, lFilePosition);
break;
case MOV_AX:
case MOV_BX:
case MOV_CX:
case MOV_DX:
case MOV_SI:
case MOV_DI:
case MOV_BP:
case MOV_SP:
printf("Suspect MOV %s,%2d at %08lX\n", reg(ch), num, lFilePosition);
break;
default:
printf("*** Huh?!?! ***\n");
break;
}
}
/*
Scan the next 4 bytes in a file and see if they're an "interesting"
number. Interesting numbers are 0, 1, -1, and -2 for MOV's.
Interesting numbers for immediate PUSH are 0 thru 127 and -1 thru -128
*/
void ScanForConstant(int ch)
{
long lFilePosition;
int i,c;
union
{
short int num16;
long num32;
char bytes[4];
} u;
lFilePosition = (ftell(hInputFile) - 1L);
/*
Collect 2 or 4 bytes - return if we hit EOF
*/
#ifdef REGS16BIT
for (i=0; i<2; i++)
#else
for (i=0; i<4; i++)
#endif
{
if ((c = getc(hInputFile)) == EOF)
return;
u.bytes[i] = c;
}
/*
Now examine the value we got to see if it's interesting
*/
lFilePosition = ftell(hInputFile);
#ifdef REGS16BIT
switch (u.num16)
#else
switch (u.num32)
#endif
{
case 0L:
Notify(ch,lFilePosition,0);
break;
#ifndef REGS16BIT
/*
1's, 2's, -1's, and -2's are mostly losers in 16 bit code.
Usually, you have to just pay the price.
*/
case 1L:
Notify(ch,lFilePosition,1);
break;
case 2L:
Notify(ch,lFilePosition,2);
break;
case -1L:
Notify(ch,lFilePosition,-1);
break;
case -2L:
Notify(ch,lFilePosition,-2);
break;
#endif
default:
{
#ifndef REGS16BIT
if (ch == PUSH_IM)
{
if ((u.num32 >= -128L) && (u.num32 <= 127L))
{
Notify(ch,lFilePosition,(int)u.num32);
break;
}
}
#endif
/*
Seek back on the input stream if we struck out.
*/
#ifdef REGS16BIT
fseek(hInputFile,-2L,SEEK_CUR);
#else
fseek(hInputFile,-4L,SEEK_CUR);
#endif
break;
}
}
}
void main(int argc,char *argv[])
{
int ch;
/*
Make sure a parameter was specified
*/
if (argc < 2)
{
puts("No input file specified!");
exit(-1);
}
/*
Try to open the specified file
*/
if ((hInputFile = fopen(argv[1],"rb")) == NULL)
{
printf("Can't open file [%s]!\n", argv[1]);
exit(-1);
}
ch = getc(hInputFile);
while (ch != EOF)
{
switch (ch)
{
#ifndef REGS16BIT
case PUSH_IM:
#endif
case MOV_AX:
case MOV_BX:
case MOV_CX:
case MOV_DX:
case MOV_SI:
case MOV_DI:
case MOV_BP:
case MOV_SP:
ScanForConstant(ch);
default:
break;
}
ch = getc(hInputFile);
}
}
|
If you uncomment this line in ZSCAN and recompile it, it will search for the 16 bit register MOV instructions.
/*#define REGS16BIT /**/ |
There's less opportunity for compacting the loading constants in 16 bit code than there is in 32 bit code though, so ZSCAN deals with 32 bit code by default. ZSCAN should work OK with just about any ANSI conforming C/C++ compiler for the Intel processors.
ZSCAN will report some false positive hits when processing code because its very dumb and brute force. All we'll use it for though is as a quickie scanning tool, so this is OK. In some module with only a few hits, they may well be false positives. If you run ZSCAN and see a whole flurry of output, then there's probably quite a few legitimate hits in there and that's a module to pay more attention to.
tonyi@ibm.net
- Shut up and jump!