executable compression
{{Short description|Means of compressing an executable file}}
{{Redir|POPCOM|the music fair|Popkomm}}{{More citations needed|date=February 2024}}
{{Use dmy dates|date=May 2019|cs1-dates=y}}
{{Use list-defined references|date=December 2021}}
Executable compression is any means of compressing an executable file and combining the compressed data with decompression code into a single executable. When this compressed executable is executed, the decompression code recreates the original code from the compressed code before executing it. In most cases this happens transparently so the compressed executable can be used in exactly the same way as the original. Executable compressors are often referred to as executable packers, runtime packers, software packers, software protectors, or even "polymorphic packers" and "obfuscating tools".
A compressed executable can be considered a self-extracting archive, where a compressed executable is packaged along with the relevant decompression code in an executable file. Some compressed executables can be decompressed to reconstruct the original program file without being directly executed. Two programs that can be used to do this are CUP386 and UNP.{{Fact|date=February 2024}}
Most compressed executables decompress the original code in memory and most require slightly more memory to run (because they need to store the decompressor code, the compressed data and the decompressed code). Moreover, some compressed executables have additional requirements, such as those that write the decompressed executable to the file system before executing it.
Executable compression is not limited to binary executables, but can also be applied to scripts, such as JavaScript. Because most scripting languages are designed to work on human-readable code, which has a high redundancy, compression can be very effective and as simple as replacing long names used to identify variables and functions with shorter versions and/or removing white-space.
Advantages and disadvantages
Software distributors use executable compression for a variety of reasons, primarily to reduce the secondary storage requirements of their software; as executable compressors are specifically designed to compress executable code, they often achieve better compression ratio than standard data compression facilities such as gzip, zip or bzip2 {{Citation needed|date=November 2007}}. This allows software distributors to stay within the constraints of their chosen distribution media (such as CD-ROM, DVD-ROM, or floppy disk), or to reduce the time and bandwidth customers require to access software distributed via the Internet.
Executable compression is also frequently used to deter reverse engineering or to obfuscate the contents of the executable (for example, to hide the presence of malware from antivirus scanners) by proprietary methods of compression and/or added encryption. Executable compression can be used to prevent direct disassembly, mask string literals and modify signatures. Although this does not eliminate the chance of reverse engineering, it can make the process more costly.
A compressed executable requires less storage space in the file system, thus less time to transfer data from the file system into memory. On the other hand, it requires some time to decompress the data before execution begins. However, the speed of various storage media has not kept up with average processor speeds, so the storage is very often the bottleneck. Thus the compressed executable will load faster on most common systems. On modern desktop computers, this is rarely noticeable unless the executable is unusually big, so loading speed is not a primary reason for or against compressing an executable.
On operating systems which page executable images on demand from the disk, compressed executables make this process less efficient. The decompressor stub allocates a block of memory to hold the decompressed data, which stays allocated as long as the executable stays loaded, whether it is used or not, competing for memory resources with other applications all along. If the operating system uses a swap file, the decompressed data has to be written to it to free up the memory instead of simply discarding unused data blocks and reloading them from the executable image if needed again. This is usually not noticeable, but it becomes a problem when an executable is loaded more than once at the same time—the operating system cannot reuse data blocks it has already loaded, the data has to be decompressed into a new memory block, and will be swapped out independently if not used. The additional storage and time requirements mean that it has to be weighed carefully whether to compress executables which are typically run more than once at the same time.
Another disadvantage is that some utilities can no longer identify run-time library dependencies, as only the statically linked extractor stub is visible.
Also, some older virus scanners simply report all compressed executables as viruses because the decompressor stubs share some characteristics with those. Most modern virus scanners can unpack several different executable compression layers to check the actual executable inside, but some popular anti-virus and anti-malware scanners have had troubles with false positive alarms on compressed executables. In an attempt to solve the problem of malware obfuscated with the help of runtime packers the IEEE Industry Connections Security Group has introduced a software taggant system.
Executable compression used to be more popular when computers were limited to the storage capacity of floppy disks, which were both slow and low capacity media, and small hard drives; it allowed the computer to store more software in the same amount of space, without the inconvenience of having to manually unpack an archive file every time the user wanted to use the software. However, executable compression has become less popular because of increased storage capacity on computers. It has its use in the demoscene where demos have to stay within a size limit, e.g. 64k intro. Only very sophisticated compression formats, which add to load time, keep an executable small enough to enter these competitions.
List of executable packers
= {{anchor|PMEXE|PMSFX|POPCOM}}CP/M and MSX-DOS executable =
Known executable compressors for CP/M-80 / MSX-DOS .COM files:
= {{anchor|SM|EXEPACK|LZEXE|PKLITE|DIET|TINYPROG|RJCRUSH|APACK|32LITE|WWPACK}}MS-DOS executable =
Known executable compressors for MS-DOS-compatible executable files (.COM or .EXE):
- Realia Spacemaker (since 1982, written by Robert B. K. Dewar, SM.COM, signature "MEMORY$")
- Microsoft EXEPACK (since 1985, written by Reuben Borman, EXEPACK.EXE, LINK.EXE /E[XEPACK], signature "RB")
- LZEXE (since 1989, written by Fabrice Bellard, LZEXE.EXE)
- PKWare PKLite (since 1990, written by Phil Katz, PKLITE.EXE)
- DIET (since 1991, written by Teddy Matsumoto, DIET.EXE)
- TINYPROG (TINYPROG.EXE)
- RJS Software RJCRUSH (since 1994, written by Roland Skinner, RJCRUSH.EXE)
- XPA (since 1995, written by JauMing Tseng, XPA.EXE)
- Ibsen Software aPACK (since 1997, written by Jørgen Ibsen, APACK.EXE)
- UPX (since 1998, written by Markus F. X. J. Oberhumer and László Molnár)
- 32LiTE (since 1998, written by Oleg Prokhorov, 32LITE.EXE)
- Knowledge Dynamics LZW Compressor
- WWpack (since 1994, written by Piotr Warezak and Rafal Wierzbicki, WWPACK.EXE)
- 624 (only for .COM files smaller than 25 KB, uses LZW)
- AINEXE
- AvPack
- ComPAck
- HASP Envelope
- LGLZ
- PMWLITE
- ProPack
- UCEXE
- WDOSX
- XE
- XPack
= OS/2 executable =
Known executable compressors under OS/2:
- NeLite
- LxLite
= New Executable =
Known executable compressors for New Executables:
- PackWin
- PKWare PKLite (from version 2.01)
- WinLite
= Portable Executable =
Known executable compressors for Portable Executables:
Note: Clients in purple are no longer in development.
class="sortable wikitable" style="text-align:center" |
Name
!Latest stable !x86-64 support |
---|
{{rh}}|32Lite
| | | |
{{rh}}|Alienyze
| 1.4 ({{Start date|2020|08|17|df=yes}}) | {{proprietary}} | {{no}} |
{{rh}}|ANDpakk2
| | | |
{{rh2}}|Armadillo
| 9.62 ({{Start date|2013|06|07|df=yes}}) | {{proprietary}} | {{yes}} |
{{rh}}|ASPack
| 2.40 ({{Start date|2018|12|07|df=yes}}) | {{proprietary}} | {{yes}} |
{{rh}}|ASPR (ASProtect)
| 2.78 ({{Start date|2018|12|07|df=yes}}) | {{proprietary}} | {{yes}} |
{{rh}}|BeRoEXEPacker
| | | |
{{rh}}|BIN-crypter
| | | |
{{rh}}|BoxedApp Packer
| 3.3 ({{Start date|2015|07|26|df=yes}}) | {{proprietary}} | {{yes}} |
{{rh2}}|CExe
| 1.0b ({{Start date|2001|07|20|df=yes}}) | {{free |
| {{no}}
|-
! {{rh}}|Crinkler
| 2.3 ({{Start date|2020|07|22|df=yes}})
| {{free|}}Zlib
| {{yes}}
|-
! {{rh}}|dotBundle
| 1.3 ({{Start date|2013|04|04|df=yes}})
| {{proprietary}}
| {{yes}}
|-
! {{rh}}|Enigma Protector
| 6.60 ({{Start date|2019|08|21|df=yes}})
| {{proprietary}}
| {{yes}}
|-
! {{rh}}|Enigma Virtual Box
| 9.40 ({{Start date|2019|10|10|df=yes}})
| {{proprietary}}
| {{yes}}
|-
! {{rh}}|exe32pack
|
|
|
|-
! {{rh}}|EXE Bundle
| 3.11 ({{Start date|2011|01|07|df=yes}})
| {{proprietary}}
| {{dunno}}
|-
! {{rh}}|EXECryptor
|
|
|
|-
! {{rh}}|EXE Stealth
| 4.14 ({{Start date|2011|06|29|df=yes}})
| {{proprietary}}
| {{dunno}}
|-
! {{rh}}|eXPressor
| 1.8.0.1 ({{Start date|2010|01|14|df=yes}})
| {{proprietary}}
| {{dunno}}
|-
! {{rh2}}|FSG
| 2.0 ({{Start date|2004|05|24|df=yes}})
| {{proprietary|}}Freeware
| {{no}}
|-
! {{rh2}}|kkrunchy src
| 0.23a4 (Unknown)
| {{free|}}Public domain
| {{no}}
|-
! {{rh2}}|MEW
| 1.1 (Unknown)
| {{proprietary|}}Freeware
| {{no}}
|-
! {{rh}}|MPRESS
| 2.19 ({{Start date|2012|01|02|df=yes}})
| {{proprietary|}}Freeware
| {{yes}}
|-
! {{rh}}|MuCruncher
|
|
|
|-
! {{rh}}|NeoLite
|
|
|
|-
! {{rh}}|NsPack
|
|
|
|-
! {{rh}}|Obsidium
| 1.6 ({{Start date|2017|04|11|df=yes}})
| {{proprietary}}
| {{yes}}
|-
! {{rh}}|PECompact
|
|
|
|-
! {{rh}}|PEPack
|
|
|
|-
! {{rh}}|PESpin
| 1.33 ({{Start date|2011|05|03|df=yes}})
| {{proprietary|}}Freeware
| {{yes}}
|-
! {{rh}}|Petite
| 2.4 ({{Start date|2016|09|22|df=yes}})
| {{proprietary|}}Freeware
| {{no}}
|-
! {{rh}}|PKLite32
|
|
|
|-
! {{rh2}}|RLPack Basic
| 1.21 ({{Start date|2008|10|31|df=yes}})
| {{free|}}GPL
| {{no}}
|-
! {{rh}}|Shrinker32
|
|
|
|-
! {{rh}}|Smart Packer Pro X
| 2.0.0.1 ({{Start date|2019|06|03|df=yes}})
| {{proprietary}}
| {{yes}}
|-
! {{rh}}|Themida/WinLicense
| 3.0 ({{Start date|2019|10|24|df=yes}})
| {{proprietary}}
| {{yes}}
|-
! {{rh}}|Upack
|
|
|
|-
! {{rh}}|UPX
| 5.0.1 ({{Start date|2025|05|06|df=yes}})
| {{free|}}GPL
| {{yes}}
|-
! {{rh}}|VMProtect
| 3.4 ({{Start date|2019|08|03|df=yes}})
| {{proprietary}}
| {{yes}}
|-
! {{rh}}|WWPack32
| 1.20 ({{Start date|2000|6|19|df=yes}})
|
| {{no}}
|-
! {{rh2}}|XComp/XPack
| 0.98 ({{Start date|2007|02|18|df=yes}})
| {{proprietary|}}Freeware
| {{no}}
|-
! {{rh}}|Yoda's Crypte
|
|
|
|-
! {{rh}}|YZPack
|
|
|
|}
= ELF files =
Known executable compressors for ELF files:
- gzexe (uses a shell script stub and gzip, works on most Unix-like systems)
- HASP Envelope
- UPX
- 624 (for Linux/386)
= CLI assembly files =
Known executable compressors for CLI assembly files:
= Mac OS Classic applications =
Executable compressors for Mac OS Classic applications:
= Mach-O (Apple Mac OS X) files =
= Commodore 64 and VIC-20 =
Known executable compressors for executables on the Commodore 64 and VIC-20:
= Amiga=
Known executable compressors for executables on the Amiga series:
= Java =
= JavaScript =
There are two types of compression that can be applied to JavaScript scripts:
- Reduce the redundancy in the script (by removing comments, white space and shorten variable and functions names). This does not alter the behavior of the script.
- Compress the original script and create a new script that contains decompression code and compressed data. This is similar to binary executable compression.
== Self-decompressing compressors ==
These compress the original script and output a new script that has a decompressor and compressed data.
- JsSfx
- Packify
== Redundancy reducing compressors ==
These remove white space, remove comments, and shorten variable and function names but do not alter the behavior of the script.
- Packer
- YUI compressor
- Shrinksafe
- JSMin
See also
References
{{reflist|refs=
{{cite web |title=PMarc help manual |editor-first=Wolverine |editor-last=Logan |author-first1=Pierre |author-last1=Gielen |author-first2=Johnathan |author-last2=Taylor |date=1997 |orig-year=1993 |url=http://oldcomputers-ddns.org/public/pub/rechner/zilog/z280/source_from_tilmann_reh/cpu280-files/pmarc_4_dos/pmmanual.txt |access-date=2019-02-22 |url-status=live |archive-url=https://archive.today/20190422123539/http://oldcomputers-ddns.org/public/pub/rechner/zilog/z280/source_from_tilmann_reh/cpu280-files/pmarc_4_dos/pmmanual.txt |archive-date=22 April 2019 |quote=[…] PMEXE.CPM […] is a module […] in combination with PMARC […] used to make executable compressed COM files (just like LZEXE or PKLITE […] type: PMARC
{{cite book |title=Cambridge University Press et al v. Patton et al, Filing 124, Supplemental Initial Disclosures by Cambridge University Press, Oxford University Press, Inc., Sage Publications, Inc. - Cambridge University Press, Oxfort University Press, Inc., and Sage Publications, Inc. v. Mark P. Becker, Georgia State University President, et al, Civil Action No. 1:08-CV-1425-ODE |publisher=United States District Court For The Northern District Of Georgia, Atlanta Division |chapter=Expert Report of Robert B. K. Dewar In Response To The Report Of Kenneth D. Crews |id=Exhibit A |type=Court document |page=18 |url=https://docs.justia.com/cases/federal/district-courts/georgia/gandce/1:2008cv01425/150651/124/1.html |access-date=2019-04-23 |url-status=live |archive-url=https://web.archive.org/web/20180501180115/https://docs.justia.com/cases/federal/district-courts/georgia/gandce/1:2008cv01425/150651/124/1.html |archive-date=2018-05-01 |quote=[…] SPACEMAKER and TERMULATOR, commodity software for IBM PC (PC DOS file compression utility and VT-100 emulator), being marketed by Realia, Inc. R.B.K. Dewar (1982–1983), 8088 assembly language, 8,000 lines […]}}{{cite journal |title=If you use DOS, you need this program. |type=advertisement |date=January 1983 |author=Realia, Inc. |journal=PC Magazine |publisher=Ziff-Davis Publishing |volume=2 |number=9 |page=417 |url=https://books.google.com/books?id=vy3cBZkjbZgC&pg=RA3-PA417 |access-date=2019-04-22 |url-status=live |archive-url=https://archive.today/20190422174428/https://books.google.de/books?id=vy3cBZkjbZgC&pg=RA3-PA417&redir_esc=y%23v=onepage&q&f=false |archive-date=2019-04-22}}
{{cite web |title=Re: [fd-dev] ANNOUNCE: CuteMouse 2.0 alpha 1 |author-first=Matthias R. |author-last=Paul |work=freedos-dev |date=2002-04-11 |url=https://marc.info/?l=freedos-dev&m=101853130816682&w=2 |access-date=2020-02-21 |url-status=live |archive-url=https://web.archive.org/web/20200221130238/https://marc.info/?l=freedos-dev&m=101853130816682&w=2 |archive-date=2020-02-21 |quote=[…] > no one packer may pack combos like .SYS+.COM or .SYS+.EXE. […] There are packers for .COM or .EXE and others for .SYS, but I too have not seen a packer which supports both in one. […] possibility to combine a program/TSR and device driver in .EXE files […] and a program/TSR.COM and device driver into a .COM program […] It might also be possible to add another self-made stub to the file, after it has already been compressed […] all the compressed DR-DOS device drivers use a similar technique to let the normal PKLITE .COM decompressor work with .SYS files (meanwhile PKLITE supports a similar feature for .SYS files itself). […]}} (NB. PKLITE 1.50 (1995) and higher gained the capability to compress device drivers, but not combined COM+SYS drivers.)
{{cite web |title=Undocumented Microsoft LINK option: /E
|author-first1=Ya'akov |author-last1=Miles |author-first2=Ed |author-last2=Nather |date=1986-05-17 |orig-year=1986-02-05, 1986-02-09 |work=INFO-IBMPC mailing list |url=https://people.dsv.su.se/~jpalme/qzkom/ibmpc-8605-000511.html |access-date=2019-04-26 |url-status=live |archive-url=https://web.archive.org/web/20180501183936/https://people.dsv.su.se/~jpalme/qzkom/ibmpc-8605-000511.html |archive-date=2018-05-01 |quote=[Miles:] There exists an undocumented […] switch to Microsoft LINK.EXE […], which will cause an automatic compaction during binding. This process will eliminate storage for uninitialized arrays from the .EXE file produced by the linker […] To use this feature, specify the /E option to the command line […] [Nather:] The option does not exist in MS Link versions 3.00 and 3.01 [Miles:] By comparing the sizes of the (packed) files generated from LINK ver 3.02 and the /E option with the size of the .EXE file manually packed with […] EXEPACK, I have come to the conclusion that LINK ver 3.02 option /E generates EXACTLY the same size file as manually running EXEPACK on a regular .EXE file output by LINK […]}}
{{cite book |title=Data Compression: The Complete Reference |chapter=Chapter 3.22: EXE Compressors |author-first=David |author-last=Salomon |date=2000 |orig-year=1998 |edition=2 |publisher=Springer-Verlag |isbn=978-3-540-78086-1 |doi=10.1007/978-3-642-86092-8 |page=212 |s2cid=35889155 |chapter-url=https://books.google.com/books?id=c9OoCAAAQBAJ&pg=PA212 |access-date=2019-04-26 |archive-date=2022-04-07 |archive-url=https://web.archive.org/web/20220407170859/https://books.google.com/books?id=c9OoCAAAQBAJ&pg=PA212 |url-status=live }}
{{cite magazine |title=A Chat With Steve Kiene |author-last1=Kiene |author-first1=Steve |author-last2=Mark |author-first2=Dave |date=1999 |volume=15 |issue=4 |magazine=MacTech |url=http://www.mactech.com/articles/mactech/Vol.15/15.04/Apr99FactoryFloor/index.html |access-date=2017-12-10 |archive-date=2017-12-10 |archive-url=https://web.archive.org/web/20171210232048/http://www.mactech.com/articles/mactech/Vol.15/15.04/Apr99FactoryFloor/index.html |url-status=live }}
{{Cite web|url=http://sed.free.fr/624/|title = 624}}
{{Cite web|url=http://www.amiga-stuff.com/crunchers-download.html|title = Crunchers to download}}
{{Cite web|url=https://github.com/askeksa/Shrinkler|title=Askeksa/Shrinkler|website=GitHub|date=25 September 2021|access-date=2022-04-07 |archive-date=2022-04-07 |archive-url=https://web.archive.org/web/20220407170939/https://github.com/askeksa/Shrinkler|url-status=live}}
}}