dd (Unix)

In 1974, the {{code|dd}} command appeared as part of Version 5 Unix. According to Dennis Ritchie, the name is an allusion to the DD statement found in IBM's Job Control Language (JCL),{{cite newsgroup |title=Re: origin of the UNIX dd command |first=Dennis |last=Ritchie |date=Feb 17, 2004 |newsgroup=alt.folklore.computers |message-id=c0s1he$1atuh9$1@ID-156882.news.uni-berlin.de |url=https://groups.google.com/forum/#!msg/alt.folklore.computers/HAWoZ8g-xYk/HDUVxwTVLKAJ |quote=dd was always named after JCL dd cards. |access-date=January 10, 2016 |archive-date=January 22, 2011 |archive-url=http://arquivo.pt/wayback/20110122130054/https://groups.google.com/forum/#!msg/alt.folklore.computers/HAWoZ8g-xYk/HDUVxwTVLKAJ |url-status=live }}{{cite web | url=http://www.catb.org/jargon/html/D/dd.html | title=dd | first=Eric S. | last=Raymond | access-date=2008-02-19 | archive-date=2018-12-13 | archive-url=https://web.archive.org/web/20181213051044/https://www.catb.org/jargon/html/D/dd.html | url-status=live }} where DD is short for data definition.{{cite book |last1=Struble |first1=George |title=Assembler language programming: the IBM System/360 |date=1969 |publisher=Reading, Mass., Addison-Wesley Pub. Co |page=[https://archive.org/details/assemblerlanguag00stru/page/123/mode/2up?q=%22DD+statement%22+%22data+definition%22 123] |url=https://archive.org/details/assemblerlanguag00stru/page/122/mode/2up?q=%22DD+statement%22+%22data+definition%22}}{{cite newsgroup |title=Re: etymology of the Unix "dd" command |first=Barry |last=Shein |date=Apr 22, 1990 |newsgroup=alt.folklore.computers |message-id=1990Apr22.191928.11180@world.std.com |url=https://groups.google.com/d/msg/alt.folklore.computers/K4uzWBugSPE/GwSgDUEU_3wJ |access-date=2016-07-14 |archive-date=2023-10-24 |archive-url=https://web.archive.org/web/20231024083144/https://groups.google.com/g/alt.folklore.computers/c/K4uzWBugSPE/m/GwSgDUEU_3wJ |url-status=live }} According to Douglas McIlroy, {{code|dd}} was "originally intended for converting files between the ASCII, little-endian, byte-stream world of DEC computers and the EBCDIC, big-endian, blocked world of IBM"; thus, explaining the cultural context of its syntax.{{cite tech report |first1=M. D. |last1=McIlroy |author-link1=Doug McIlroy |year=1987 |url=http://www.cs.dartmouth.edu/~doug/reader.pdf |title=A Research Unix reader: annotated excerpts from the Programmer's Manual, 1971–1986 |series=CSTR |number=139 |institution=Bell Labs}} Eric S. Raymond believes "the interface design was clearly a prank", due to the command's syntax resembling a JCL statement more than other Unix commands do.

In 1987, the {{code|dd}} command is specified in the X/Open Portability Guide issue 2 of 1987{{citation needed|date=April 2025}}. This is inherited by IEEE Std 1003.1-2008 (POSIX){{citation needed|date=April 2025}}, which is part of the Single UNIX Specification.{{man|1|dd|SUS}}

In 1990, David MacKenzie announced GNU fileutils (now part of coreutils) which includes the dd command;{{cite web |title=GNU file utilities release 1.0 |url=https://groups.google.com/g/gnu.utils.bug/c/CviP42X_hCY/m/YssXFn-JrX4J |website=groups.google.com |access-date=2023-04-28 |archive-date=2023-04-28 |archive-url=https://web.archive.org/web/20230428222649/https://groups.google.com/g/gnu.utils.bug/c/CviP42X_hCY/m/YssXFn-JrX4J |url-status=live }} it was written by Paul Rubin, David MacKenzie, and Stuart Kemp.{{man|1|dd|Linux}} Since 1991, Jim Meyering is its maintainer.{{cite web |title=GNU's Who |url=https://www.gnu.org/people/people.html#j |access-date=2023-04-28 |archive-date=2023-04-28 |archive-url=https://web.archive.org/web/20230428041041/http://www.gnu.org/people/people.html#j |url-status=live }}

In 1995, Plan 9 2nd edition was released with a {{code|dd}} command with a more traditional command-line option style than the JCL statement style.{{man|1|dd|Plan 9}}

Since at least 1999, UnxUtils has provided a native implementation for the Windows platform.{{cite web |title=Native Win32 ports of some GNU utilities |url=http://www.weihenstephan.de:80/~syring/win32/UnxUtils.html |date=15 August 2000|archive-url=https://web.archive.org/web/20000815200240/http://www.weihenstephan.de:80/~syring/win32/UnxUtils.html |archive-date=2000-08-15 }}{{Cite web|url=http://unxutils.sourceforge.net/|title=Native Win32 ports of some GNU utilities|website=unxutils.sourceforge.net|access-date=2022-02-23|archive-date=2006-02-09|archive-url=https://web.archive.org/web/20060209022842/http://unxutils.sourceforge.net/|url-status=live}}

The command line interface significantly differs from most modern shell commands in that an option is formatted as option{{=}}value instead of the more typical syntax that denotes an option with a dash prefix such as: -x, -y value, --abc, --def value.

By default, {{code|dd}} reads from standard input and writes to standard output, but input and output can be overridden. Option {{code|if}} specifies an input file and option {{code|of}} specifies an output file.

Non-standardized aspects of {{code|dd}} depend on the underlying system or implementation, including:

• Direct memory access
• Signal handling
• End-of-file (EOF) handling; in particular the Windows ports vary: Cygwin uses {{key|Ctrl|D}} (the usual for Unix) and MKS Toolkit uses {{key|Ctrl|Z}} (the usual for Windows)

On completion, {{code|dd}} writes statistics to standard error. The format is standardized in POSIX.{{rp|at=STDERR}} The manual page for GNU dd does not describe this format, but the BSD manuals do. Each of the "Records in" and "Records out" lines shows the number of complete blocks transferred + the number of partial blocks, e.g. because the physical medium ended before a complete block was read, or a physical error prevented reading the complete block.

If {{code|dd}} receives a SIGINFO signal while it's running {{endash}} typically triggered by the user pressing {{key|Ctrl|T}} {{endash}} it writes intermediate statistics to standard error and continues processing.

The command processes data in blocks. The default size is 512 (the POSIX-mandated size and a common legacy size for disk hardware) but can be specified via command-line options. Option {{code|bs}} specifies the size for both input (read) and output (write) operations. Alternatively, option {{code|ibs}} specifies the size for input operations and {{code|obs}} for output operations. Option {{code|cbs}} affects conversion operations.

Options {{code|count}}, {{code|skip}} and {{code|seek}} specify a number of blocks: maximum to read, to start reading at offset from the start of the input, and to start writing at offset from the start of the output, respectively.{{rp|at=OPERANDS}}

A block size option value is specified as a whole decimal number of bytes with an optional suffix to indicate a multiplier. POSIX requires suffixes {{code|b}} (blocks) for 512 and {{code|k}} (kibibytes) for 1024,{{rp|at=OPERANDS}} but implementations differ on other suffixes. (Free) BSD uses {{code|m}} for mebibytes, {{code|g}} for gibibytes and so on for larger power of two units.{{man|1|dd|FreeBSD}} GNU uses {{code|M}} and {{code|G}} and so on for these units and uses {{code|kB}}, {{code|MB}}, and {{code|GB}} for SI units. For example, for GNU dd, {{code|1=bs=16M}} indicates a size of 16 mebibytes (16777216 bytes) and {{code|1=bs=3kB}} specifies 3000 bytes.

For POSIX compliance, some implementations interpret the {{code|x}} character as a multiplication operator for both block size and count option values. For example, {{code|bs{{=}}2x80x18b}} is interpreted as 2 × 80 × 18 × 512 = {{val|1474560|u=bytes}}, the size of a 1440 KiB floppy disk.{{rp|at=OPERANDS}} For implementations that do not support this feature, the POSIX shell arithmetic syntax of bs=$((2*80*18))b may be used.

Block size affects performance. Many small reads and writes is often slower than fewer, larger ones. On the downside, larger blocks require more RAM and can complicate error recovery.

When used with a variable block size device such as a tape drive or a network, the block size may determine the tape record size or network packet size, depending on the network protocol.

The examples below apply to many implementations, but are specifically written for GNU dd. Generally, the only difference between implementations is block size values and can be portable by using shell arithmetic expression instead of a size multiplier suffix. For example, instead of {{code|1=bs=64M}} use {{code|1=bs=$((64*1024*1024))|2=bash}} or {{code|1=bs=$((64 << 20))|2=bash}}.

The command can duplicate data across files, devices, partitions and volumes, and it can transform data during transfer as specified via option {{code|conv}}. In some cases, data transfer is faster with {{code|cat}}.

To create an ISO disk image from a CD-ROM, DVD or Blu-ray disc:{{cite web | url=https://wiki.archlinux.org/title/Optical_disc_drive#Creating_an_ISO_image_from_a_CD,_DVD,_or_BD | title=Creating an ISO image from a CD, DVD, or BD | work=ArchWiki | accessdate=April 18, 2022 | archive-date=April 18, 2022 | archive-url=https://web.archive.org/web/20220418171303/https://wiki.archlinux.org/title/Optical_disc_drive#Creating_an_ISO_image_from_a_CD,_DVD,_or_BD | url-status=live }}

blocks=$(isosize -d 2048 /dev/sr0)

dd if=/dev/sr0 of=isoimage.iso bs=2048 count=$blocks status=progress

To restore a drive from an image file:

dd if=system.img of=/dev/sdc bs=64M conv=noerror

To create an image of partition sdb2, using a 64 MiB block size:

dd if=/dev/sdb2 of=partition.image bs=64M conv=noerror

To clone one partition to another:

dd if=/dev/sda2 of=/dev/sdb2 bs=64M conv=noerror

To clone drive ad0 to ad1; ignoring any errors:

dd if=/dev/ad0 of=/dev/ad1 bs=64M conv=noerror

The command can modify data in place. For example, this overwrites the first 512 bytes of a file with null bytes:

dd if=/dev/zero of=path/to/file bs=512 count=1 conv=notrunc

Option {{code|conv{{=}}notrunc}} requests to not truncate the output file. That is, if the output file already exists, replace the specified bytes and leave the rest of the output file as-is. Without this option, the command would create an output file 512 bytes long.

The example above can also be used to backup and restore any region of a device to a file; including a master boot record. To duplicate the first two sectors of a floppy disk:

dd if=/dev/fd0 of=MBRboot.img bs=512 count=2

{{Main|Data erasure}}

For security reasons, it is sometimes necessary to have a disk wipe of a discarded device. This can be achieved by a "data transfer" from the Unix special files.

• To write zeros to a disk, use dd if=/dev/zero of=/dev/sda bs=16M.
• To write random data to a disk, use dd if=/dev/urandom of=/dev/sda bs=16M.

When compared to the data modification example above, {{code|notrunc}} conversion option is not required as it has no effect when the output file is a block device.{{cite web |url=https://stackoverflow.com/questions/20526198/why-using-conv-notrunc-when-cloning-a-disk-with-dd |title=linux - Why using conv=notrunc when cloning a disk with dd? |publisher=Stack Overflow |date=2013-12-11 |access-date=2014-03-24 |archive-date=2014-03-24 |archive-url=https://web.archive.org/web/20140324201126/http://stackoverflow.com/questions/20526198/why-using-conv-notrunc-when-cloning-a-disk-with-dd |url-status=live }}

Option {{code|bs{{=}}16M}} makes dd read and write 16 mebibytes at a time. For modern systems, an even greater block size may be faster. Note that filling the drive with random data may take longer than zeroing the drive, because the random data must be created by the CPU, while creating zeroes is very fast. On modern hard-disk drives, zeroing the drive will render most data it contains permanently irrecoverable.{{cite conference

| last1 = Wright | first1 = Craig S.

| last2 = Kleiman | first2 = Dave

| last3 = S. | first3 = Shyaam Sundhar R.

| editor1-last = Sekar | editor1-first = R.

| editor2-last = Pujari | editor2-first = Arun K.

| contribution = Overwriting Hard Drive Data: The Great Wiping Controversy

| doi = 10.1007/978-3-540-89862-7_21

| pages = 243–257

| publisher = Springer

| series = Lecture Notes in Computer Science

| title = Information Systems Security, 4th International Conference, ICISS 2008, Hyderabad, India, December 16-20, 2008. Proceedings

| volume = 5352

| year = 2008}} However, with other kinds of drives such as flash memories, much data may still be recoverable by data remanence.

Modern hard disk drives contain a Secure Erase command designed to permanently and securely erase every accessible and inaccessible portion of a drive. It may also work for some solid-state drives (flash drives). As of 2017, it does not work on USB flash drives nor on Secure Digital flash memories.{{citation needed|date=August 2017}} When available, this is both faster than using dd, and more secure.{{citation needed|date=August 2017}} On Linux machines it is accessible via the hdparm command's {{code|--security-erase-enhanced}} option.

The shred program offers multiple overwrites, as well as more secure deletion of individual files.

Data recovery involves reading from a drive with some parts potentially inaccessible. The command is a good fit with this job with its flexible skipping ({{code|seek}}) and other low-level settings. The vanilla {{code|dd}}, however, is clumsy to use as the user has to read the error messages and manually calculate the regions that can be read. The single block size also limits the granularity of the recovery, as a trade-off has to be made: either use a small one for more data recovered or use a large one for speed.

A C program called dd_rescue{{cite web|url=http://www.garloff.de/kurt/linux/ddrescue/|title=dd_rescue|work=garloff.de|access-date=2006-11-10|archive-date=2001-05-16|archive-url=https://web.archive.org/web/20010516091650/http://www.garloff.de/kurt/linux/ddrescue/|url-status=live}} was written in October 1999. It did away with the conversion functionality of {{code|dd}}, and supports two block sizes to deal with the dilemma. If a read using a large size fails, it falls back to the smaller size to gather as much as data possible. It can also run backwards. In 2003, a dd_rhelp script was written to automate the process of using dd_rescue, keeping track of what areas have been read on its own.{{cite web|url=http://www.kalysto.org/utilities/dd_rhelp/index.en.html|title=dd_rhelp author's repository|author=LAB Valentin|date=19 September 2011|quote=Important note : For some times, dd_rhelp was the only tool (AFAIK) that did this type of job, but since a few years, it is not true anymore: Antonio Diaz did write a ideal replacement for my tool: GNU 'ddrescue'.|access-date=13 May 2008|archive-date=16 May 2008|archive-url=https://web.archive.org/web/20080516231947/http://www.kalysto.org/utilities/dd_rhelp/index.en.html|url-status=live}}

In 2004, GNU wrote a separate utility, unrelated to {{code|dd}}, called ddrescue. It has a more sophisticated dynamic block-size algorithm and keeps track of what has been read internally. The authors of both dd_rescue and dd_rhelp consider it superior to their implementation.{{cite web|url=https://www.gnu.org/software/ddrescue/ddrescue.html|title=Ddrescue - GNU Project - Free Software Foundation (FSF)|work=gnu.org|access-date=2016-07-22|archive-date=2021-07-02|archive-url=https://web.archive.org/web/20210702144117/https://www.gnu.org/software/ddrescue/ddrescue.html|url-status=live}} To help distinguish the newer GNU program from the older script, alternate names are sometimes used for GNU's ddrescue, including addrescue (the name on freecode.com and freshmeat.net), gddrescue (Debian package name), and gnu_ddrescue (openSUSE package name).

Another open-source program called savehd7 uses a sophisticated algorithm, but it also requires the installation of its own programming-language interpreter.

To make drive benchmark test and analyze the sequential (and usually single-threaded) system read and write performance for 1024-byte blocks:

• Write performance: dd if=/dev/zero bs=1024 count=1000000 of=1GB_file_to_write
• Read performance: dd if=1GB_file_to_read of=/dev/null bs=1024

To make a file of 100 random bytes using the random driver:

dd if=/dev/urandom of=myrandom bs=100 count=1

To convert a file to uppercase:

dd if=filename of=filename1 conv=ucase,notrunc

On request, the command reports progress. When it receives signal {{code|USR1}} ({{code|INFO}} on BSD systems), it writes the number of transferred blocks to standard error.

The following bash script requests progress every 10 seconds until the transfer completes. The text PID stands for the {{code|dd}} process identifier.

while kill -USR1 PID ; do sleep 10 ; done

Newer versions of GNU dd support the option {{code|status{{=}}progress}} which enables periodic status feedback.{{cite web |url=https://www.gnu.org/software/coreutils/manual/html_node/dd-invocation.html |title=GNU Coreutils: dd invocation |website=The GNU Operating System and the Free Software Movement |access-date=2019-08-26 |archive-date=2019-08-22 |archive-url=https://web.archive.org/web/20190822014954/http://www.gnu.org/software/coreutils/manual/html_node/dd-invocation.html |url-status=live }}

dcfldd is a fork of GNU dd that is an enhanced version developed by Nick Harbour, who at the time was working for the United States' Department of Defense Computer Forensics Lab.{{cite web | url=http://dcfldd.sourceforge.net/ | title=DCFLDD at Source Forge | publisher=Source Forge | access-date=2013-08-17 | archive-date=2013-08-02 | archive-url=https://web.archive.org/web/20130802114625/http://dcfldd.sourceforge.net/ | url-status=live }}{{cite book |last=Jeremy Faircloth, Chris Hurley |title=Penetration Tester's Open Source Toolkit |year=2007 |publisher=Syngress |isbn=9780080556079 |pages=470–472}}{{cite book |last=Jack Wiles, Anthony Reyes |title=The Best Damn Cybercrime and Digital Forensics Book Period |year=2011 |publisher=Syngress |isbn=9780080556086 |pages=408–411}} Compared to {{code|dd}}, dcfldd allows more than one output file, supports simultaneous multiple checksum calculations, provides a verification mode for file matching, and can display the percentage progress of an operation. As of February 2024, the last release was 1.9.1 from April 2023.{{cite web | url=https://github.com/resurrecting-open-source-projects/dcfldd | title=dcfldd: Enhanced version of dd for forensics and security | publisher=GitHub | access-date=2020-11-19 | archive-date=2020-10-31 | archive-url=https://web.archive.org/web/20201031000409/https://github.com/resurrecting-open-source-projects/dcfldd | url-status=live }}

dc3dd is another fork of GNU dd from the United States Department of Defense Cyber Crime Center (DC3). It can be seen as a continuation of the dcfldd, with a stated aim of updating whenever the GNU upstream is updated. {{As of|2023|06}}, the last release was 7.3.1 from April 2023.{{cite web |title=dc3dd |url=https://sourceforge.net/projects/dc3dd/ |website=SourceForge |date=25 April 2023 |language=en |access-date=24 April 2020 |archive-date=25 February 2020 |archive-url=https://web.archive.org/web/20200225233213/https://sourceforge.net/projects/dc3dd/ |url-status=live }}

{{Portal|Free and open-source software}}

• {{Annotated link|Backup}}
• {{Annotated link|Disk cloning}}
• {{Annotated link|Disk Copy}}
• {{Annotated link|Disk image}}
• {{Annotated link|List of POSIX commands}}

{{Reflist|30em}}{{Wikibooks|Guide to Unix|Commands}}{{Optical disc image software}}

{{Unix commands}}

{{Plan 9 commands}}

{{Core Utilities commands}}

{{Backup software}}

Category:Data recovery software

Category:Disk cloning

Category:Hard disk software

Category:Standard Unix programs

Category:Unix SUS2008 utilities

Category:Plan 9 commands

Category:Inferno (operating system) commands

Category:Data erasure software