Конец строки в linux и windows

The sad state of «record separators» or «line terminators» is a legacy of the dark ages of computing.

Now, we take it for granted that anything we want to represent is in some way structured data and conforms to various abstractions that define lines, files, protocols, messages, markup, whatever.

But once upon a time this wasn’t exactly true. Applications built-in control characters and device-specific processing. The brain-dead systems that required both CR and LF simply had no abstraction for record separators or line terminators. The CR was necessary in order to get the teletype or video display to return to column one and the LF (today, NL, same code) was necessary to get it to advance to the next line. I guess the idea of doing something other than dumping the raw data to the device was too complex.

Unix and Mac actually specified an abstraction for the line end, imagine that. Sadly, they specified different ones. (Unix, ahem, came first.) And naturally, they used a control code that was already «close» to S.O.P.

Since almost all of our operating software today is a descendent of Unix, Mac, or Microsoft operating software, we are stuck with the line ending confusion.

В разных операционных системах перевод строки обозначается по-разному:
GNU/Linux — \n;
Apple Macintosh (Mac) — \r;
Microsoft Windows — \r\n.

Это следует учитывать при составлении шаблонов регулярных выражений для соответствующих функции PHP и, чтобы парсинг производился правильно, можно использовать вместо них универсальную экранирующую последовательность «\R», которая соответствует любому из трёх, указанных выше, вариантов:

<?php

$string_n = "\n";
$string_r = "\r";
$string_rn = "\r\n";

var_dump( preg_match( '=\R=', $string_n ) );        // int(1)
var_dump( preg_match( '=\R=', $string_r ) );        // int(1)
var_dump( preg_match( '=\R{2}=', $string_rn ) );    // int(0)

?>

В коде выше, в последнем поиске соответствий, указан шаблон '=\R{2}=', чтобы показать, что управляющий символ «\R» соответствует последовательности «\r\n» как одному символу.

  • Перевод строки
  • Экранирующие последовательности

CR and LF

The American Standard Code for Information Interchange (ASCII) defined control-characters including CARRIAGE-RETURN (CR) and LINE-FEED (LF) that were (and still are) used to control the print-position on printers in a way analogous to the mechanical typewriters that preceded early computer printers.

Platform dependency

In Windows the traditional line-separator in text files is CR followed by LF

In old (pre OSX) Apple Macintosh systems the traditional line separator in text files was CR

In Unix and Linux, the traditional line-separator in text files is LF.

\n and \r

In many programming and scripting languages \n means «new line». Sometimes (but not always) this means the ASCII LINE-FEED character (LF), which, as you say, moves the cursor (or print position) down one line. In a printer or typewriter, this would actually move the paper up one line.

Invariably \r means the ASCII CARRIAGE-RETURN character (CR) whose name actually comes from mechanical typewriters where there was a carriage-return key that caused the roller («carriage») that carried the paper to move to the right, powered by a spring, as far as it would go. Thus setting the current typing position to the left margin.

Programming

In some programming languages \n can mean a platform-dependent sequence of characters that end or separate lines in a text file. For example in Perl, print "\n" produces a different sequence of characters on Linux than on Windows.

In Java, best practise, if you want to use the native line endings for the runtime platform, is not to use \n or \r at all. You should use System.getProperty("line.separator"). You should use \n and \r where you want LF and CR regardless of platform (e.g. as used in HTTP, FTP and other Internet communications protocols).

Unix stty

In a Unix shell, the stty command can be used to cause the shell to translate between these various conventions. For example stty -onlcr will cause the shell to subsequently translate all outgoing LFs to CR LF.

Linux and OSX follow Unix conventions

Text files

Text files are still enormously important and widely used. For example, HTML and XML are examples of text file. Most of the important Internet protocols, such as HTTP, follow text-file conventions and include specifications for line-endings.

Printers

Most printers other than the very cheapest, still respect CR and LF. In fact they are fundamental to the most widely used page description languages — PCL and Postscript.

Newline inserted between the words «Hello» and «world»

A newline (frequently called line ending, end of line (EOL), next line (NEL) or line break) is a control character or sequence of control characters in character encoding specifications such as ASCII, EBCDIC, Unicode, etc. This character, or a sequence of characters, is used to signify the end of a line of text and the start of a new one.[1]

History[edit]

In the mid-1800s, long before the advent of teleprinters and teletype machines, Morse code operators or telegraphists invented and used Morse code prosigns to encode white space text formatting in formal written text messages. In particular the Morse prosign BT (mnemonic break text) represented by the concatenation of literal textual Morse codes «B» and «T» characters sent without the normal inter-character spacing is used in Morse code to encode and indicate a new line or new section in a formal text message.

Later, in the age of modern teleprinters, standardized character set control codes were developed to aid in white space text formatting. ASCII was developed simultaneously by the International Organization for Standardization (ISO) and the American Standards Association (ASA), the latter being the predecessor organization to American National Standards Institute (ANSI). During the period of 1963 to 1968, the ISO draft standards supported the use of either CR+LF or LF alone as a newline, while the ASA drafts supported only CR+LF.

The sequence CR+LF was commonly used on many early computer systems that had adopted Teletype machines—typically a Teletype Model 33 ASR—as a console device, because this sequence was required to position those printers at the start of a new line. The separation of newline into two functions concealed the fact that the print head could not return from the far right to the beginning of the next line in time to print the next character. Any character printed after a CR would often print as a smudge in the middle of the page while the print head was still moving the carriage back to the first position. «The solution was to make the newline two characters: CR to move the carriage to column one, and LF to move the paper up.»[2] In fact, it was often necessary to send extra characters—extraneous CRs or NULs—which are ignored but give the print head time to move to the left margin. Many early video displays also required multiple character times to scroll the display.

On such systems, applications had to talk directly to the Teletype machine and follow its conventions since the concept of device drivers hiding such hardware details from the application was not yet well developed. Therefore, text was routinely composed to satisfy the needs of Teletype machines. Most minicomputer systems from DEC used this convention. CP/M also used it in order to print on the same terminals that minicomputers used. From there MS-DOS (1981) adopted CP/M’s CR+LF in order to be compatible, and this convention was inherited by Microsoft’s later Windows operating system.

The Multics operating system began development in 1964 and used LF alone as its newline. Multics used a device driver to translate this character to whatever sequence a printer needed (including extra padding characters), and the single byte was more convenient for programming. What seems like a more obvious choice—CR—was not used, as CR provided the useful function of overprinting one line with another to create boldface, underscore and strikethrough effects. Perhaps more importantly, the use of LF alone as a line terminator had already been incorporated into drafts of the eventual ISO/IEC 646 standard. Unix followed the Multics practice, and later Unix-like systems followed Unix. This created conflicts between Windows and Unix-like operating systems, whereby files composed on one operating system could not be properly formatted or interpreted by another operating system (for example a UNIX shell script written in a Windows text editor like Notepad[3][4]).

Representation[edit]

The concepts of carriage return (CR) and line feed (LF) are closely associated and can be considered either separately or together. In the physical media of typewriters and printers, two axes of motion, «down» and «across», are needed to create a new line on the page. Although the design of a machine (typewriter or printer) must consider them separately, the abstract logic of software can combine them together as one event. This is why a newline in character encoding can be defined as CR and LF combined into one (commonly called CR+LF or CRLF).

Some character sets provide a separate newline character code. EBCDIC, for example, provides an NL character code in addition to the CR and LF codes. Unicode, in addition to providing the ASCII CR and LF control codes, also provides a «next line» (NEL) control code, as well as control codes for «line separator» and «paragraph separator» markers.

  • EBCDIC systems—mainly IBM mainframe systems, including z/OS (OS/390) and IBM i (OS/400)—use NL (New Line, 0x15)[8] as the character combining the functions of line feed and carriage return. The equivalent Unicode character (0x85) is called NEL (Next Line). EBCDIC also has control characters called CR and LF, but the numerical value of LF (0x25) differs from the one used by ASCII (0x0A). Additionally, some EBCDIC variants also use NL but assign a different numeric code to the character. However, those operating systems use a record-based file system, which stores text files as one record per line. In most file formats, no line terminators are actually stored.
  • Operating systems for the CDC 6000 series defined a newline as two or more zero-valued six-bit characters at the end of a 60-bit word. Some configurations also defined a zero-valued character as a colon character, with the result that multiple colons could be interpreted as a newline depending on position.
  • RSX-11 and OpenVMS also use a record-based file system, which stores text files as one record per line. In most file formats, no line terminators are actually stored, but the Record Management Services facility can transparently add a terminator to each line when it is retrieved by an application. The records themselves can contain the same line terminator characters, which can either be considered a feature or a nuisance depending on the application. RMS not only stores records, but also stores metadata about the record separators in different bits for the file to complicate matters even more (since files can have fixed length records, records that are prefixed by a count or records that are terminated by a specific character). The bits are not generic, so while they can specify that CRLF or LF or even CR is the line terminator, they can not substitute some other code.
  • Fixed line length was used by some early mainframe operating systems. In such a system, an implicit end-of-line was assumed every 72 or 80 characters, for example. No newline character was stored. If a file was imported from the outside world, lines shorter than the line length had to be padded with spaces, while lines longer than the line length had to be truncated. This mimicked the use of punched cards, on which each line was stored on a separate card, usually with 80 columns on each card, often with sequence numbers in columns 73–80. Many of these systems added a carriage control character to the start of the next record; this could indicate whether the next record was a continuation of the line started by the previous record, or a new line, or should overprint the previous line (similar to a CR). Often this was a normal printing character such as # that thus could not be used as the first character in a line. Some early line printers interpreted these characters directly in the records sent to them.

Communication protocols[edit]

Many communications protocols have some sort of new line convention. In particular, protocols published by the Internet Engineering Task Force (IETF) typically use the ASCI CRLF sequence.

In some older protocols, the new line may be followed by a checksum or parity character.

Unicode[edit]

«Paragraph separator» redirects here. For the symbol also known as a «paragraph sign», see Pilcrow.

The Unicode standard defines a number of characters that conforming applications should recognize as line terminators:[9]

 LF:    Line Feed, U+000A
 VT:    Vertical Tab, U+000B
 FF:    Form Feed, U+000C
 CR:    Carriage Return, U+000D
 CR+LF: CR (U+000D) followed by LF (U+000A)
 NEL:   Next Line, U+0085
 LS:    Line Separator, U+2028
 PS:    Paragraph Separator, U+2029

This may seem overly complicated compared to an approach such as converting all line terminators to a single character, for example LF. However, Unicode was designed to preserve all information when converting a text file from any existing encoding to Unicode and back. Therefore, Unicode should contain characters included in existing encodings.

For example: NL is part of EBCDIC, which uses code 0x15; it is normally mapped to Unicode NEL, 0x85, which is a control character in the C1 control set.[10] As such, it is defined by ECMA 48,[11] and recognized by encodings compliant with ISO/IEC 2022 (which is equivalent to ECMA 35).[12] C1 control set is also compatible with ISO-8859-1.[citation needed] The approach taken in the Unicode standard allows round-trip transformation to be information-preserving while still enabling applications to recognize all possible types of line terminators.

Recognizing and using the newline codes greater than 0x7F (NEL, LS and PS) is not often done. They are multiple bytes in UTF-8, and the code for NEL has been used as the ellipsis () character in Windows-1252. For instance:

  • ECMAScript accepts LS and PS as line breaks,[13] but considers U+0085 (NEL) whitespace instead of a line break.[14]
  • Windows 10 does not treat any of NEL, LS, or PS as line breaks in its default text editor, Notepad.
  • gedit, the default text editor of the GNOME desktop environment, treats LS and PS, but not NEL, as newlines.
  • JSON[15] allows LS and PS characters within strings, while ECMAScript prior to ES2019[16][17] treated them as newlines, and therefore illegal syntax.[18]
  • YAML[19] no longer recognizes them as special as of version 1.2, in order to be compatible with JSON.

The Unicode special characters U+2424 (SYMBOL FOR NEWLINE, ), U+23CE (RETURN SYMBOL, ), U+240D (SYMBOL FOR CARRIAGE RETURN, ) and U+240A (SYMBOL FOR LINE FEED, ) are glyphs intended for presenting a user-visible character to the reader of the document, and are thus not recognized themselves as a newline.

In programming languages[edit]

To facilitate the creation of portable programs, programming languages provide some abstractions to deal with the different types of newline sequences used in different environments.

The C programming language provides the escape sequences ‘\n’ (newline) and ‘\r’ (carriage return). However, these are not required to be equivalent to the ASCII LF and CR control characters. The C standard only guarantees two things:

  1. Each of these escape sequences maps to a unique implementation-defined number that can be stored in a single char value.
  2. When writing to a file, device node, or socket/fifo in text mode, ‘\n’ is transparently translated to the native newline sequence used by the system, which may be longer than one character. When reading in text mode, the native newline sequence is translated back to ‘\n’. In binary mode, no translation is performed, and the internal representation produced by ‘\n’ is output directly.

On Unix platforms, where C originated, the native newline sequence is ASCII LF (0x0A), so ‘\n’ was simply defined to be that value. With the internal and external representation being identical, the translation performed in text mode is a no-op, and Unix has no notion of text mode or binary mode. This has caused many programmers who developed their software on Unix systems simply to ignore the distinction completely, resulting in code that is not portable to different platforms.

The C library function fgets() is best avoided in binary mode because any file not written with the Unix newline convention will be misread. Also, in text mode, any file not written with the system’s native newline sequence (such as a file created on a Unix system, then copied to a Windows system) will be misread as well.

Another common problem is the use of ‘\n’ when communicating using an Internet protocol that mandates the use of ASCII CR+LF for ending lines. Writing ‘\n’ to a text mode stream works correctly on Windows systems, but produces only LF on Unix, and something completely different on more exotic systems. Using «\r\n» in binary mode is slightly better.

Many languages, such as C++, Perl,[20] and Haskell provide the same interpretation of ‘\n’ as C. C++ has an alternative I/O model where the manipulator std::endl can be used to output a newline (and flushes the stream buffer).

Java, PHP,[21] and Python[22] provide the ‘\r\n’ sequence (for ASCII CR+LF). In contrast to C, these are guaranteed to represent the values U+000D and U+000A, respectively.

The Java I/O libraries do not transparently translate these into platform-dependent newline sequences on input or output. Instead, they provide functions for writing a full line that automatically add the native newline sequence, and functions for reading lines that accept any of CR, LF, or CR+LF as a line terminator (see BufferedReader.readLine()). The System.lineSeparator() method can be used to retrieve the underlying line separator.

Example:

   String eol = System.lineSeparator();
   String lineColor = "Color: Red" + eol;

Python permits «Universal Newline Support» when opening a file for reading, when importing modules, and when executing a file.[23]

Some languages have created special variables, constants, and subroutines to facilitate newlines during program execution. In some languages such as PHP and Perl, double quotes are required to perform escape substitution for all escape sequences, including ‘\n’ and ‘\r’. In PHP, to avoid portability problems, newline sequences should be issued using the PHP_EOL constant.[24]

Example in C#:

   string eol = Environment.NewLine;
   string lineColor = "Color: Red" + eol;
   
   string eol2 = "\n";
   string lineColor2 = "Color: Blue" + eol2;

Issues with different newline formats[edit]

A text file created with gedit and viewed with a hex editor. Besides the text objects, there are only EOL markers with the hexadecimal value 0A.

The different newline conventions cause text files that have been transferred between systems of different types to be displayed incorrectly.

Text in files created with programs which are common on Unix-like or classic Mac OS, appear as a single long line on most programs common to MS-DOS and Microsoft Windows because these do not display a single line feed or a single carriage return as a line break.

Conversely, when viewing a file originating from a Windows computer on a Unix-like system, the extra CR may be displayed as a second line break, as ^M, or as <cr> at the end of each line.

Furthermore, programs other than text editors may not accept a file, e.g. some configuration file, encoded using the foreign newline convention, as a valid file.

The problem can be hard to spot because some programs handle the foreign newlines properly while others do not. For example, a compiler may fail with obscure syntax errors even though the source file looks correct when displayed on the console or in an editor. Modern text editors generally recognize all flavours of CR+LF newlines and allow users to convert between the different standards. Web browsers are usually also capable of displaying text files and websites which use different types of newlines.

Even if a program supports different newline conventions, these features are often not sufficiently labeled, described, or documented. Typically a menu or combo-box enumerating different newline conventions will be displayed to users without an indication if the selection will re-interpret, temporarily convert, or permanently convert the newlines. Some programs will implicitly convert on open, copy, paste, or save—often inconsistently.

Most textual Internet protocols (including HTTP, SMTP, FTP, IRC, and many others) mandate the use of ASCII CR+LF (‘\r\n’, 0x0D 0x0A) on the protocol level, but recommend that tolerant applications recognize lone LF (‘\n’, 0x0A) as well. Despite the dictated standard, many applications erroneously use the C newline escape sequence ‘\n’ (LF) instead of the correct combination of carriage return escape and newline escape sequences ‘\r\n’ (CR+LF) (see section Newline in programming languages above). This accidental use of the wrong escape sequences leads to problems when trying to communicate with systems adhering to the stricter interpretation of the standards instead of the suggested tolerant interpretation. One such intolerant system is the qmail mail transfer agent that actively refuses to accept messages from systems that send bare LF instead of the required CR+LF.[25]

The standard Internet Message Format[26] for email states: «CR and LF MUST only occur together as CRLF; they MUST NOT appear independently in the body».

The File Transfer Protocol can automatically convert newlines in files being transferred between systems with different newline representations when the transfer is done in «ASCII mode». However, transferring binary files in this mode usually has disastrous results: any occurrence of the newline byte sequence—which does not have line terminator semantics in this context, but is just part of a normal sequence of bytes—will be translated to whatever newline representation the other system uses, effectively corrupting the file. FTP clients often employ some heuristics (for example, inspection of filename extensions) to automatically select either binary or ASCII mode, but in the end it is up to users to make sure their files are transferred in the correct mode. If there is any doubt as to the correct mode, binary mode should be used, as then no files will be altered by FTP, though they may display incorrectly.[27]

Conversion between newline formats[edit]

Text editors are often used for converting a text file between different newline formats; most modern editors can read and write files using at least the different ASCII CR/LF conventions.

For example, the editor Vim can make a file compatible with the Windows Notepad text editor. Within vim

Editors can be unsuitable for converting larger files or bulk conversion of many files. For larger files (on Windows NT/2000/XP) the following command is often used:

D:\>TYPE unix_file | FIND /V "" > dos_file

Special purpose programs to convert files between different newline conventions include unix2dos and dos2unix, mac2unix and unix2mac, mac2dos and dos2mac, and flip.[28]
The tr command is available on virtually every Unix-like system and can be used to perform arbitrary replacement operations on single characters. A DOS/Windows text file can be converted to Unix format by simply removing all ASCII CR characters with

$ tr -d '\r' < inputfile > outputfile

or, if the text has only CR newlines, by converting all CR newlines to LF with

$ tr '\r' '\n' < inputfile > outputfile

The same tasks are sometimes performed with awk, sed, or in Perl if the platform has a Perl interpreter:

$ awk '{sub("$","\r\n"); printf("%s",$0);}' inputfile > outputfile  # UNIX to DOS  (adding CRs on Linux and BSD based OS that haven't GNU extensions)
$ awk '{gsub("\r",""); print;}' inputfile > outputfile              # DOS to UNIX  (removing CRs on Linux and BSD based OS that haven't GNU extensions)
$ sed -e 's/$/\r/' inputfile > outputfile              # UNIX to DOS  (adding CRs on Linux based OS that use GNU extensions)
$ sed -e 's/\r$//' inputfile > outputfile              # DOS  to UNIX (removing CRs on Linux based OS that use GNU extensions)
$ perl -pe 's/\r?\n|\r/\r\n/g' inputfile > outputfile  # Convert to DOS
$ perl -pe 's/\r?\n|\r/\n/g'   inputfile > outputfile  # Convert to UNIX
$ perl -pe 's/\r?\n|\r/\r/g'   inputfile > outputfile  # Convert to old Mac

The file command can identify the type of line endings:

 $ file myfile.txt
 myfile.txt: ASCII English text, with CRLF line terminators

The Unix egrep (extended grep) command can be used to print filenames of Unix or DOS files (assuming Unix and DOS-style files only, no classic Mac OS-style files):

$ egrep -L '\r\n' myfile.txt # show UNIX style file (LF terminated)
$ egrep -l '\r\n' myfile.txt # show DOS style file (CRLF terminated)

Other tools permit the user to visualise the EOL characters:

$ od -a myfile.txt
$ cat -e myfile.txt
$ cat -v myfile.txt
$ hexdump -c myfile.txt

Interpretation[edit]

Two ways to view newlines, both of which are self-consistent, are that newlines either separate lines or that they terminate lines. If a newline is considered a separator, there will be no newline after the last line of a file. Some programs have problems processing the last line of a file if it is not terminated by a newline. On the other hand, programs that expect newline to be used as a separator will interpret a final newline as starting a new (empty) line. Conversely, if a newline is considered a terminator, all text lines including the last are expected to be terminated by a newline. If the final character sequence in a text file is not a newline, the final line of the file may be considered to be an improper or incomplete text line, or the file may be considered to be improperly truncated.

In text intended primarily to be read by humans using software which implements the word wrap feature, a newline character typically only needs to be stored if a line break is required independent of whether the next word would fit on the same line, such as between paragraphs and in vertical lists. Therefore, in the logic of word processing and most text editors, newline is used as a paragraph break and is known as a «hard return», in contrast to «soft returns» which are dynamically created to implement word wrapping and are changeable with each display instance. In many applications a separate control character called «manual line break» exists for forcing line breaks inside a single paragraph. The glyph for the control character for a hard return is usually a pilcrow (¶), and for the manual line break is usually a carriage return arrow (↵).

Reverse and partial line feeds[edit]

RI (U+008D REVERSE LINE FEED,[29] ISO/IEC 6429 8D, decimal 141) is used to move the printing position back one line (by reverse feeding the paper, or by moving a display cursor up one line) so that other characters may be printed over existing text. This may be done to make them bolder, or to add underlines, strike-throughs or other characters such as diacritics.

Similarly, PLD (U+008B PARTIAL LINE FORWARD, decimal 139) and PLU (U+008C PARTIAL LINE BACKWARD, decimal 140) can be used to advance or reverse the text printing position by some fraction of the vertical line spacing (typically, half). These can be used in combination for subscripts (by advancing and then reversing) and superscripts (by reversing and then advancing), and may also be useful for printing diacritics.

See also[edit]

  • End-of-file
  • Enter key
  • Page break

References[edit]

  1. ^ «What is a Newline?». www.computerhope.com. Retrieved 10 May 2021.
  2. ^ Qualline, Steve (2001). Vi Improved — Vim (PDF). Sams Publishing. p. 120. ISBN 9780735710016. Archived from the original (PDF) on 8 April 2022. Retrieved 4 January 2023.
  3. ^ Duckett, Chris. «Windows Notepad finally understands everyone else’s end of line characters». ZDNet. Archived from the original on 13 May 2018. Retrieved 4 January 2023. [A]fter decades of frustration, and having to download a real text editor to change a single line in a config file from a Linux box, Microsoft has updated Notepad to be able to handle end of line characters used in Unix, Linux, and macOS environments.
  4. ^ Lopez, Michel (8 May 2018). «Introducing extended line endings support in Notepad». Windows Command Line. Archived from the original on 6 April 2019. Retrieved 4 January 2023. As with any change to a long-established tool, there’s a chance that this new behavior may not work for your scenarios, or you may prefer to disable this new behavior and return to Notepad’s original behavior. To do this, you can change […registry keys…] to tweak how Notepad handles pasting of text, and which EOL character to use when Enter/Return is hit
  5. ^ «ASCII Chart».
  6. ^ Bray, Andrew C.; Dickens, Adrian C.; Holmes, Mark A. (1983). The Advanced User Guide for the BBC Microcomputer (PDF). pp. 103, 104. ISBN 978-0946827008. Retrieved 30 January 2019.
  7. ^ «RISC OS 3 Programmers’ Reference Manual». Retrieved 18 July 2018.
  8. ^ IBM System/360 Reference Data Card, Publication GX20-1703, IBM Data Processing Division, White Plains, NY
  9. ^ «UAX #14: Unicode Line Breaking Algorithm». www.unicode.org.
  10. ^ «C1 Control Character Set of ISO 6429» (PDF). ITSCJ. IPSJ. 1 October 1983. Retrieved 3 March 2022.
  11. ^ Control Functions for Coded Character Sets (PDF) (Report). ECMA International. June 1991.
  12. ^ Character Code Structure and Extension Techniques (PDF) (Report) (6th ed.). ECMA International. December 1994.
  13. ^ «ECMAScript 2019 Language Specification». ECMA International. June 2019. 11.3 Line Terminators.
  14. ^ «ECMAScript 2019 Language Specification». ECMA International. June 2019. 11.2 White Space.
  15. ^ Bray, Tim (March 2014). «Strings». The JavaScript Object Notation (JSON) Data Interchange Format. sec. 7. doi:10.17487/RFC7159. RFC 7159.
  16. ^ «Subsume JSON (a.k.a. JSON ⊂ ECMAScript)». GitHub. 22 May 2018.
  17. ^ «ECMAScript 2019 Language Specification». ECMA International. June 2019. 11.8.4 String Literals.
  18. ^ «ECMAScript 2018 Language Specification». ECMA International. June 2018. 11.8.4 String Literals.
  19. ^ «YAML Ain’t Markup Language (YAML) Version 1.2». yaml.org. 5.4. Line Break Characters.
  20. ^ «binmode — perldoc.perl.org». perldoc.perl.org.
  21. ^ «PHP: Strings — Manual». www.php.net.
  22. ^ «Lexical analysis – Python v3.11 documentation». docs.python.org.
  23. ^ «What’s new in Python 2.3».
  24. ^ «PHP: Predefined Constants — Manual». www.php.net.
  25. ^ Bernstein, D.J. «Bare LFs in SMTP».
  26. ^ Resnick, Pete (April 2001). Internet Message Format. doi:10.17487/RFC2822. RFC 2822.
  27. ^ «File Transfer». Archived from the original on 14 May 2016. When in doubt, transfer in binary mode.
  28. ^ «ASCII text conversion between UNIX, Macintosh, MS-DOS». Archived from the original on 9 February 2009.
  29. ^ «C1 Controls and Latin-1 Supplement» (PDF). unicode.org. Retrieved 13 February 2016.

External links[edit]

  • The Unicode reference; see paragraph 5.8 in Chapter 5 of the Unicode 4.0 standard (PDF)
  • «The [NEL] Newline Character».
  • The End of Line Puzzle
  • Line counter based on Newline Character
  • Understanding Newlines at the Wayback Machine (archived 20 August 2006)
  • «The End-of-Line Story»

Yes yes, I am aware that '\n' writes a newline in UNIX while for Windows there is the two character sequence: '\r\n'. All this is very nice in theory, but my question is why? Why the carriage return character is extra in Windows? If UNIX can do it in \n why does it take Windows two characters to do this?

I am reading David Beazley’s Python book and he says:

For example, on Windows, writing the
character ‘\n’ actually outputs the
two- character sequence ‘\r\n’ (and
when reading the file back, ‘\r\n’ is
translated back into a single ‘\n’
character).

Why the extra effort?

I will be honest. I have known the difference for a long time but have never bothered to ask WHY. I hope that is answered today.

Thanks for your time.

asked Dec 22, 2010 at 11:38

sukhbir's user avatar

6

Backward compatibility.

Windows is backward compatible with MS-DOS (aggressively so, even) and MS-DOS used the CR-LF convention because MS-DOS was compatible with CP/M-80 (somewhat by accident) which used the CR-LF convention because that was how you drove a printer (because printers were originally computer controlled typewriters).

Printers have a separate command to move the paper up one line to a new line, and a separate command for returning the carriage (where the paper was mounted) back to the left margin.

That’s why. And, yes, it is an annoyance, but it is part of the package deal that allowed MS-DOS to win over CP/M, and Windows 95 to win over all the other GUI’s on top of DOS, and Windows XP to take over from Windows 98.

(Note: Modern laser printers still have these commands because they too are backwards compatible with earlier printers — HP in particular do this well)

For those unfamiliar with typewriters, here is a video showing how typing was done: http://www.youtube.com/watch?v=LJvGiU_UyEQ. Notice that the paper is first moved up, and then the carriage is returned, even if it happens in a simple movement. The ding notified the typist that the end was near, and to prepare for it.

answered Dec 22, 2010 at 12:10

11

As far as I’m aware this harks back to the days of typewriters.

\r is carriage return, which is what moves where you are typing on the page back to the left (or right if that is your culture)

\n is new line, which moves your paper up a line.

Doing only one of these on a typewriter would put you in the wrong place to start writing a new line of text.

When computers came about I guess some people kept the old model, but others realised that it wasn’t necessary and encapsulated a full newline as one character.

answered Dec 22, 2010 at 11:45

Matt Ellen's user avatar

Matt EllenMatt Ellen

3,3684 gold badges31 silver badges37 bronze badges

11

I don’t know if this is common knowledge, but it should be noted that CR is still understood by modern terminal emulators:

$ printf "hey world\rsup\n"
sup world

It’s handy for progress indicators, e.g.

for i in {1..100}
do
    printf "\rLoading... %d%%" $i
    sleep 0.01
done
echo

answered Jul 2, 2011 at 8:01

Daniel Lubarov's user avatar

1

History of the Newline Character (Wikipedia):

ASCII was developed simultaneously by the ISO and the ASA, the predecessor organization to ANSI. During the period of 1963–1968, the ISO draft standards supported the use of either CR+LF or LF alone as a newline, while the ASA drafts supported only CR+LF.

The sequence CR+LF was in common use on many early computer systems that had adopted teletype machines, typically an ASR33, as a console device, because this sequence was required to position those printers at the start of a new line. On these systems, text was often routinely composed to be compatible with these printers, since the concept of device drivers hiding such hardware details from the application was not yet well developed; applications had to talk directly to the teletype machine and follow its conventions.

The separation of the two functions concealed the fact that the print head could not return from the far right to the beginning of the next line in one-character time. That is why the sequence was always sent with the CR first. In fact, it was often necessary to send extra characters (extraneous CRs or NULs, which are ignored) to give the print head time to move to the left margin.

Even after teletypes were replaced by computer terminals with higher baud rates, many operating systems still supported automatic sending of these fill characters, for compatibility with cheaper terminals that required multiple character times to scroll the display.

MS-DOS (1981) adopted CP/M’s CR+LF; CP/M’s use of CR+LF made sense for using computer terminals via serial lines. This convention was inherited by Microsoft’s later Windows operating system.

The Multics operating system began development in 1964 and used LF alone as its newline. Unix followed the Multics practice, and later systems followed Unix.

Community's user avatar

answered Dec 22, 2010 at 13:59

Craige's user avatar

CraigeCraige

3,79121 silver badges30 bronze badges

1

Historically, line feed meant that the platen — the roller on which you type — rotated one line, causing text to appear on the next line… but in the next column.

Carriage return meant «return the bit with which you type to the beginning of the line».

Windows uses CR+LF because MS-DOS did, because CP/M did, because it made sense for serial lines.

Unix copied its \n convention because Multics did.

I suspect if you dig far enough back, you’ll find a political disagreement between implementors!

(You left out the extra fun bit, where Mac convention is (or used to be) to just use CR to separate lines. And now Unicode also has its own line separator, U+2028!)

answered Dec 22, 2010 at 11:40

Frank Shearar's user avatar

Frank SheararFrank Shearar

16.7k7 gold badges49 silver badges84 bronze badges

3

What is it with people asking «why can Unix do \n and not Windows»? It’s such a strange question.

  1. The OS has almost nothing to do with it. It’s more a matter of how apps, libraries, protocols and file formats deal with things. Other than where the OS reads/writes text-based configuration or command line commands, it makes no sense to fault the OS.
  2. Most Windows apps can read both \n and \r\n just fine. They also output \r\n so that everyone’s happy. A program doesn’t simply «do» either \n or \r\n — it accepts one, the other, or both, and outputs one, the other, or both.
  3. As a programmer this should really almost never bother you. Practically every language/platform has facilities to write the correct end-line and read most robustly. The only time I’ve had to deal with the problem was when I wrote an HTTP server — and it was because a certain browser (hint: the next most popular browser after IE) was doing \n instead of the correct \r\n.
  4. A much more pertinent question is, why do so many modern Unix apps output only \n fully knowing that there are some protocols and programs that don’t like it?

answered Dec 22, 2010 at 14:51

Rei Miyasaka's user avatar

Rei MiyasakaRei Miyasaka

4,5411 gold badge32 silver badges36 bronze badges

2

The reason the conventions hold on their various systems (\n on unix type systems, \r\n on Windows, etc) is that once you’ve picked a convention you CAN’T change it without breaking a bunch of people’s files. And that’s generally frowned upon.

Unix-type systems were developed (very early days) using various models of teletype, and at some point someone decided the equipment should carriage return when it did a line feed.

Windows came from DOS, so for Windows the question really is: Why did DOS use this cr/lf sequence? I’m guessing it has something to do with CP/M, where DOS has some of it’s roots. Again, specific models of teletype may have played a role.

answered Dec 22, 2010 at 11:54

Michael Kohne's user avatar

Michael KohneMichael Kohne

10k1 gold badge36 silver badges45 bronze badges

3

Here is an answer from the best source — Microsoft.
Why is the line terminator CR+LF?

This protocol dates back to the days of teletypewriters. CR stands for
«carriage return» — the CR control character returned the print head
(«carriage») to column 0 without advancing the paper. LF stands for
«linefeed» — the LF control character advanced the paper one line
without moving the print head. So if you wanted to return the print
head to column zero (ready to print the next line) and advance the
paper (so it prints on fresh paper), you need both CR and LF.

If you go to the various internet protocol documents, such as RFC 0821
(SMTP), RFC 1939 (POP), RFC 2060 (IMAP), or RFC 2616 (HTTP), you’ll
see that they all specify CR+LF as the line termination sequence. So
the the real question is not «Why do CP/M, MS-DOS, and Win32 use CR+LF
as the line terminator?» but rather «Why did other people choose to
differ from these standards documents and use some other line
terminator?»

Unix adopted plain LF as the line termination sequence. If you look at
the stty options, you’ll see that the onlcr option specifies whether a
LF should be changed into CR+LF. If you get this setting wrong, you
get stairstep text, where

each
    line
        begins

where the previous line left off. So even unix, when left in raw mode,
requires CR+LF to terminate lines. The implicit CR before LF is a unix
invention, probably as an economy, since it saves one byte per line.

The unix ancestry of the C language carried this convention into the C
language standard, which requires only «\n» (which encodes LF) to
terminate lines, putting the burden on the runtime libraries to
convert raw file data into logical lines.

The C language also introduced the term «newline» to express the
concept of «generic line terminator». I’m told that the ASCII
committee changed the name of character 0x0A to «newline» around 1996,
so the confusion level has been raised even higher.

Justine Krejcha's user avatar

answered Sep 30, 2017 at 5:47

Ondra Žižka's user avatar

Ondra ŽižkaOndra Žižka

2673 silver badges6 bronze badges

  • Конвертировать в windows 10 pro
  • Консоль управления дисками windows 10
  • Консоль windows перейти на другой диск
  • Контакты windows скачать на русском
  • Конвертировать windows 1251 в utf 8 в windows 1251