Kalpana Kalpana (Editor)

Unicode and email

Updated on
Edit
Like
Comment
Share on FacebookTweet on TwitterShare on LinkedInShare on Reddit

Many email clients now offer some support for Unicode. While some use Unicode by default, many others will automatically choose between a legacy encoding and Unicode depending on the mail's content, either automatically or when the user requests it.

Contents

Technical requirements for sending of messages containing non-ASCII characters by email include

  • encoding of certain header fields (subject, sender’s and recipient’s names, sender’s organization and reply-to name) and, optionally, body in a content-transfer encoding
  • encoding of non-ASCII characters in one of the Unicode transforms
  • negotiating the use of UTF-8 encoding in email addresses and reply codes (SMTPUTF8)
  • sending the information about the content-transfer encoding and the Unicode transform used so that the message can be correctly displayed by the recipient (see Mojibake).

    • If the sender’s or recipient’s email address contains non-ASCII characters, sending of a message requires also encoding of these to a format that can be understood by mail servers.

    Unicode support in protocols

  • RFC 6531 provides a mechanism for allowing non-ASCII email addresses encoded as UTF-8 in an SMTP or LMTP protocol

    • Unicode support in message header

    To use Unicode in certain email header fields, e.g. subject lines, sender and recipient names, the Unicode text has to be encoded using a MIME "Encoded-Word" with a Unicode encoding as the charset. To use Unicode in domain part of email addresses, IDNA encoding must traditionally be used. Alternatively, SMTPUTF8 allows the use of UTF-8 encoding in email addresses (both in a local part and in domain name) as well as in a mail header section. Various standards had been created to retrofit the handling of non-ASCII data to the originally ASCII-only email protocol:

  • RFC 2047 provides support for encoding non-ASCII values such as real names and subject lines in email header
  • RFC 5890 provides support for encoding non-ASCII domain names in the Domain Name System
  • RFC 6532 allows the use of UTF-8 in a mail header section

    • Unicode support in message bodies

    As with all encodings apart from US-ASCII, when using Unicode text in email, MIME must be used to specify that a Unicode transformation format is being used for the text.

    UTF-7, although sometimes considered deprecated, has an advantage over other Unicode encodings in that it does not require a transfer encoding to fit within the seven-bit limits of many legacy Internet mail servers. On the other hand, UTF-16 must be transfer encoded to fit SMTP data format. Although not strictly required, UTF-8 is usually also transfer encoded to avoid problems across seven-bit mail servers. MIME transfer encoding of UTF-8 makes it either unreadable as a plain text (in the case of base64) or, for some languages and types of text, heavily size inefficient (in the case of quoted-printable).

    Some document formats, such as HTML, PostScript and Rich Text Format have their own 7-bit encoding schemes for non-ASCII characters and can thus be sent without using any special email encodings. E.g. HTML email can use HTML entities to use characters from anywhere in Unicode even if the HTML source text for the email is in a legacy encoding (e.g. 7-bit ASCII). For details of this see Unicode and HTML. The rest of this article deals with email messages where the actual raw text (whether markup or plain text) is in an encoding that covers the whole of Unicode.

    References

    Unicode and email Wikipedia
    (Text) CC BY-SA