Skip to main content
How to add a BOM
Source Link
Chris Davies
  • 128.4k
  • 16
  • 179
  • 324

Your vim hasn't recognised the encoding, and is showing the 16-bit characters as 8-bit characters. The ^@ markers represent the higher order 8-bits, which for common Latin characters will be zero valued.

You can type this after reading in the file to force recognition of UTF-16LE

:e ++enc=utf-16le

(Credit: StackOverflow)

It seems that this would also work, but whether it's ideal is far from clear to me

vim -c 'e ++enc=utf-16le' dictionary.dsl

Finally, from your comment it seems that a BOM would be quite acceptable. You can't use iconv to add a BOM but you can add one yourself

(
    printf "%s" $'\xFF\xFE'
    iconv -f iso-8859-1 -t utf-16le dictionary.dsl
) > dictionary-utf16le.dsl

Confirm with

file dictionary-utf16le.dsl
dictionary-utf16le.dsl: Little-endian UTF-16 Unicode text

Your vim hasn't recognised the encoding, and is showing the 16-bit characters as 8-bit characters. The ^@ markers represent the higher order 8-bits, which for common Latin characters will be zero valued.

You can type this after reading in the file to force recognition of UTF-16LE

:e ++enc=utf-16le

(Credit: StackOverflow)

It seems that this would also work, but whether it's ideal is far from clear to me

vim -c 'e ++enc=utf-16le' dictionary.dsl

Your vim hasn't recognised the encoding, and is showing the 16-bit characters as 8-bit characters. The ^@ markers represent the higher order 8-bits, which for common Latin characters will be zero valued.

You can type this after reading in the file to force recognition of UTF-16LE

:e ++enc=utf-16le

(Credit: StackOverflow)

It seems that this would also work, but whether it's ideal is far from clear to me

vim -c 'e ++enc=utf-16le' dictionary.dsl

Finally, from your comment it seems that a BOM would be quite acceptable. You can't use iconv to add a BOM but you can add one yourself

(
    printf "%s" $'\xFF\xFE'
    iconv -f iso-8859-1 -t utf-16le dictionary.dsl
) > dictionary-utf16le.dsl

Confirm with

file dictionary-utf16le.dsl
dictionary-utf16le.dsl: Little-endian UTF-16 Unicode text
Source Link
Chris Davies
  • 128.4k
  • 16
  • 179
  • 324

Your vim hasn't recognised the encoding, and is showing the 16-bit characters as 8-bit characters. The ^@ markers represent the higher order 8-bits, which for common Latin characters will be zero valued.

You can type this after reading in the file to force recognition of UTF-16LE

:e ++enc=utf-16le

(Credit: StackOverflow)

It seems that this would also work, but whether it's ideal is far from clear to me

vim -c 'e ++enc=utf-16le' dictionary.dsl