ProgramGuru

Text and Binary Files: Encoding and Bytes

A student name needs an encoding to become bytes

The name José includes é. A report must preserve that character when written and read on different computers; counting characters alone does not determine the number of stored bytes.

All files store bytes. A text encoding defines how characters map to those bytes. A binary format can organize bytes as images, records or other structures that should not be decoded blindly as text.

  1. Distinguish counts

    Explain why a character may occupy several UTF-8 bytes.

  2. Choose a mode

    Read text as characters or binary data as bytes.

  3. Preserve meaning

    Use the specified encoding and reject unsuitable decoding.

One character, two UTF-8 bytes

  1. Character

    The intended text is one character.

    é

  2. Encode

    UTF-8 defines the byte sequence.

    195, 169

  3. Decode

    The same encoding restores the text.

    é
Read these stages alongside the complete example. They explain the relationship; they do not execute code.

Binary does not mean unreadable by a computer

Text is one interpretation of bytes using an encoding. A binary image format instead has rules for headers, dimensions and pixel data. Both are meaningful when the correct format is known.

Python text mode returns str and binary mode returns bytes. UTF-8 is a common encoding, but the file’s declared format determines what you should use. Invalid decoding can fail or lose information under a replacement policy.

Before starting, review file modes and cleanup. These examples use JavaScript; other languages can use different syntax and rules.

Write UTF-8 text and read its actual bytes

This complete Python 3 exercise creates name.txt in a disposable directory. Binary mode rb returns the raw bytes rather than decoded characters.

with open('name.txt', 'w', encoding='utf-8') as file:
    file.write('é')
with open('name.txt', 'rb') as file:
    raw = file.read()
print(len(raw))
print(raw.decode('utf-8'))

Expected output: 2, then é. decode converts bytes according to UTF-8; it does not infer an arbitrary file format.

Inspect encoding without accessing the filesystem

The JavaScript TextEncoder and TextDecoder operate on local data. The single character becomes two UTF-8 bytes and returns to é.

No file is read here. This isolates the encoding relationship already demonstrated by the actual Python file workflow.

JavaScript · local teaching example

const text = 'é';
const bytes = new TextEncoder().encode(text);
console.log(text.length);
console.log(bytes.length);
console.log(new TextDecoder('utf-8').decode(bytes));

Expected output

1
2
é

A binary value is not a decimal text digit

The number 65 encoded as a byte and the string “65” encoded as text are different data. UTF-8 text “65” requires two ASCII-compatible bytes, 54 and 53.

The format decides whether bytes represent a numeric field or characters. Do not compare a byte count with a digit count as though they were interchangeable.

JavaScript · local teaching example

const bytes = new TextEncoder().encode('65');
console.log(bytes.length);
console.log(bytes[0]);
console.log(bytes[1]);

Expected output

2
54
53

Apply the rule to a small change

Encode José, report its UTF-8 byte count and decode it back. Predict why the counts differ.

Worked answer

The four JavaScript code units here encode as five UTF-8 bytes because é uses two bytes. The round trip restores the text. For other Unicode characters, JavaScript string length can count multiple code units per visible character, so do not generalize this simple count.

JavaScript · local teaching example

const name = 'José';
const bytes = new TextEncoder().encode(name);
console.log(name.length);
console.log(bytes.length);
console.log(new TextDecoder().decode(bytes));

Expected output

4
5
José

Recognize mistakes and limits

JavaScript string length counts UTF-16 code units, not necessarily user-perceived characters. Byte length, code-point count and displayed-character count can all differ.

Opening an image in text mode does not extract its meaning. Use a parser for its binary format. Specify error handling for invalid encodings and avoid silently replacing data when exact preservation matters.

For the language or tool rules used here, see Python file modes and encodings.

Check the core idea

Why does é occupy two bytes in this exercise?
Choose one answer. You can try again.

Answer and explanation

Its UTF-8 representation consists of two bytes.

Yes. Character and byte counts differ.

Apply the idea to a new situation

What should choose the decoder for a file?
Choose one answer. You can try again.

Answer and explanation

The documented encoding and file format.

Yes. Meaning depends on those rules.

Treat bytes according to their documented format

Text needs an encoding; binary data needs its own format. Read actual bytes deliberately and keep byte counts separate from character measures.

Connect this idea with locating a file and decoding and access failures.

Reaching the recap and completing the quizzes are separate milestones.

    Next lesson: Handling File Exceptions and Failed Reads →