Tokens in C Programming: Types, Examples, and Explanation

Every C program is made up of small individual units called tokens. A token is the smallest meaningful element recognized by the C compiler. Just as a sentence is formed using words and punctuation, a C program is constructed using different types of tokens. Tokens are the building blocks of C programming that help the compiler understand the structure and meaning of a program.

What Are Tokens in C?

• Token: The smallest meaningful element of a C program recognized by the compiler.
• Keyword: A reserved word with a predefined meaning in C that cannot be used as an identifier.
• Identifier: A programmer-defined name given to variables, functions, arrays, or other program elements.
• String Literal: A sequence of characters enclosed within double quotation marks.
• Special Symbol: A punctuator or separator that helps organize the structure of a C program.

For example, the statement int sum = 10; consists of the following tokens: int (Keyword), sum (Identifier), = (Operator), 10 (Constant), and ; (Special Symbol).

C programming language has six main types of tokens:

1. Keywords
2. Identifiers
3. Constants
4. String Literals
5. Operators
6. Special Symbols

Keywords

Keywords are reserved words that have predefined meanings in C. They are recognized by the compiler and cannot be used as variable names, function names, or identifiers.

Example

int age = 20;

Here, int is a keyword that tells the compiler that age is an integer variable.

Commonly Used Keywords

• int - Declares an integer variable
• float - Declares a floating-point variable
• char - Declares a character variable
• if - Conditional statement
• else - Alternative condition
• for - Loop
• while - Loop
• return - Returns a value from a function
• const - Declares a constant variable
• void - Specifies no return value
• break - Exits a loop or switch statement
• continue - Skips the current loop iteration

Identifiers

Identifiers are names given by the programmer to variables, functions, arrays, structures, and other program elements.

Rules for Naming Identifiers

• Must begin with a letter or underscore (_).
• Can contain letters, digits, and underscores.
• Cannot contain spaces.
• Cannot contain special characters except _.
• Cannot be a keyword.
• Are case-sensitive.

Valid Identifiers

• sum
• studentName
• roll_no
• _count
• totalMarks

Invalid Identifiers

• 2marks - Starts with a digit
• roll no - Contains space
• float - Keyword
• %total - Special character not allowed

Example

int marks;

Here, marks is an identifier.

Constants

A constant is a fixed value that does not change during program execution. Constants may be numeric or non-numeric.

Integer Constants

• 10
• 250
• -45

Floating-Point Constants

• 3.14
• 25.75
• 0.001

Character Constants

• 'A'
• '9'
• '@'

Example

const int MAX = 100;

Here, 100 is an integer constant.

String Literals

A string literal is a sequence of characters enclosed within double quotation marks. String literals are automatically terminated with a null character (\0).

• "Hello"
• "C Programming"
• "Welcome to C Programming"

Example

#include <stdio.h>

int main()
{
    printf("Hello World");

    return 0;
}

Here, "Hello World" is a string literal.

• Null Character (\0): A special character automatically appended at the end of every string literal in C.

Operators

Operators are symbols used to perform operations on variables and constants.

• Arithmetic: +, -, *, /, %
• Relational: ==, !=, >, <, >=, <=
• Logical: &&, ||, !
• Assignment: =, +=, -=, *=, /=
• Bitwise: &, |, ^, ~, <<, >>

Example

sum = a + b;

Tokens in this statement: sum (Identifier), = (Assignment Operator), a (Identifier), + (Arithmetic Operator), b (Identifier), ; (Special Symbol).

Special Symbols

Special symbols (also called punctuators or separators) help organize the structure of a C program.

• ; - Ends a statement
• { } - Defines a block of code
• ( ) - Function calls and expressions
• [ ] - Array declaration
• , - Separates variables or arguments
• # - Preprocessor directive
• : - Used in labels and conditional operator

Example

int main()
{
    return 0;
}

Here, ( ) define the function, { } define the function body, and ; terminates the statement.

Example of Tokens in a C Program

#include <stdio.h>

int main()
{
    int num = 10;

    printf("Number = %d", num);

    return 0;
}

Tokens used: #include (Preprocessor Directive), stdio.h (Header File), int (Keyword), main (Identifier), ( ) and { } (Special Symbols), num (Identifier), = (Operator), 10 (Constant), printf (Identifier / Library Function), "Number = %d" (String Literal), ; (Special Symbol), return (Keyword), 0 (Constant).

Best Practices

• Use meaningful identifiers.
• Never use keywords as variable names.
• Follow standard naming conventions.
• Use constants for fixed values.
• Write readable expressions using appropriate operators.
• Keep string literals meaningful and concise.
• Avoid beginning identifiers with numbers.
• Avoid using invalid special characters in variable names.

FAQ About Tokens in C

1. What is a token in C programming?

A token is the smallest meaningful element of a C program recognized by the compiler. Every C program is made up of tokens such as keywords, identifiers, constants, operators, and special symbols.

2. How many types of tokens are there in C?

There are six main types of tokens in C: Keywords, Identifiers, Constants, String Literals, Operators, and Special Symbols.

3. What is the difference between a keyword and an identifier?

A keyword is a reserved word with a predefined meaning in C, while an identifier is a user-defined name given to variables, functions, arrays, or other program elements.

4. Are string literals considered tokens?

Yes. A string literal enclosed in double quotation marks (e.g., "Hello") is treated as a single token by the C compiler.

5. Can a keyword be used as an identifier?

No. Keywords are reserved by the C language and cannot be used as variable names, function names, or any other identifiers.

6. Why are tokens important in C programming?

Tokens are the building blocks of C programs. Understanding them helps you write correct programs, avoid syntax errors, and understand how the compiler interprets your code.