Demystifying Tokens in C: Everything You Need to Know

In this post, we will cover the basics of tokens in the C language, their types, and where they are used in C programs. From identifiers to special symbols, each token plays a crucial role in shaping the behavior and functionality of your programs. We will learn more about these essential building blocks of code and discover how they bring your programs to life.

What are tokens in C?

Tokens are the smallest individual units of a program. They include individual words and punctuation marks. In the C language, there are six types of tokens. C programs are written using these tokens.

The classification is given below:

C Keywords

In C, every word falls into either the category of a keyword or an identifier. Keywords possess fixed meanings that cannot be altered.

• Keywords form the basic building blocks for program statements and commands.
• Programs in C cannot redefine or repurpose these words for other purposes.
• Identifiers play a fundamental role in the syntax and structure of C programs, enabling various functionalities such as control flow, data types, and declarations.
• Since C is a case-sensitive language, all keywords must be written in lowercase.
• Examples of keywords in C include:

C Identifiers

• Identifiers refer to the names of the variables, functions, and arrays.
• In C programming, programmers define identifiers as names comprising a sequence of letters and digits, with the first character being a letter.
• Although both uppercase and lowercase letters are permissible, people more commonly use lowercase letters.
• Additionally, developers allow the underscore character in identifiers, often using it as a connector between two words in lengthy identifiers.

Examples of identifiers:

• Variable Names

int age;
float distance;

• Function Names

int calculatequotient(int a, int b)
{
      (function_logic)
}

• Array Names

int numbers[50];

• Structure Names

struct Person {...};

• Enumeration Constants

enum Color {RED, GREEN, BLUE};

Rules for Identifiers in C

• Identifiers must begin with a letter (uppercase or lowercase) or an underscore (_).
• After the initial character, identifiers can contain letters (both uppercase and lowercase), digits, or underscores.
• The case sensitivity of identifiers implies that the programming language treats uppercase and lowercase letters as distinct.
• Identifiers cannot represent keywords or reserved words.
• There’s no specified limit on the length of an identifier, but only the first 31 characters hold significance.
• Identifiers cannot include spaces or special characters (except underscores).
• Avoid starting identifiers with an underscore followed by an uppercase letter, as this usage is typically reserved for system libraries and compilers.

Constants in C

Constants in C refer to the fixed values that remain unchanged during the execution of a program. C supports several types of constants.

Integer Constants

• In C, an integer constant represents a whole number without a decimal point. It can be either positive or negative, and programmers can write integer constants in decimal, octal, or hexadecimal format.
• Decimal Integers: In C, decimal integers represent whole numbers without a decimal point, expressing them in base-10 notation. Programmers use digits from 0 to 9 to represent these integers, which can be positive or negative (+ or – ). Decimal integers do not have any prefixes.

Examples: 1234, 312, -902

• Octal Integers: Octal integers in C are whole numbers expressed in base-8 notation. They use digits from 0 to 7 and are prefixed with a leading zero (0) to indicate octal representation. Octal integers can be positive or negative.

Examples: 037, 023, 045

• Hexadecimal Integers: Hexadecimal integers in C represent whole numbers in base-16 notation, utilizing digits from 0 to 9 and letters from A to F to denote values from 10 to 15. Programmers prefix hexadecimal integers with “0x” or “0X” to denote their hexadecimal representation. They can denote positive or negative values.

Examples: 0x2, 0x9F, 0x5

Real/Float Constants

• Integer numbers are inadequate to represent quantities that vary continuously, such as distances, heights, temperatures, and so on.

• Numbers containing a fractional part represent these quantities.

For example: 21.434, 12.687

• Such numbers are real/floating-point numbers.

For example: 45.0909, 0.32, 0.87

Exponential Notations

• A real number may also be expressed in exponential notation. The general format for exponential notation is mantissa e exponent.

For example: A value of 760.90 may be written as 7.6090e2

For example:

• 45488.098 – decimal form

• 4.5488098e4 – exponential notation

Single Character Constants

A single character constant or simple character constant contains a single character enclosed within a pair of single quotation marks (‘ ‘).

For Example: ‘a’, ‘A’, ’50’, ‘9’

NOTE: The character constant ‘5’ is not the same as number 5. Character constants have integer values known as ASCII values.

String Constants

It refers to a sequence of characters enclosed in double quotes (” “). The characters may be numbers, letters, special characters, or blank spaces.

Examples are: “Hello World!”, “21213”, “bye09”

Special characters in C: