Tokens in C Program Language

A token is the smallest element of a program that is meaningful to the compiler. We can define a token as the smallest individual element in the C programming language. In other words, in a C language program, the smallest individual unit is called a C token.

C token

Tokens can be classified as follows:

1. Keywords
2. Identifiers
3. Constants
4. Strings
5. Special Symbols
6. Operators

1. Keywords in C Program

Keywords in a programming language are pre-defined or reserved words, each intended to perform a specific function in a program. As keywords are reserved names for the compiler, they cannot be used as variable names, as doing so would mean assigning a new meaning to the keyword, which is not allowed. Keywords cannot be redefined. However, using C/C++ preprocessor directives, you can specify the text to be substituted for keywords before compilation.

C language supports the following 32 keywords:

All these keywords, their syntax, and application will be discussed in their respective topics. 

2.  Identifiers in C Program

Identifiers are used as the general terminology for the naming of variables, functions and arrays. These are user-defined names consisting of an arbitrarily long sequence of letters and digits with either a letter or the underscore(_) as a first character.

2.1 The Rules for Constructing C Identifiers are as Follows:

1. The first character of an identifier should be a letter or underscore, followed by any character, digit, or underscore.
2. After the first character, the identifier may contain any combination of letters, underscores, and digits (0 to 9).
3. Reserved keywords such as “if”, “else”, “while”, and “for” cannot be used as identifiers.
4. It cannot start with any digit.
5. In identifiers, uppercase and lowercase letters are considered different. Therefore, identifiers are case-sensitive.
6. Commas or whitespaces cannot be specified in identifiers. White space includes blank space, newline, horizontal tab, carriage return and form feed.
7. Keywords cannot be used as identifiers.
8. The length of an identifier cannot exceed 31 characters. The length of an identifier is not limited, but only the first 31 characters are significant. Identifiers that differ only in their 32nd character are considered identical.
9. Identifiers cannot contain special characters such as punctuation marks or operators.
10. It is considered good practice to choose descriptive and meaningful names for identifiers to make the code easier to read and understand.

3. Constants 

In C programming, a constant is a value that cannot be altered during the execution of a program. Constants are similar to variables, except that their values cannot be modified once they are defined. They are also referred to as “literals” in some contexts.

Constants are commonly used in C programs to define values that are fixed or unchanging, such as mathematical constants like pi (3.14159) or the speed of light (299792458 m/s). Constants can also be used to define symbolic names that represent fixed values, such as the number of days in a week or the maximum size of a file.

In C programming, there are several types of constants, including integer constants, floating-point constants, character constants, and string literals. Integer constants represent whole numbers, floating-point constants represent decimal numbers, character constants represent individual characters, and string literals represent strings of characters. Constants in C can be defined using the const keyword or using preprocessor directives like #define.

3.1 There are two ways to declare constants:

1. Using the const keyword.
2. Using the #define preprocessor.

If you want to define a variable whose value cannot be changed, you can use the const keyword. This will create a constant. For example,

const double PI = 3.14;

Notice, we have added keyword const.

Here, PI is a symbolic constant; its value cannot be changed.

You can also define a constant using the #define preprocessor directive. We will learn about it in C Macros chapter.

3.2 Types of Constants:

1. Integer constants – Example: 0, 1, 1218, 12482
2. Real or Floating-point constants – Example: 0.0, 1203.03, 30486.184
3. Octal & Hexadecimal constants – Example: octal: (013 )₈ = (11)_10, Hexadecimal: (013)₁₆ = (19)₁₀
4. Character constants -Example: ‘a’, ‘A’, ‘z’
5. String constants -Example: “Icstutorial”

3.3 An Example of Constant

#include <stdio.h>

#define LENGTH 10 // constant declaration
#define WIDTH  5  //constant declaration
#define NEWLINE '\n' //Characteristic constant declaration

int main()
{

   int area;

   area = LENGTH * WIDTH;
   printf("value of area : %d", area);
   printf("%c", NEWLINE); 
   printf("\"NEWLINE\" is a char constant");
   return 0;
}

4. Strings in C

In C programming, a string is a sequence of characters stored as an array of characters. A string in C is terminated with a null character (‘\0’) which indicates the end of the string.

Strings in C can be defined as a character array, where each element of the array is a character in the string, and the last element is the null character. For example, “hello” is a string of 5 characters, and it is represented in C as {‘h’, ‘e’, ‘l’, ‘l’, ‘o’, ‘\0’}.

4.1 Declarations for String:

• char string[20] = {‘i’, ’c’, ‘s’, ‘t’, ‘u’, ‘t’, ‘o’, ‘r’, ‘i’, ’a’, ‘l’,  ‘\0’};
• char string[20] = “icstutorial”;
• char string [] = “icstutorial”;
• when we declare char as “string[20]”, 20 bytes of memory space is allocated for holding the string value.
• When we declare char as “string[]”, memory space will be allocated as per the requirement during the execution of the program.

5. Special Symbols

 The following special symbols are used in C having some special meaning and thus, cannot be used for some other purpose.[] () {}, ; * = #

• Brackets[]: Opening and closing brackets are used as array element reference. These indicate single and multidimensional subscripts.
• Parentheses(): These special symbols are used to indicate function calls and function parameters.
• Braces{}: These opening and ending curly braces mark the start and end of a block of code containing more than one executable statement.
• Comma (, ): It is used to separate more than one statements like for separating parameters in function calls.
• Colon(:): It is an operator that essentially invokes something called an initialization list.
• Semicolon(;): It is known as a statement terminator.  It indicates the end of one logical entity. That’s why each individual statement must be ended with a semicolon.
• Asterisk (*): It is used to create a pointer variable and  for the multiplication of variables.
• Assignment operator(=): It is used to assign values and for the logical operation validation.
• Pre-processor (#): The preprocessor is a macro processor that is used automatically by the compiler to transform your program before actual compilation.
• Period (.): Used to access members of a structure or union.
• Tilde(~): Used as destructor to free some space from memory.

6. Operators in C Program

Operators in C are special symbols used to perform operations. The data items on which operators are applied are called operands. Operators are applied between operands.

Based on the number of operands, operators are classified as follows:
 

6.1Unary Operators

Unary operators are operators applied to a single operand.

Examples include increment operator (+), decrement operator (-), sizeof, (type)*.

6.2 Binary Operators

Binary operators are operators applied between two operands. Here is a list of binary operators:

• Arithmetic operators
• Relational operators
• Shift operators
• Logical operators
• Bitwise operators
• Conditional operator
• Assignment operator
• Miscellaneous operators

6.3 Ternary Operators

The operator that require three operands to act upon are called ternary operator. Conditional Operator(?) is also called ternary operator.
Syntax: (Expression1)? expression2: expression3;