Write a Program to Count The Number of Lexeme using JavaScript in Compiler Design

In the field of compiler design, the identification and counting of lexemes are fundamental tasks in the lexical analysis phase. Lexemes are the building blocks of programming languages, consisting of sequences of alphanumeric characters within tokens. This article will explore how to write a program to count the number of lexemes in a given program using JavaScript. We will provide a theoretical overview of lexemes, present the code implementation, and discuss the input/output scenarios.

What is Lexeme?

A lexeme represents a meaningful sequence of characters that forms a part of a token. 

In the context of computer programming, lexemes are extracted from the input stream during lexical analysis to identify tokens. 

Each lexeme plays a crucial role in the overall syntax and semantics of a programming language. 

Understanding and counting lexemes is essential for accurately analyzing and processing programs.

Program to Count The Number of Lexeme in JavaScript:

Let's dive into the code that counts the number of lexemes using JavaScript:

let countKeywords = 0;
let countIdentifierConstant = 0;
let countSeparators = 0;

// Array of keywords
let keywords = ["auto","double","int","struct","break","else","long",
"switch","case","enum","register","typedef","char",
"extern","return","union","const","float","short",
"unsigned","continue","for","signed","void","default",
"goto","sizeof","voltile","do","if","static","while"
];

// Input expression
let expression = "if(float(b) + double(c) == 9text--) { char(ABC+++)}";

// Array of separators
let separators = ["==","=",",","++","+","--","-","*","/","%","{","}","(",")","<",">","[","]"];

function hasSeparators(separators) {
    for (let newline of expression) {
        for (let separator of separators) {
            if (expression.includes(separator)) {
                countSeparators++;
                expression = expression.replace(`${separator}`, ' ');
            }
        }
    }
    return expression;
}

function detectKeyword(keywords, newExpression) {
    let resultKeywords = [];
    for (let expression of newExpression) {
        for (let keyword of keywords) {
            if (keyword == expression) {
                countKeywords++;
                resultKeywords.push(keyword);
            }
        }
    }
    return resultKeywords;
}

function detectIdentifierConstant(keywordList, newExpression) {
    newExpression.filter(keyword => !keywordList.includes(keyword))
                 .forEach(element => countIdentifierConstant++);
}

function detectKeywordIdentifier() {
    console.log(`The given expression:\n ${expression}\n`);
    let newExpression = hasSeparators(separators).split(" ")
                                                .filter(e => e != "");
    let keywordList = detectKeyword(keywords, newExpression);
    detectIdentifierConstant(keywordList, newExpression);
}

detectKeywordIdentifier();
let countLexeme = countIdentifierConstant + countKeywords + countSeparators;
console.log(`The total lexeme: ${countLexeme}`);

Input/Output:

Let's go through the code step by step to understand how it detects keywords and identifiers:

In the provided code, we start by initializing variables to count the number of keywords, identifiers/constants, and separators. 

We have an array of keywords and an input expression representing the program. Additionally, we have an array of separators that define the punctuation symbols in the program.

The hasSeparators function iterates through the expression and checks for occurrences of separators. If a separator is found, it increments the countSeparators variable and replaces the separator with a whitespace.

The detectKeyword function compares each lexeme with the keywords array and increments the countKeywords variable if a match is found.

The detectIdentifierConstant function filters out the lexemes that are already identified as keywords and increments the countIdentifierConstant variable for each remaining lexeme.

Finally, the detectKeywordIdentifier function executes the overall lexeme detection process by calling the appropriate functions and displaying the given expression. The total number of lexemes is calculated by summing the counts of keywords, identifiers/constants, and separators.

Conclusion:

In this article, we have explored the process of counting lexemes in a program using a JavaScript program. We discussed the theoretical concept of lexemes and their significance in the lexical analysis phase of compiler design. By examining the provided code, you can understand how to implement lexeme counting and incorporate it into your own compiler design projects.

Happy coding!

Next Article:

Write a Program to Convert Expression from Infix to Postfix  using JavaScript in Compiler Design

Previous Article:

Write a Program to Detect Integer and Real Number using JavaScript in Compiler Design

 

Ashraf Compiler Design