Generating Web Pages Dynamically - Templating

This appendix presents templating, a technique widely used to generate web pages dynamically. Templating is found in many programming languages, either natively as in PHP or as a framework for other languages (EJS for Node, JSP for Java, ASP for .NET and C#, etc.).

Here we describe the need that templating addresses, present its basic principles and propose a small implementation of this approach to help you understand its advantages and how it works.

The drawbacks of string concatenation

Generating web pages dynamically consists of building and returning a character string that must follow the HTML format. While building this character string, you must include elements computed from information stored in a database or provided in the request received by the web server.

In Chapter 4 we showed the dynamic generation of the image wall of our running example. Code 13.1 shows this generation. Building the HTML character string starts on line 20. You can see that the variable html is declared and contains the HTML characters. The dynamic information is added on lines 23 to 25 with the for loop, which creates one <img> tag per image in the public directory. Finally, the generation ends by closing the <body> and <html> tags. Once the character string is built, it just needs to be sent back to the browser.

const fs = require("fs");
const http = require("http");
const host = 'localhost';
const port = 8080;
const server = http.createServer();

server.on("request", (req, res) => {
    if (req.url.startsWith('/public/')) {
        try {
            const fichier = fs.readFileSync('.'+req.url);
            res.end(fichier);
        } catch (err) {
            console.log(err);
            res.end("error: resource");
        }
    } else if (req.url === '/mur-image') {
        let files = fs.readdirSync('./public');
        let sFiles = files.filter(f => f.endsWith('_small.jpg'));

        let html = '<!DOCTYPE html><html lang="en">';
        html += '<head><title>Today\'s image wall</title></head>';
        html += '<body> <h1>Wall</h1>';
        for (let f of sFiles) {
            html += '<img src="/public/' + f + '">';
        }
        html += '</body></html>';
        res.end(html);
    } else {
        res.end("error: URL");
    }
});

server.listen(port, host, () => {
    console.log(`Server running at http://${host}:${port}/`);
});

Code 13.1: Web server that dynamically generates an image wall.

The HTML generation part of Code 13.1 is not very readable. First, the HTML code is embedded as a plain character string, so the code editor does not know it is HTML and cannot offer syntax highlighting to help readability. Moreover, special characters must be escaped inside a character string; this is the case for the text on line 21, where the apostrophe has to be escaped. Escaping characters disrupts the reading of the HTML. Finally, the for loop makes the code harder to read, because you have to understand that line 24 adds one tag per image and that the image name (variable f) is used in the src property of the image tag.

This code example illustrates the drawbacks of the string concatenation approach. Because the HTML code appears as a plain character string buried in JavaScript code, it is hard to read. Ideally, you would start from an HTML page and then specify where the dynamic information must be inserted: this is the basic idea of templating.

The basic principle: the HTML template

The basic principle of templating is to write the HTML as you would like it to be generated and to specify how to insert the dynamic information. For example, Code 13.2 shows a template illustrating what the image wall could look like with a templating approach. You can see that HTML is the base language of the template, which makes it easier to read and helps you understand the result that will be generated. You can clearly see that some parts of the template are directives that insert the dynamic information. Here we placed these directives between <% %> tags so that they can be spotted quickly (this is the notation of the EJS framework).

<!DOCTYPE html>
<html lang="en">
    <head>
        <title>Image wall</title>
    </head>
    <body> 
        <h1>Wall</h1>
        <% for (let f of sFiles) { %>
            <img src="/public/<%=f%>">
        <% } %>
    </body>
</html>

Code 13.2: The template of the image wall.

An HTML page template, such as the one shown in Code 13.2, must be executed to generate the corresponding HTML document. The template must be executed by the web server before the HTML document is sent.

All frameworks that support templating provide a mechanism for executing templates. For example, Code 13.3 illustrates the use of the EJS framework, calling the render method to execute the image wall template (variable mur, which contains the text of the template). More precisely, line 1 imports the framework. Line 4 calls the render method, which executes the template mur, passing sFiles as a parameter. The render method returns the character string corresponding to the generated HTML. Finally, line 5 sends the HTML back to the web browser.

let ejs = require('ejs');
let files = fs.readdirSync('./public');
let sFiles = files.filter(f => f.endsWith('_small.jpg'));
const html = ejs.render(mur, {sFiles});
res.end(html);

Code 13.3: Web server that dynamically executes the image wall template.

The lines of Code 13.3 replace lines 17 to 27 of Code 13.1. You can see that the HTML code is now in the template and no longer in the web server. This separation between the template and the server makes the code easier to read. Another benefit, which we do not show in our examples, is that templates can be composed and assembled to build complex web pages. All these possibilities offered by templates greatly simplify the generation of dynamic web pages.

How a templating framework works

A framework that supports templating must: (1) offer a way to embed directives in the HTML code and (2) provide a way to execute templates and generate the HTML. Here we propose to code such a framework in order to better understand how templating works internally.

The framework we propose is inspired by EJS. Directives are embedded with two tags: <% %> and <%= %>. The <% %> tag embeds JavaScript that will be executed when the template is executed. The <%= %> tag embeds JavaScript that will also be executed and whose value will be concatenated to the generated HTML. Code 13.2 is therefore compatible with our framework.

Our framework executes a template in three steps. The first step consists of reading the template and separating the HTML parts from the parts containing the two types of directive (<% %> or <%= %>). The second step consists of building a JavaScript function whose code generates the HTML corresponding to the template. The third step consists of executing the generation function to obtain the HTML.

For the first step, we propose the parse function, which reads the characters of the template one after the other and detects the HTML, JS (<% %> tag) and JS_VAL (<%= %> tag) parts of the template. The parse function returns an array containing the parts of the template with their type (HTML, JS or JS_VAL). Code 13.4 shows the parse function. The for loop is the main loop. It iterates over all the characters of the template, detects the tags, builds the parts of the template and adds them to the parts array.

function parse(template) {
    let parts = [];
    let current = {type: HTML, value: ''};
    for (let i=0 ; i < template.length ; i++) {
        if (current.type === HTML) {
            if (i < template.length-1 && template[i] === '<' && template[i+1] === '%') {
                parts.push(current);
                if (i < template.length-2 && template[i+2] === '=') {
                    current = {type: JS_VAL, value: ''};
                    i += 2;
                } else {
                    current = {type: JS, value: ''};
                    i++;
                }
            } else {
                current.value += template[i];
            }   
        } else {
            if (i < template.length-1 && template[i] === '%' && template[i+1] === '>') {
                parts.push(current);
                current = {type: HTML, value: ''};
                i++;
            } else {
                current.value += template[i];
            }   
        }
    }
    if (current.value !== '') {
        parts.push(current);
    }
    return parts;
}

Code 13.4: The parse function, which identifies the different parts of a template.

Running the parse function on the example in Code 13.2 gives an array with seven parts (see Code 13.5).

[{ 
    "type": "HTML",
    "value": "<!DOCTYPE html><html lang='en'><head><title>Image wall</title></head><body><h1>Wall</h1>"
}, {
    "type": "JS",
    "value" : "for (let f of sFiles) {"
}, {
    "type": "HTML",
    "value": "<img src='/public/"
}, {
    "type": "JS_VAL",
    "value": "f"
}, {
    "type": "HTML",
    "value": "'>"
}, {
    "type": "JS",
    "value": "}"
}, {
    "type": "HTML",
    "value": "</body></html>"
}]

Code 13.5: The result of parsing the example template.

The second step of our framework consists of creating a generation function. This function is created from a template and a set of parameters. When executed, it generates the HTML corresponding to the template with the given parameters. Code 13.6 shows the function created from the template in Code 13.2, with an array of two images (image1 and image2) as a parameter. Executing this generation function returns the HTML code of the image wall.

function () {
    let sFiles = ["image1","image2"];
    let html = `<!DOCTYPE html>
        <html lang="en">
        <head>
            <title>Image wall</title>
        </head>
        <body>
            <h1>Wall</h1>
            `;

    for (let f of sFiles) { 
        html += `<img src="/public/${f}">`;
    } 

    html += `</body></html>`;
    return html
}

Code 13.6: The function created from the template in Code 13.2 with image1 and image2 as a parameter. When executed, this function returns the HTML document of the image wall.

For this second step, we chose to write a function that takes as input the parts of the template and the parameters needed to execute it, and returns the generation function (see Code 13.7). This approach is similar to what the EJS framework does. The main idea is to iterate over the parts of the template and generate the corresponding source code. The generated source code depends on the type of the template part being visited (HTML, JS or JS_VAL) and on the current context of code generation. Once the source code has been generated, a Function object corresponding to the generation function can be built (last line of Code 13.7).

function createGenerateFunction(templateParts, parameters) {
    let dataAsVariable = Object.getOwnPropertyNames(parameters).map(p => `let ${p} = ${JSON.stringify(parameters[p])};\n`).join('');
    let rendering = {type:HTML, value:dataAsVariable+'let html = `'};
    for (let i = 0; i < templateParts.length; i++) {
        if (rendering.type === HTML) {
            if (templateParts[i].type === HTML) {
                rendering.value += templateParts[i].value;
            } else if (templateParts[i].type === JS) {
                rendering.value += '`;\n';
                rendering.value += templateParts[i].value; 
                rendering.type = JS;
            } else {
                rendering.value += '${'+templateParts[i].value+'}';
            }
        } else if (rendering.type === JS) {
            if (templateParts[i].type === HTML) {
                rendering.value += 'html += `' + templateParts[i].value;
                rendering.type = HTML
            } else if (templateParts[i].type === JS) {
                rendering.value += templateParts[i].value; 
            } else {
                rendering.value += 'html += `${'+templateParts[i].value+'}';
                rendering.type = HTML;
            }
        }  
    }
    if (rendering.type === HTML) {
        rendering.value += '`;';
    }
    rendering.value += 'return html';
    return new Function(rendering.value);
}

Code 13.7: The function that builds the HTML generation function from a template and parameters.

The third and last step of our framework consists of executing the generation function to obtain the HTML code. Code 13.8 starts by reading the template from the file, then shows these three steps. You can see the first step, which splits the template into parts, the second step, which creates the generation function, and the third and last step, which executes the generation function.

function render(file, parameters) {
    const template = fs.readFileSync(file, { encoding: 'utf8' });

    const templateParts = parse(template);
    const generate = createGenerateFunction(templateParts, parameters)
    return generate();    
}

Code 13.8: The render function.

The framework we propose here follows the design of the EJS framework. It shows clearly that templating translates a template into a function that generates the HTML document. From the developer's point of view, using such a framework means writing templates that look very much like HTML pages, then using the framework to execute them and obtain the corresponding HTML documents.

Note that real templating frameworks also take care of security: EJS, for example, automatically escapes the values inserted with <%= %> (characters such as < and > are converted), so that data typed by users cannot inject HTML or JavaScript into the page. Our small framework does not do this.