Capstone B: the index as one module

Project B has three modules and a caller that wants one. The facade of this lesson is a function that takes the documents, builds the index and returns an object with the search, the two counts and the plain-data form. Every caller after this lesson talks to that object and none of them imports the three modules.

The lesson also writes the two test files of the project and a check program that asserts a fixed answer. Nothing is timed: a duration is a number of the machine that ran it, so a check that prints one cannot be compared from one run to the next, and the check of this project compares results instead.

The folder holds the manifest, the lint config, the tokenizer, the index and the query. This lesson adds src/search.js and the test folder, and rewrites the check program.

mkdir test

One module in front of three

A facade is worth its line count when the three modules behind it are an implementation. A caller that wants a search over three documents should not have to know that the score counts the words again, or that the index has a plain form; the object below gives it a search, two numbers and a conversion, and the next project builds an HTTP endpoint on top of that without importing the tokenizer.

The index is built once, when the facade is created. A server that built it per request would pay for the tokenizing on every call, and a test that built it per case would not see the object that a server holds.

The file src/search.js:

vi src/search.js

const { buildIndex, toJSON } = require('./index'); const { format, search } = require('./query'); function createSearch(documents) { const index = buildIndex(documents); return { documents: index.documents.length, terms: index.terms.size, find: (text) => search(index, text), toJSON: () => toJSON(index), }; } module.exports = { createSearch, format };

The object is a closure over one index, which is the pattern of the lesson on closures: the two numbers are values, the two functions keep the index alive, and nothing else can reach it. The name find is the verb of the object and search is the verb of the module, so a reader can tell the two apart at the call site.

The tests

Five tests cover the tokenizer and seven cover the search. The first file is the one that pins the rules of the words: the stop words are dropped by looping over the exported set rather than over a copy of it, the digits and the punctuation separate, and an empty text gives no words.

The file test/tokenizer.test.js:

vi test/tokenizer.test.js

const test = require('node:test'); const assert = require('node:assert/strict'); const { STOP_WORDS, tokenize } = require('../src/tokenizer'); test('a sentence becomes its words', () => { assert.deepEqual(tokenize('The cat sat on the mat.'), ['cat', 'sat', 'mat']); }); test('the case does not matter', () => { assert.deepEqual(tokenize('Dog DOG dog'), ['dog', 'dog', 'dog']); }); test('punctuation and digits separate the words', () => { assert.deepEqual(tokenize('12 apples, 3 pears; 4 plums.'), ['apples', 'pears', 'plums']); }); test('every stop word is dropped', () => { for (const word of STOP_WORDS) { assert.deepEqual(tokenize('x ' + word + ' y'), ['x', 'y']); } }); test('an empty text has no words', () => { assert.deepEqual(tokenize(''), []); assert.deepEqual(tokenize(' ... '), []); });

The second file tests the three layers through the facade, which is what a caller sees. It pins the size of the index, the postings of one term, the order of a query, the tie, the empty query and the plain-data form. The order is compared as an array of ids, which is only possible because the tie is broken by the id.

The file test/search.test.js:

vi test/search.test.js

const test = require('node:test'); const assert = require('node:assert/strict'); const { DOCUMENTS, buildIndex } = require('../src/index'); const { createSearch } = require('../src/search'); test('the index holds one entry per term', () => { const index = buildIndex(DOCUMENTS); assert.equal(index.terms.size, 7); assert.deepEqual(index.terms.get('cat'), [1, 3]); }); test('a word that is not there is not an entry', () => { const index = buildIndex(DOCUMENTS); assert.equal(index.terms.has('zebra'), false); }); test('a query returns the documents that hold a term', () => { const search = createSearch(DOCUMENTS); assert.deepEqual( search.find('slept').map((result) => result.id), [1, 2], ); }); test('a higher count ranks first', () => { const search = createSearch(DOCUMENTS); const results = search.find('cat mat'); assert.deepEqual( results.map((result) => result.score), [4, 2], ); }); test('a tie is broken by the id', () => { const search = createSearch(DOCUMENTS); assert.deepEqual( search.find('cat and dog').map((result) => result.id), [1, 3, 2], ); }); test('a query of stop words alone finds nothing', () => { const search = createSearch(DOCUMENTS); assert.deepEqual(search.find('the and a'), []); }); test('the JSON form is plain data', () => { const search = createSearch(DOCUMENTS); assert.deepEqual(search.toJSON().terms.mat, [1, 3]); assert.equal(search.toJSON().documents, 3); });

The seven tests use deepEqual on arrays of numbers rather than on the objects of the results, so a change to a title does not fail the ranking test. A test that pinned the whole result object would fail for a reason that has nothing to do with the order.

The check without a stopwatch

The check program of the earlier lessons printed what it found. This one asserts: it runs a query, compares the ids with the fixed answer and throws if they differ, and it prints the result only after the comparison. The difference matters when the program is part of a pipeline, because a confirmed answer and a printed one are not the same thing to a reader in a hurry.

Nothing here measures time. A benchmark prints a duration that changes with the machine, the page cache and the load of the moment, so a lesson that showed one would have to explain why the number on the page is not the number a reader sees. The exercise at the end asks what a benchmark of this project would have to report to be worth reading.

The file check.js:

vi check.js

const assert = require('node:assert/strict'); const { DOCUMENTS } = require('./src/index'); const { createSearch, format } = require('./src/search'); const search = createSearch(DOCUMENTS); console.log('documents: ' + search.documents + ', terms: ' + search.terms); const results = search.find('cat mat'); assert.deepEqual( results.map((result) => result.id), [1, 3], ); console.log(format(results)); console.log('the index is plain data: ' + JSON.stringify(search.toJSON().terms.cat)); console.log('the check passed');

The three lines of output are the size of the index, the ranked answer and one entry of the plain form. The last line is printed after the assertion, so a reader knows the check reached the end; a program that only printed the results would print them whether they were right or not.

The run

The check program, the twelve tests and the linter run in one block. The check prints four lines, the dot reporter prints twelve characters for the twelve tests, and the linter prints nothing because the folder is clean. The two statuses at the end are the ones a pipeline would read.

node check.js
node --test --test-reporter=dot
echo "the exit status: $?"
npx eslint .
echo "the lint status: $?"

documents: 3, terms: 7
1 (4) The cat on the mat
3 (2) The cat and the dog
the index is plain data: [1,3]
the check passed
............
the exit status: 0
the lint status: 0

Twelve tests, five of the tokenizer and seven of the search, and one check program over the facade. Add the three modules of the previous lessons and the folder is project B: six files in src and test, a manifest and a lint config.

Project C starts from this folder. The next lesson copies it, adds a server and answers the same query over HTTP.

Exercises

  1. Add a titles method to the facade that returns the title of an id. Which layer does the lookup belong to, and which test covers it?
  2. Remove the format export and print the results with the query module. Which import does the HTTP lesson of project C then need?
  3. Build the index per call instead of once. Which test still passes, and which one would notice the change?
  4. Write a test for a document with an empty text. Which number of the facade changes, and what should the search return?
  5. Design a benchmark for this project: which number would you print, how many times would you run it, and why is one run not enough?

Questions

A new set of questions in a new order every time. Choose one answer for each, then check.

  1. What does createSearch hide from its callers?

    Answer

    The tokenizer, the index and the query behind one object

    A caller asks for a search and the three modules are an implementation detail.

  2. What does this program print? const search = { documents: 3, terms: 7 }; console.log(search.documents + search.terms);

    Answer

    10

    The two numbers are fields of the object and the plus adds them.

  3. When is the index built in this project?

    Answer

    Once, when the facade is created

    The closure keeps one index, so the tokenizing is paid for once.

  4. What does this program print? const make = () => { let count = 0; return () => { count += 1; return count; }; }; const next = make(); console.log(next(), next(), next());

    Answer

    1 2 3

    Each call changes the counter that the closure holds.

  5. Why does the check compare the ids instead of printing them?

    Answer

    A comparison fails the program when the answer is wrong, and a print does not

    The assertion is the difference between a demonstration and a check.

  6. What does this program print? const results = [{ id: 1 }, { id: 3 }]; console.log(results.map((result) => result.id).join(","));

    Answer

    1,3

    The map keeps the ids and the join puts a comma between them.

  7. Why is nothing timed in the check of this project?

    Answer

    A duration changes with the machine, so the page and the run would disagree

    A number that differs on every run cannot be copied into a lesson that a reader compares with a run.

  8. What does this program print? const tests = [true, true, false]; console.log(tests.filter((passed) => !passed).length);

    Answer

    1

    One of the three values is false, and the filter keeps the failures.

  9. Why does a test of the ranking compare arrays of ids rather than the whole result objects?

    Answer

    So a change to a title does not fail a test about the order

    A test should fail for one reason, and the order is the reason this one exists.

  10. What does this program print? const index = { terms: new Map([["a", [1]], ["b", [1, 2]]]) }; console.log(index.terms.size, index.terms.get("b").length);

    Answer

    2 2

    The map has two terms and the second one has two ids.

The programs and commands on this page were run with Node.js 24.22.1 and linted with ESLint 10.14.0 on Alpine Linux 3.24.3 (x86_64) in a Docker container.

Article author: Arthur Isaev