Capstone: wordfreq CLI

The spec

The wordfreq tool counts words in text and prints the most frequent ones with their counts. It reads files given as arguments, or standard input when no files are given.

A flag named n sets how many top words to show, with ten as the default. Words compare case insensitively, and ties break in plain word order so output stays stable.

The tool follows the plain style of Golang lessons in this course: small functions, explicit errors, and no hidden behavior.

Structure of main.go

The first version reads standard input only and always prints the top ten. Three parts do the work: countWords builds the frequency map, topWords sorts a copy of it, and main connects input to output.

Sorting uses a slice of pairs because maps have no order. The Less function sorts by count from large to small and falls back to word order for equal counts.

package main import ( "bufio" "fmt" "os" "sort" "strings" ) func countWords(text string) map[string]int { out := make(map[string]int) for _, w := range strings.Fields(text) { out[strings.ToLower(w)]++ } return out } type pair struct { word string count int } func topWords(freq map[string]int, n int) []pair { all := make([]pair, 0, len(freq)) for w, c := range freq { all = append(all, pair{word: w, count: c}) } sort.Slice(all, func(i, j int) bool { if all[i].count == all[j].count { return all[i].word < all[j].word } return all[i].count > all[j].count }) if n > len(all) { n = len(all) } return all[:n] } func main() { text := "" scan := bufio.NewScanner(os.Stdin) for scan.Scan() { text += scan.Text() + " " } for _, p := range topWords(countWords(text), 10) { fmt.Println(p.count, p.word) } }

flags and stdin

The second version adds the n flag and reads files. flag.Int defines the flag with a default and a help line, Parse reads the command line, and Args returns the remaining file names.

Empty arguments mean standard input, so readAll falls back to ReadAll on os.Stdin. File errors print a message and skip the file instead of aborting the whole run.

package main import ( "flag" "fmt" "io" "os" "sort" "strings" ) func countWords(text string) map[string]int { out := make(map[string]int) for _, w := range strings.Fields(text) { out[strings.ToLower(w)]++ } return out } type pair struct { word string count int } func topWords(freq map[string]int, n int) []pair { all := make([]pair, 0, len(freq)) for w, c := range freq { all = append(all, pair{word: w, count: c}) } sort.Slice(all, func(i, j int) bool { if all[i].count == all[j].count { return all[i].word < all[j].word } return all[i].count > all[j].count }) if n > len(all) { n = len(all) } return all[:n] } func readReader(r io.Reader) string { data, err := io.ReadAll(r) if err != nil { fmt.Println(err) return "" } return string(data) } func readAll(paths []string) string { if len(paths) == 0 { return readReader(os.Stdin) } out := "" for _, path := range paths { data, err := os.ReadFile(path) if err != nil { fmt.Println(err) continue } out += string(data) } return out } func main() { top := flag.Int("n", 10, "show top N words") flag.Parse() text := readAll(flag.Args()) for _, p := range topWords(countWords(text), *top) { fmt.Println(p.count, p.word) } }

printf "go go run" | go run . -n 2

2 go

1 run

Table test for count

A table test packs many cases into one slice and loops over it. Each row holds input plus the expected result, so adding a case means adding one line.

The test calls countWords and looks the probed word up in the returned map. A missing word reads as zero from a nil friendly map lookup, which covers the empty input case for free.

package main import ( "testing" ) func TestCountWords(t *testing.T) { cases := []struct { text string word string want int }{ {text: "go go run", word: "go", want: 2}, {text: "Go GO go", word: "go", want: 3}, {text: "", word: "go", want: 0}, } for _, c := range cases { got := countWords(c.text)[c.word] if got != c.want { t.Errorf("countWords(%q) = %d, want %d", c.text, got, c.want) } } }

go test ./...

ok wordfreq 0.008s

Next: Capstone: JSON API server

Article author: Arthur Isaev