mcloc(1) FreeBSD General Commands Manual mcloc(1)

mcloc —
count lines of code in source files

mcloc [-C] [-E extension(s)] [-L language(s)] [-S scanner(s)] [-d | -e | -l | -s] [-g] [-j numThreads] [-o] [-q] [-a | -r] path(s)

mcloc(1) is a simple command line utility for obtaining a rough count of lines of code and comments in Ada, bison(1), C, C++, C#, Clojure, CSS, D, Erlang, flex(1), Go, Groovy, Haskell, java, JSON, Kotlin, Lisp, m4, make, man, Objective-C, PHP, Python, R, Ruby, Rust, Scala, Swift, TOML, TypeScript and shell script source files.

The count is rough because the speed of processing is more important to me than high accuracy. For high accuracy, we'd have to fully parse the source code, which is a non-trivial activity (slow). The whole point of mcloc(1) is quick assessments of source code size, mainly to give a developer a sense of the effort required to maintain a particular set of source code files. It's particularly useful in refactoring efforts, where reduction of source code size may be a stated objective.

The following optional command line arguments are available:
Print the configuration and exit (do not scan files). Note this will incorporate any changes made with the -E and -S arguments.
extension(s)
Only scan files with an extension specified in extensions(s), which is a comma-separated list of file extensions (e.g. .lex,.c,.h).
language(s)
Only scan files for the languages specified in language(s), which is a comma-separated list of languages (e.g. c++,python).
scanners(s)
Only activate the scanners named in the comma-separated list of scanners, from the following list:

Ada
 
CSS
 
Clojure
 
Cpp
 
D
 
Erlang
 
Flex
 
Haskell
 
Kotlin
 
Lisp
 
m4
 
Manpage
 
NoComments
 
Python
 
Ruby
 
Shell
 
Sort output entries by lines of code, in ascending order.
Print counters for each directory (do not show files).
Print counters for each file extension instead of counters for each source file and/or directory.
Print counters for each language instead of counters for each source file and/or directory.
Print counters for each scanner instead of counters for each source file and/or directory.
Include files generated by flex or bison.
numThreads
Specify the number of threads to use to parse source files. Note there are tradeoffs here, and more threads may or may not be faster than fewer threads.

The default thread count for parsing source files is the concurrency reported by std::thread::hardware_concurrency().

Do not recurse into subdirectories.
Only print totals (quiet).
Sort output entries by lines of code, in descending order.

mcloc(1) requires one or more path(s) after the optional arguments. Each of these arguments must be a path to a file or directory.

The output of mcloc(1) has 4 columns. In the first line (column titles) and the last line (TOTAL), the first column is intentially left blank to make it easy to grep for only data lines in the output. For all other lines in the output, the first column contains a single-character indicator of the type of the row:

D
directory
F
file
G
generated file
E
filename extension
L
language
S
scanner

The second column of a data row contains the lines of code counter.

The third column of a data row contains the comment counter.

The fourth column contains the row's label. This may be a path to a directory, a filename, a file extension, a language, a scanner, or TOTAL.

See EXAMPLES for example output.

The main goal of mcloc(1) is fast, reasonably accurate counting of lines of code and comments in Ada, bison(1), C, C++, C#, Clojure, CSS, D, Erlang, flex(1), Go, Groovy, Haskell, java, JSON, Kotlin, Lisp, m4, make, man, Objective-C, PHP, Python, R, Ruby, Rust, Scala, Swift, TOML, TypeScript and shell script source files. It can likely be fooled by odd commenting style and comment delimiters inside string literals and the like, but there's a wide chasm between high speed and complete accuracy. mcloc(1) occupies an area somewhere closer to high speed than complete accuracy, by design. For my personal source code style, I've found it to produce reliable lines-of-code metrics, and it's always done so at a rate that makes me happy to use it. Your mileage may vary, as I wrote this tool for my own use.

Below is a table showing lines/second processed on a few machines in my home.

CPU I/O O/S lines/sec
Threadripper 3960X NVMe (PCI 4.0 x4) Ubuntu 20.04 20,000,000+
Core i9 2.9GHz NVMe (PCI 3.0 x4) MacOS Mojave 3,000,000+
E3-1270 V2 3.50GHz SATA SSD FreeBSD 11.3 2,700,000
dual Xeon L5640 SATA SSD FreeBSD 11.3 4,000,000+
Core i5 2405S 2.50GHz SATA SSD FreeBSD 11.3 1,500,000
ARM Cortex-A72 1.5GHz microSD Raspbian 10 900,000

I find these numbers sufficient for my own use. Much of the time I'm looking at a smaller code base or a smaller piece of a larger code base. Usually less than 200,000 lines of code.

How many lines of code comprise the source for mcloc(1) itself? 
% mcloc .
      CODE COMMENTS PATH                                               
D     3431     2854 .
F       24       52   DwmMclocAdaFileScanner.hh
F       40       42   DwmMclocAdaFileScanner.lex
F       24       52   DwmMclocClojureFileScanner.hh
F      101       42   DwmMclocClojureFileScanner.lex
F       70       83   DwmMclocCodeCounter.hh
F       99       72   DwmMclocCodeCounterMap.hh
F      155       82   DwmMclocCodeUtils.cc
F       25       74   DwmMclocCodeUtils.hh
F      232       84   DwmMclocConfig.cc
F       66      140   DwmMclocConfig.hh
F       46       39   DwmMclocConfigLex.lex
F      140       43   DwmMclocConfigParse.y
F       24       54   DwmMclocCppFileScanner.hh
F       83       44   DwmMclocCppFileScanner.lex
F       24       52   DwmMclocCssFileScanner.hh
F       62       42   DwmMclocCssFileScanner.lex
F       24       52   DwmMclocDFileScanner.hh
F       71       43   DwmMclocDFileScanner.lex
F       24       52   DwmMclocErlangFileScanner.hh
F       40       42   DwmMclocErlangFileScanner.lex
F       16       50   DwmMclocFileScanner.hh
F       24       52   DwmMclocFlexFileScanner.hh
F       94       47   DwmMclocFlexFileScanner.lex
F       24       52   DwmMclocHaskellFileScanner.hh
F       51       42   DwmMclocHaskellFileScanner.lex
F       13       42   DwmMclocHowToSort.hh
F       24       52   DwmMclocKotlinFileScanner.hh
F       62       43   DwmMclocKotlinFileScanner.lex
F       31       57   DwmMclocLanguage.cc
F       27       42   DwmMclocLanguage.hh
F       24       52   DwmMclocLispFileScanner.hh
F       40       42   DwmMclocLispFileScanner.lex
F       24       54   DwmMclocM4FileScanner.hh
F       40       42   DwmMclocM4FileScanner.lex
F       24       52   DwmMclocPythonFileScanner.hh
F       59       42   DwmMclocPythonFileScanner.lex
F       24       52   DwmMclocRubyFileScanner.hh
F       50       42   DwmMclocRubyFileScanner.lex
F       37       51   DwmMclocScanWorker.cc
F       26       51   DwmMclocScanWorker.hh
F       24       52   DwmMclocShellFileScanner.hh
F       40       42   DwmMclocShellFileScanner.lex
F      512      123   DwmMclocSourceCollection.cc
F      103      105   DwmMclocSourceCollection.hh
F       40       67   DwmMclocSourceFileInfo.hh
F       55       53   DwmMclocTerminalTricks.cc
F       71       64   DwmMclocTerminalTricks.hh
F      225      146   DwmThreadQueue.hh
F       17        1   configure.ac
F      220       54   mcloc.cc
F       36        2   mcloc.m4
D       58        0 ./docs/man
F       58        0   mcloc.css                                       
      3491     2862 TOTAL  (0.002729 secs, 2,327,958 lines/sec, 672% cpu)

What if I include generated source files and sort by size? Note how in this case, the flex-generated code (which we never modify) is 10 times more code than all of the code we need to maintain for mcloc(1). This is a case where it's clearly misleading to include generated code in a lines-of-code metric.

% mcloc -a -g .
      CODE COMMENTS PATH                                               
D       58        0 ./docs/man
F       58        0   mcloc.css
D    45547    10507 .
F       13       42   DwmMclocHowToSort.hh
F       16       50   DwmMclocFileScanner.hh
F       17        1   configure.ac
F       24       52   DwmMclocShellFileScanner.hh
F       24       52   DwmMclocClojureFileScanner.hh
F       24       54   DwmMclocCppFileScanner.hh
F       24       52   DwmMclocCssFileScanner.hh
F       24       52   DwmMclocKotlinFileScanner.hh
F       24       52   DwmMclocAdaFileScanner.hh
F       24       52   DwmMclocDFileScanner.hh
F       24       52   DwmMclocFlexFileScanner.hh
F       24       52   DwmMclocLispFileScanner.hh
F       24       52   DwmMclocRubyFileScanner.hh
F       24       52   DwmMclocHaskellFileScanner.hh
F       24       52   DwmMclocErlangFileScanner.hh
F       24       54   DwmMclocM4FileScanner.hh
F       24       52   DwmMclocPythonFileScanner.hh
F       25       74   DwmMclocCodeUtils.hh
F       26       51   DwmMclocScanWorker.hh
F       27       42   DwmMclocLanguage.hh
F       31       57   DwmMclocLanguage.cc
F       36        2   mcloc.m4
F       37       51   DwmMclocScanWorker.cc
F       40       42   DwmMclocErlangFileScanner.lex
F       40       42   DwmMclocLispFileScanner.lex
F       40       67   DwmMclocSourceFileInfo.hh
F       40       42   DwmMclocM4FileScanner.lex
F       40       42   DwmMclocShellFileScanner.lex
F       40       42   DwmMclocAdaFileScanner.lex
F       46       39   DwmMclocConfigLex.lex
G       47       42   DwmMclocConfigParse.hh (DwmMclocConfigParse.y)
F       50       42   DwmMclocRubyFileScanner.lex
F       51       42   DwmMclocHaskellFileScanner.lex
F       55       53   DwmMclocTerminalTricks.cc
F       59       42   DwmMclocPythonFileScanner.lex
F       62       42   DwmMclocCssFileScanner.lex
F       62       43   DwmMclocKotlinFileScanner.lex
F       66      140   DwmMclocConfig.hh
F       70       83   DwmMclocCodeCounter.hh
F       71       64   DwmMclocTerminalTricks.hh
F       71       43   DwmMclocDFileScanner.lex
F       83       44   DwmMclocCppFileScanner.lex
F       94       47   DwmMclocFlexFileScanner.lex
F       99       72   DwmMclocCodeCounterMap.hh
F      101       42   DwmMclocClojureFileScanner.lex
F      103      105   DwmMclocSourceCollection.hh
F      140       43   DwmMclocConfigParse.y
F      155       82   DwmMclocCodeUtils.cc
F      220       54   mcloc.cc
F      225      146   DwmThreadQueue.hh
F      232       84   DwmMclocConfig.cc
F      512      123   DwmMclocSourceCollection.cc
G     1161      391   DwmMclocConfigParse.cc (DwmMclocConfigParse.y)
G     1384      448   DwmMclocConfigLex.cc (DwmMclocConfigLex.lex)
G     1733      478   DwmMclocLispFileScanner.cc (DwmMclocLispFileScanner.lex)
G     1733      478   DwmMclocErlangFileScanner.cc (DwmMclocErlangFileScanner.lex)
G     1733      478   DwmMclocShellFileScanner.cc (DwmMclocShellFileScanner.lex)
G     1761      478   DwmMclocAdaFileScanner.cc (DwmMclocAdaFileScanner.lex)
G     1789      478   DwmMclocM4FileScanner.cc (DwmMclocM4FileScanner.lex)
G     2250      482   DwmMclocHaskellFileScanner.cc (DwmMclocHaskellFileScanner.lex)
G     2348      482   DwmMclocRubyFileScanner.cc (DwmMclocRubyFileScanner.lex)
G     2551      483   DwmMclocPythonFileScanner.cc (DwmMclocPythonFileScanner.lex)
G     2617      483   DwmMclocCssFileScanner.cc (DwmMclocCssFileScanner.lex)
G     2793      485   DwmMclocKotlinFileScanner.cc (DwmMclocKotlinFileScanner.lex)
G     3192      488   DwmMclocDFileScanner.cc (DwmMclocDFileScanner.lex)
G     3856      489   DwmMclocClojureFileScanner.cc (DwmMclocClojureFileScanner.lex)
G     4553      498   DwmMclocFlexFileScanner.cc (DwmMclocFlexFileScanner.lex)
G     6615      492   DwmMclocCppFileScanner.cc (DwmMclocCppFileScanner.lex)
     45605    10507 TOTAL  (0.002858 secs, 19,636,808 lines/sec, 640% cpu)

What about per file extension, sorted by lines of code?

% mcloc -ae .
         CODE    COMMENTS EXTENSION/EXPRESSION                             
E           2           8 .d
E          17           1 .ac
E          36           2 .m4
E          58           0 .css
E          63           2 "Makefile"
E         206          43 .y
E         337         111 ".+\-sh"
E         481           1 ".+\.[1-9]"
E         958         719 .lex
E       1,265       1,843 .hh
E       1,410         536 .cc                                              
        4,833       3,266 TOTAL  (2.60ms, 3.12M lines/s, 166% sys 485% usr)

How many lines of code per language in libDwm, sorted by lines of code?

% mcloc -al libDwm/trunk
         CODE    COMMENTS LANGUAGE                                         
L         217           8 CSS
L         323           0 man
L         439         111 Shell
L         593          19 Make
L         698          58 m4
L         880         308 flex
L       1,061         102 bison
L      34,307      22,730 C++                                              
       38,518      23,336 TOTAL  (9.65ms, 6.41M lines/s, 219% sys 141% usr)

How many lines of code per language in qt5?

% mcloc -a -l ~/gits/qt5
         CODE    COMMENTS LANGUAGE                                         
L         222          51 Scheme
L         351          58 Swift
L         662         275 Ruby
L       1,116          47 D
L       4,057         160 flex
L       7,887       2,804 Make
L      10,098       8,415 PHP
L      12,155         904 bison
L      16,257       2,564 Go
L      40,095         466 man
L      42,749         524 Lisp
L      53,406      18,938 Shell
L      54,689       5,720 CSS
L      58,464      13,021 C#
L      64,504       2,531 m4
L     130,627      42,713 Java
L     233,967      42,445 Objective-C
L     475,669     187,102 Python
L     492,351       2,086 TypeScript
L     765,125           0 JSON
L   1,418,368   1,077,973 JavaScript
L   7,807,396   2,711,326 C
L  13,029,190   2,361,738 C++                                              
   24,719,405   6,481,861 TOTAL  (3.39s, 9.20M lines/s, 258% sys 153% usr)

How many lines of code in each directory in libDwm?

% mcloc -d libDwm/trunk
         CODE    COMMENTS DIR                                             
D       1,191         169 libDwm/trunk/
D          16           0 libDwm/trunk/apps
D         112          44 libDwm/trunk/apps/dwmnet
D         121          60 libDwm/trunk/apps/dwmspath
D         256          56 libDwm/trunk/apps/dwmwhat
D         191          51 libDwm/trunk/apps/fbsddeps
D          47          43 libDwm/trunk/apps/fixcflags
D         276          54 libDwm/trunk/apps/mcping
D         537         111 libDwm/trunk/apps/mkfbsdmnfst
D         139          50 libDwm/trunk/apps/psg
D         239           8 libDwm/trunk/doc
D         176          13 libDwm/trunk/doc/examples
D       8,342      11,515 libDwm/trunk/include
D          12           0 libDwm/trunk/packaging
D      16,486       6,996 libDwm/trunk/src
D      10,377       4,166 /Users/dwm/svn/dwm/libDwm/trunk/tests      
       38,518      23,336 TOTAL  (2.65ms, 23.4M lines/s, 455% sys 450% usr)

/usr/local/etc/mcloc.cfg is the global configuration file. If present, ~/.mcloc will be used instead of the global configuration file.

flex(1), bison(1)

I first wrote mcloc(1) in 2000. At the time, we didn't have a lot of tools to count lines of C++ code, and none of them were smart enough to keep generator files (flex and bison input) and ignore generated files (flex and bison output). mcloc(1) was born when I needed to keep track of my progress on various flex/bison lexers/parsers (MIB compiler, BGP route log searcher, etc.). It's fairly typical for flex and bison output to be two decimal orders of magnitude larger than the input, but we don't maintain the output, we maintain the input. I needed something quick and dirty to recognize generated files and not include them in my LOC metrics (unless requested).

mcloc(1) was updated with relatively minor tweaks throughout the 2000's and 2010's.

In 2019 and early 2020, I rewrote mcloc(1) to use a reentrant lexical analyzer and allow multiple threads. With today's solid state storage, multiple threads can be an advantage (we're not always I/O bound). I also restructured to use a two-level storage (directories and files within a directory) and std::unordered_map. The original std::map based implementation wasn't terribly slow, but the way I used it (full paths as keys) meant longer keys versus the two-level scheme and no fast way to calculate numbers per directory. These changes increased performance by roughly a factor of 3.

Since I'm now using std::filesystem, the code should be portable to Windows with a getopt() replacement (I haven't tried yet).

I also added the ability to show lines of code and comments per file extension or per type, as well as the ability to sort by size. In addition, I added smarter code to recognize source code generated by flex(1) or bison(1); the current version recognizes comments and preprocessor directives in the generated code instead of relying on matching filenames. This allows me to ignore generated files even if I can't find the generator source code in the given directories, and doesn't require parsing flex and bison options in the generator sources nor peeking at build infrastructure. However, flex(1) and bison(1) don't produce anything in their output to indicate the source of the generated code if you disable #line directives. If you use the -L option of flex(1) or the -l option of bison(1), for example, mcloc(1) will still recognize the generated code as generated, but will not be able to name the source.

Daniel W. McRobb dwm@mcplex.net
January 18, 2020 mcplex.net