sfos/ledger
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Merge pull request #240 from afh/pull/DocTests

Browse source 
Validate examples in the texinfo documentation when running tests.
This commit is contained in:
Craig Earls 2014-02-09 11:31:23 -07:00
commit 050edd56ce
3 changed files with 343 additions and 48 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

194
doc/ledger3.texi
View file

@ -20,6 +20,67 @@
@c Restructuring manual ideas @c Restructuring manual ideas
@c http://beyondgrep.com/documentation/ack-2.04-man.html @c http://beyondgrep.com/documentation/ack-2.04-man.html
@c How to make documented ledger examples validate automatically.
@c
@c The test/DocTests.py script will be run along the with the other
@c tests when using ctest or acprep check.
@c The script parses the texinfo file and looks for three kinds of
@c specially marked @smallexamples, then it will run the ledger
@c command from the exmaple, and compare the results with the output
@c from the documentation.
@c
@c To specially mark a @smallexample append @c command:UUID, where
@c UUID is the first 7 digits from the commands sha1sum, e.g.:
@c
@c @smallexample @c command:CDE330A
@c $ ledger -f sample.dat reg expenses
@c @end smallexample
@c
@c Then DocTests.py will look for corresponding documented output,
@c which may appear anywhere in the file, and is marked with
@c @smallexample @c output:UUID where UUID is the UUID from the
@c corresponding ledger command example, e.g.:
@c
@c @smallexample @c output:CDE330A
@c 04-May-27 Book Store Expenses:Books $20.00 $20.00
@c Expenses:Cards $40.00 $60.00
@c Expenses:Docs $30.00 $90.0
@c @end smallexample
@c
@c Now where does this data in sample.dat come from?
@c DocTests.py is a bit smart about ledger's file argument, since
@c it will check if the given filename exists in the test/input/
@c directory.
@c
@c Sometimes the journal data for an example is specified within
@c the documentation itself, in that case the journal example data
@c needs to be specially marked as well using @smallexample @c input:UUID,
@c again with the UUID being the UUID of the corresponding ledger example
@c command, e.g.:
@c
@c @smallexample @c input:35CB2A3
@c 2014/02/09 The Italian Place
@c Expenses:Food:Dining $ 36.84
@c Assets:Cash
@c @end smallexample
@c
@c @smallexample @c command:35CB2A3
@c $ ledger -f inline.dat accounts
@c @end smallexample
@c
@c @smallexample @c output:35CB2A3
@c Assets:Cash
@c Expenses:Food:Dining
@c @end smallexample
@c
@c Additionally DocTests.py will pass --init-file /dev/null to ledger to
@c ignore any default arguments to ledger the user running the tests
@c has configured.
@c
@c To manually run the tests in this file run:
@c $ ./test/DocTests.py -vv --ledger ./ledger --file ./test/ledger3.texi
@copying @copying
Copyright @copyright{} 2003–2014, John Wiegley. All rights reserved. Copyright @copyright{} 2003–2014, John Wiegley. All rights reserved.
@ -346,13 +407,13 @@ for each.
To find the balances of all of your accounts, run this command: To find the balances of all of your accounts, run this command:
@smallexample @smallexample @c command:1071890
$ ledger -f drewr3.dat balance $ ledger -f drewr3.dat balance
@end smallexample @end smallexample
Ledger will generate: Ledger will generate:
@smallexample @smallexample @c output:1071890
$ -3,804.00 Assets $ -3,804.00 Assets
$ 1,396.00 Checking $ 1,396.00 Checking
$ 30.00 Business $ 30.00 Business
@ -381,8 +442,11 @@ pare this down to show only the accounts you want.
A more useful report is to show only your Assets and Liabilities: A more useful report is to show only your Assets and Liabilities:
@smallexample @smallexample @c command:5BF4D8E
$ ledger -f drewr3.dat balance Assets Liabilities $ ledger -f drewr3.dat balance Assets Liabilities
@end smallexample
@smallexample @c output:5BF4D8E
$ -3,804.00 Assets $ -3,804.00 Assets
$ 1,396.00 Checking $ 1,396.00 Checking
$ 30.00 Business $ 30.00 Business
@ -402,16 +466,16 @@ $ ledger -f drewr3.dat balance Assets Liabilities
To show all transactions and a running total: To show all transactions and a running total:
@smallexample @smallexample @c command:66E3A2C
$ ledger -f drewr3.dat register $ ledger -f drewr3.dat register
@end smallexample @end smallexample
@noindent @noindent
Ledger will generate: Ledger will generate:
@smallexample @smallexample @c output:66E3A2C
10-Dec-01 Checking balance Assets:Checking $ 1,000.00 $ 1,000.00 10-Dec-01 Checking balance Assets:Checking $ 1,000.00 $ 1,000.00
Equity:Opening Balances $ -1,000.00 0 Equit:Opening Balances $ -1,000.00 0
10-Dec-20 Organic Co-op Expense:Food:Groceries $ 37.50 $ 37.50 10-Dec-20 Organic Co-op Expense:Food:Groceries $ 37.50 $ 37.50
Expense:Food:Groceries $ 37.50 $ 75.00 Expense:Food:Groceries $ 37.50 $ 75.00
Expense:Food:Groceries $ 37.50 $ 112.50 Expense:Food:Groceries $ 37.50 $ 112.50
@ -450,8 +514,11 @@ interested in seeing transactions for:
@cindex accounts, limiting by @cindex accounts, limiting by
@cindex limiting by accounts @cindex limiting by accounts
@smallexample @smallexample @c command:96B0EB3
$ ledger -f drewr3.dat register Groceries $ ledger -f drewr3.dat register Groceries
@end smallexample
@smallexample @c output:96B0EB3
10-Dec-20 Organic Co-op Expense:Food:Groceries $ 37.50 $ 37.50 10-Dec-20 Organic Co-op Expense:Food:Groceries $ 37.50 $ 37.50
Expense:Food:Groceries $ 37.50 $ 75.00 Expense:Food:Groceries $ 37.50 $ 75.00
Expense:Food:Groceries $ 37.50 $ 112.50 Expense:Food:Groceries $ 37.50 $ 112.50
@ -465,8 +532,11 @@ $ ledger -f drewr3.dat register Groceries
@noindent @noindent
Which matches the balance reported for the @samp{Groceries} account: Which matches the balance reported for the @samp{Groceries} account:
@smallexample @smallexample @c command:AECD64E
$ ledger -f drewr3.dat balance Groceries $ ledger -f drewr3.dat balance Groceries
@end smallexample
@smallexample @c output:AECD64E
$ 334.00 Expenses:Food:Groceries $ 334.00 Expenses:Food:Groceries
@end smallexample @end smallexample
@ -474,8 +544,11 @@ $ ledger -f drewr3.dat balance Groceries
If you would like to find transaction to only a certain payee use If you would like to find transaction to only a certain payee use
@samp{payee} or @samp{@@}: @samp{payee} or @samp{@@}:
@smallexample @smallexample @c command:C6BC57E
$ ledger -f drewr3.dat register payee "Organic" $ ledger -f drewr3.dat register payee "Organic"
@end smallexample
@smallexample @c output:C10BC57E
10-Dec-20 Organic Co-op Expense:Food:Groceries $ 37.50 $ 37.50 10-Dec-20 Organic Co-op Expense:Food:Groceries $ 37.50 $ 37.50
Expense:Food:Groceries $ 37.50 $ 75.00 Expense:Food:Groceries $ 37.50 $ 75.00
Expense:Food:Groceries $ 37.50 $ 112.50 Expense:Food:Groceries $ 37.50 $ 112.50
@ -497,8 +570,11 @@ a check to clear, but you should treat it as money spent. The
@command{cleared} report will not format correctly for accounts that @command{cleared} report will not format correctly for accounts that
contain multiple commodities): contain multiple commodities):
@smallexample @smallexample @c command:B86F6A6
$ ledger -f drewr3.dat cleared $ ledger -f drewr3.dat cleared
@end smallexample
@smallexample @c output:B86F6A6
$ -3,804.00 $ 775.00 Assets $ -3,804.00 $ 775.00 Assets
$ 1,396.00 $ 775.00 10-Dec-20 Checking $ 1,396.00 $ 775.00 10-Dec-20 Checking
$ 30.00 0 Business $ 30.00 0 Business
@ -517,8 +593,8 @@ $ ledger -f drewr3.dat cleared
$ -20.00 0 MasterCard $ -20.00 0 MasterCard
$ 200.00 0 Mortgage:Principal $ 200.00 0 Mortgage:Principal
$ -243.60 0 Tithe $ -243.60 0 Tithe
---------------- ---------------- --------- ---------------- ---------------- ---------
$ -243.60 0 $ -243.60 0
@end smallexample @end smallexample
@noindent @noindent
@ -1488,7 +1564,7 @@ entry.
For example, the following entries reflect transaction made for a For example, the following entries reflect transaction made for a
business trip to Europe from the US: business trip to Europe from the US:
@smallexample @smallexample @c input:82150D9
2011/09/23 Cash in Munich 2011/09/23 Cash in Munich
Assets:Cash E50.00 Assets:Cash E50.00
Assets:Checking $-66.00 Assets:Checking $-66.00
@ -1504,8 +1580,11 @@ spent on Dinner in Munich.
Running a ledger balance report shows: Running a ledger balance report shows:
@smallexample @smallexample @c command:82150D9
$ ledger -f example.dat bal $ ledger -f example.dat bal
@end smallexample
@smallexample @c output:82150D9
$-66.00 $-66.00
E15.00 Assets E15.00 Assets
E15.00 Cash E15.00 Cash
@ -3626,7 +3705,7 @@ the money to be evenly distributed over the next six months so that
your monthly budgets gradually take a hit for the vegetables you'll your monthly budgets gradually take a hit for the vegetables you'll
pick up from the co-op, even though you've already paid for them. pick up from the co-op, even though you've already paid for them.
@smallexample @smallexample @c input:6453542
2008/10/16 * (2090) Bountiful Blessings Farm 2008/10/16 * (2090) Bountiful Blessings Farm
Expenses:Food:Groceries $ 37.50 ; [=2008/10/01] Expenses:Food:Groceries $ 37.50 ; [=2008/10/01]
Expenses:Food:Groceries $ 37.50 ; [=2008/11/01] Expenses:Food:Groceries $ 37.50 ; [=2008/11/01]
@ -3644,15 +3723,17 @@ really knows that it debited $225 this month.
And using @option{--effective} option, initial date will be overridden And using @option{--effective} option, initial date will be overridden
by effective dates. by effective dates.
@smallexample @smallexample @c command:6453542
$ ledger --effective register Groceries $ ledger --effective register Groceries
@end smallexample
08-Oct-01 Bountiful Blessi.. Expe:Food:Groceries $ 37.50 $ 37.50 @smallexample @c output:6453542
08-Nov-01 Bountiful Blessi.. Expe:Food:Groceries $ 37.50 $ 75.00 08-Oct-01 Bountiful Blessings.. Expense:Food:Groceries $ 37.50 $ 37.50
08-Dec-01 Bountiful Blessi.. Expe:Food:Groceries $ 37.50 $ 112.50 08-Nov-01 Bountiful Blessings.. Expense:Food:Groceries $ 37.50 $ 75.00
09-Jan-01 Bountiful Blessi.. Expe:Food:Groceries $ 37.50 $ 150.00 08-Dec-01 Bountiful Blessings.. Expense:Food:Groceries $ 37.50 $ 112.50
09-Feb-01 Bountiful Blessi.. Expe:Food:Groceries $ 37.50 $ 187.50 09-Jan-01 Bountiful Blessings.. Expense:Food:Groceries $ 37.50 $ 150.00
09-Mar-01 Bountiful Blessi.. Expe:Food:Groceries $ 37.50 $ 225.00 09-Feb-01 Bountiful Blessings.. Expense:Food:Groceries $ 37.50 $ 187.50
09-Mar-01 Bountiful Blessings.. Expense:Food:Groceries $ 37.50 $ 225.00
@end smallexample @end smallexample
@node Periodic Transactions, Concrete Example of Automated Transactions, Effective Dates, Automated Transactions @node Periodic Transactions, Concrete Example of Automated Transactions, Effective Dates, Automated Transactions
@ -3777,14 +3858,14 @@ options.
The balance report is the most commonly used report. The simplest The balance report is the most commonly used report. The simplest
invocation is: invocation is:
@smallexample @smallexample @c command:1D00D56
$ ledger balance -f drewr3.dat $ ledger balance -f drewr3.dat
@end smallexample @end smallexample
@noindent @noindent
which will print the balances of every account in your journal. which will print the balances of every account in your journal.
@smallexample @smallexample @c output:1D00D56
$ -3,804.00 Assets $ -3,804.00 Assets
$ 1,396.00 Checking $ 1,396.00 Checking
$ 30.00 Business $ 30.00 Business
@ -3811,8 +3892,11 @@ Most times this is more than you want. Limiting the results to
specific accounts is as easy as entering the names of the accounts specific accounts is as easy as entering the names of the accounts
after the command. after the command.
@smallexample @smallexample @c command:06B2AD4
$ ledger balance -f drewr3.dat Auto MasterCard $ ledger balance -f drewr3.dat Auto MasterCard
@end smallexample
@smallexample @c output:06B2AD4
$ 5,500.00 Expenses:Auto $ 5,500.00 Expenses:Auto
$ -20.00 Liabilities:MasterCard $ -20.00 Liabilities:MasterCard
-------------------- --------------------
@ -3826,8 +3910,11 @@ note the implicit logical and between @samp{Auto} and
If you want the entire contents of a branch of your account tree, use If you want the entire contents of a branch of your account tree, use
the highest common name in the branch: the highest common name in the branch:
@smallexample @smallexample @c command:B0468E1
$ ledger balance -f drewr3.dat Income $ ledger balance -f drewr3.dat Income
@end smallexample
@smallexample @c output:B0468E1
$ -2,030.00 Income $ -2,030.00 Income
$ -2,000.00 Salary $ -2,000.00 Salary
$ -30.00 Sales $ -30.00 Sales
@ -3838,15 +3925,25 @@ $ ledger balance -f drewr3.dat Income
You can use general regular expressions in nearly anyplace Ledger You can use general regular expressions in nearly anyplace Ledger
needs a string: needs a string:
@smallexample @smallexample @c command:EAE389F
$ ledger balance -f drewr3.dat ^Bo $ ledger balance -f drewr3.dat ^Bo
@end smallexample
@smallexample @c output:EAE389F
@end smallexample
This first example looks for any account starting with @samp{Bo}, of
which there are none.
@smallexample @c command:E2AF6AD
$ ledger balance -f drewr3.dat Bo $ ledger balance -f drewr3.dat Bo
@end smallexample
@smallexample @c output:E2AF6AD
$ 20.00 Expenses:Books $ 20.00 Expenses:Books
@end smallexample @end smallexample
The first example looks for any account starting with @samp{Bo}, of This second example looks for any account with @samp{Bo}, which is
which there are none. The second looks for any account with @samp{Bo}, @samp{Expenses:Books}.
which is @samp{Expenses:Books}.
@cindex limit by payees @cindex limit by payees
@findex --limit @var{EXPR} @findex --limit @var{EXPR}
@ -4964,7 +5061,7 @@ earlier postings. Here's how it works:
Say you currently have this posting in your ledger file: Say you currently have this posting in your ledger file:
@smallexample @smallexample @c input:03ACB97
2004/03/15 * Viva Italiano 2004/03/15 * Viva Italiano
Expenses:Food $12.45 Expenses:Food $12.45
Expenses:Tips $2.55 Expenses:Tips $2.55
@ -4975,17 +5072,17 @@ Now it's @samp{2004/4/9}, and you've just eating at @samp{Viva Italiano}
again. The exact amounts are different, but the overall form is the again. The exact amounts are different, but the overall form is the
same. With the @command{xact} command you can type: same. With the @command{xact} command you can type:
@smallexample @smallexample @c command:03ACB97
$ ledger xact 2004/4/9 viva food 11 tips 2.50 $ ledger xact 2004/4/9 viva food 11 tips 2.50
@end smallexample @end smallexample
This produces the following output: This produces the following output:
@smallexample @smallexample @c output:03ACB97
2004/04/09 Viva Italiano 2004/04/09 Viva Italiano
Expenses:Food $11.00 Expenses:Food $11.00
Expenses:Tips $2.50 Expenses:Tips $2.50
Liabilities:MasterCard $-13.50 Liabilities:MasterCard
@end smallexample @end smallexample
It works by finding a past posting matching the regular expression It works by finding a past posting matching the regular expression
@ -5596,22 +5693,23 @@ Format Codes}).
@item --master-account @var{STR} @item --master-account @var{STR}
Prepend all account names with the argument. Prepend all account names with the argument.
@smallexample @smallexample @c command:A76BB56
$ ledger -f test/input/drewr3.dat bal --master-account HUMBUG $ ledger -f drewr3.dat bal --no-total --master-account HUMBUG
@end smallexample
@smallexample @c output:A76BB56
0 HUMBUG 0 HUMBUG
$ -3,804.00 Assets $ -3,804.00 Assets
$ 1,396.00 Checking $ 1,396.00 Checking
$ 30.00 Business $ 30.00 Business
$ -5,200.00 Savings $ -5,200.00 Savings
$ 20.00 Books
$ -1,000.00 Equity:Opening Balances $ -1,000.00 Equity:Opening Balances
$ 6,634.00 Expenses $ 6,654.00 Expenses
$ 11,000.00 Auto $ 5,500.00 Auto
$ 20.00 Books $ 20.00 Books
$ 300.00 Escrow $ 300.00 Escrow
$ 334.00 Food:Groceries $ 334.00 Food:Groceries
$ 500.00 Interest:Mortgage $ 500.00 Interest:Mortgage
$ -5,520.00 Assets:Checking
$ -2,030.00 Income $ -2,030.00 Income
$ -2,000.00 Salary $ -2,000.00 Salary
$ -30.00 Sales $ -30.00 Sales
@ -6462,7 +6560,7 @@ In the balance report, it shows all the accounts affected by
transactions having a related posting. For example, if a file had transactions having a related posting. For example, if a file had
this transaction: this transaction:
@smallexample @smallexample @c input:94C5675
2004/03/20 Safeway 2004/03/20 Safeway
Expenses:Food $65.00 Expenses:Food $65.00
Expenses:Cash $20.00 Expenses:Cash $20.00
@ -6471,16 +6569,16 @@ this transaction:
And the register command was: And the register command was:
@smallexample @smallexample @c command:94C5675
$ ledger -r register food $ ledger -f example.dat -r register food
@end smallexample @end smallexample
The following would be output, showing the postings related to the The following would be output, showing the postings related to the
posting that matched: posting that matched:
@smallexample @smallexample @c output:94C5675
2004/03/20 Safeway Expenses:Cash $-20.00 $-20.00 04-Mar-20 Safeway Expenses:Cash $20.00 $20.00
Assets:Checking $85.00 $65.00 Assets:Checking $-85.00 $-65.00
@end smallexample @end smallexample
@item --budget @item --budget

12
test/CMakeLists.txt
View file

@ -38,4 +38,16 @@ add_subdirectory(manual)
add_subdirectory(baseline) add_subdirectory(baseline)
add_subdirectory(regress) add_subdirectory(regress)
if(PYTHONINTERP_FOUND)
set(_class DocTests)
file(GLOB ${_class}_TESTS ${PROJECT_SOURCE_DIR}/doc/*.texi)
foreach(TestFile ${${_class}_TESTS})
get_filename_component(TestFile_Name ${TestFile} NAME_WE)
add_test(${_class}Test_${TestFile_Name}
${PYTHON_EXECUTABLE} ${PROJECT_SOURCE_DIR}/test/DocTests.py
--ledger ${LEDGER_LOCATION} --file ${TestFile})
set_target_properties(check PROPERTIES DEPENDS ${_class}Test_${TestFile_Name})
endforeach()
endif()
### CMakeLists.txt ends here ### CMakeLists.txt ends here

185
test/DocTests.py Executable file
View file

@ -0,0 +1,185 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import os
import re
import sys
import hashlib
import argparse
import subprocess
from difflib import unified_diff
class DocTests:
def __init__(self, args):
scriptpath = os.path.dirname(os.path.realpath(__file__))
self.ledger = os.path.abspath(args.ledger)
self.sourcepath = os.path.abspath(args.file)
self.verbose = args.verbose
self.examples = dict()
self.test_files = list()
self.testin_token = 'command'
self.testout_token = 'output'
self.testdat_token = 'input'
def read_example(self):
endexample = re.compile(r'^@end\s+smallexample\s*$')
example = str()
while True:
line = self.file.readline()
self.current_line += 1
if len(line) <= 0 or endexample.match(line): break
example += line
return example
def test_id(self, example):
return hashlib.sha1(example.rstrip()).hexdigest()[0:7].upper()
def find_examples(self):
startexample = re.compile(r'^@smallexample\s+@c\s+(%s|%s|%s)(?::([\dA-Fa-f]+))?'
% (self.testin_token, self.testout_token, self.testdat_token))
while True:
line = self.file.readline()
self.current_line += 1
if len(line) <= 0: break
startmatch = startexample.match(line)
if (startmatch):
test_begin_pos = self.file.tell()
test_begin_line = self.current_line
test_kind = startmatch.group(1)
test_id = startmatch.group(2)
example = self.read_example()
test_end_pos = self.file.tell()
test_end_line = self.current_line
if not test_id:
print >> sys.stderr, 'Example', test_kind, 'in line', test_begin_line, 'is missing id.'
test_id = self.test_id(example)
if test_kind == self.testin_token:
print >> sys.stderr, 'Use', self.test_id(example)
elif test_kind == self.testin_token and test_id != self.test_id(example):
print >> sys.stderr, 'Expected test id', test_id, 'for example' \
, test_kind, 'on line', test_begin_line, 'to be', self.test_id(example)
try:
self.examples[test_id]
except KeyError:
self.examples[test_id] = dict()
self.examples[test_id][test_kind] = {
'bpos': test_begin_pos,
'epos': test_end_pos,
'blin': test_begin_line,
'elin': test_end_line,
test_kind: example,
}
def test_examples(self):
failed = set()
for test_id in self.examples:
example = self.examples[test_id]
try:
command = example[self.testin_token][self.testin_token]
except KeyError:
command = None
try:
output = example[self.testout_token][self.testout_token]
except KeyError:
output = None
try:
input = example[self.testdat_token][self.testdat_token]
except KeyError:
input = None
if command and output:
command = command.rstrip().split()
if command[0] == '$': command.remove('$')
index = command.index('ledger')
command[index] = self.ledger
command.insert(index+1, '--init-file')
command.insert(index+2, '/dev/null')
try:
findex = command.index('-f')
except ValueError:
try:
findex = command.index('--file')
except ValueError:
findex = index+1
command.insert(findex, '--file')
command.insert(findex+1, test_id + '.dat')
if findex:
scriptpath = os.path.dirname(os.path.realpath(__file__))
test_input_dir = scriptpath + '/../test/input/'
test_file = command[findex+1]
test_file_created = False
if not os.path.exists(test_file):
if input:
test_file_created = True
with open(test_file, 'w') as f:
f.write(input)
elif os.path.exists(test_input_dir + test_file):
command[findex+1] = test_input_dir + test_file
try:
verify = subprocess.check_output(command)
except:
verify = str()
if test_file_created:
os.remove(test_file)
valid = (output == verify)
if self.verbose > 0:
print test_id, ':', 'Passed' if valid else 'FAILED'
else:
sys.stdout.write('.' if valid else 'E')
if not valid:
failed.add(test_id)
if self.verbose > 1:
print ' '.join(command)
for line in unified_diff(output.split('\n'), verify.split('\n'), fromfile='generated', tofile='expected'):
print(line)
print
if not self.verbose:
print
if len(failed) > 0:
print "\nThe following examples failed:"
print " ", "\n ".join(failed)
return len(failed)
def main(self):
self.file = open(self.sourcepath)
self.current_line = 0
self.find_examples()
failed_examples = self.test_examples()
self.file.close()
return failed_examples
if __name__ == "__main__":
def getargs():
parser = argparse.ArgumentParser(description='DocTests', prefix_chars='-')
parser.add_argument('-v', '--verbose',
dest='verbose',
action='count',
help='be verbose. Add -vv for more verbosity')
parser.add_argument('-l', '--ledger',
dest='ledger',
type=str,
action='store',
required=True,
help='the path to the ledger executable to test with')
parser.add_argument('-f', '--file',
dest='file',
type=str,
action='store',
required=True,
help='the texinfo documentation file to run the examples from')
return parser.parse_args()
args = getargs()
script = DocTests(args)
status = script.main()
sys.exit(status)