Opened 12 years ago

Closed 11 years ago

#4651 closed enhancement (fixed)

[patch] [cla] Adding the module dojox.testing.DocTest

Reported by: wolfram Owned by: alex
Priority: high Milestone: 1.2
Component: Dojox Version: 0.9
Keywords: Cc:
Blocked By: Blocking:

Description (last modified by wolfram)

This class executes doctests. DocTests? are tests that are normally defined inside the comment, a doctest looks as if it was copied from the shell (which it mostly is). The lines of the test to execute start with ">>>" and the first line following that doesnt start like it is the exptected result. I.e. the following is a simple doctest, that will actually also be run if you run this class against this file here:

>>> 1+1
2
>>> "a"+"b"
"ab"

To run the doctests of the DocTest? class itself, go in the FireBug? console and type:

>>> dojo.require("dojox.testing.DocTest"); 
>>> var doctest = new dojox.testing.DocTest(); 
>>> doctest.run("dojox.testing.DocTest"); 

it will then print out on the firebug console:

Test 1: OK: 1+1 // A simple test case. Terminated by an empty ...
Test 2: OK: 1==2 
Test 3: OK: "a"+"b" // Also without the empty line before, thi...
Test 4: OK: var anything = "anything" // Multiple commands for...
Test 5: OK: true==true // Test a new line terminating the test...
Test 6: OK: true==true // Test a new test terminating the test...
Test 7: OK: true==true // Test a "not a comment"-line, especia...
Test 8: OK: [1,2,3,4] 
Test 9: OK: true==true // Test code on new line terminating th...
Test 10: OK: false 
Test 11: OK: "false"
Test 12: OK: true 
Test 13: OK: 1 
Test 14: OK: "s"
Test 15: OK: dojo.toJson({one:1}) 

for integrating it into your unittests you can simply check the property error, see the firebug output for what how it works:

>>> doctest.errors.length==0
true

Find some more in depth discussion in this article.

Attachments (8)

DocTest.patch (9.2 KB) - added by wolfram 12 years ago.
A much more stable version
some_dojo_doctests.patch (8.5 KB) - added by wolfram 12 years ago.
Added some doctests for dojo._base, modified some existing examples into doctests
DocTest.js (7.4 KB) - added by alex 12 years ago.
DocTest.2.js (7.9 KB) - added by wolfram 12 years ago.
modified to work with doh
doh_runner.diff (951 bytes) - added by wolfram 12 years ago.
doh enabled to run DocTests?
DocTest.3.js (7.9 KB) - added by wolfram 12 years ago.
Tiny fix, proper line numbering
doh_runner.2.diff (1.2 KB) - added by wolfram 12 years ago.
Make it a little more verbose, use the comment on the first line as text-name
DocTest.diff (9.8 KB) - added by wolfram 12 years ago.
Some bug fixes, add self testing

Download all attachments as: .zip

Change History (18)

comment:1 Changed 12 years ago by alex

Milestone: 1.1
Owner: changed from Tom Trenka to alex
Status: newassigned

comment:2 Changed 12 years ago by wolfram

Description: modified (diff)

Changed 12 years ago by wolfram

Attachment: DocTest.patch added

A much more stable version

Changed 12 years ago by wolfram

Attachment: some_dojo_doctests.patch added

Added some doctests for dojo._base, modified some existing examples into doctests

comment:3 Changed 12 years ago by wolfram

Description: modified (diff)

comment:4 Changed 12 years ago by wolfram

Description: modified (diff)

Changed 12 years ago by alex

Attachment: DocTest.js added

comment:5 Changed 12 years ago by alex

I've attached version of DocTest?.js that more closely meets the style guide. I am, however, worried that there's no integration with DOH in this module. Have you evaluated providing a way to plug into to DOH with doctesting?

Changed 12 years ago by wolfram

Attachment: DocTest.2.js added

modified to work with doh

Changed 12 years ago by wolfram

Attachment: doh_runner.diff added

doh enabled to run DocTests?

comment:6 Changed 12 years ago by wolfram

To use doh to include the doctests into unittesting, just include the following line in your testfile:

    tests.registerDocTests("dojox.data.QueryReadStore");

Or like so

    tests.registerDocTests("dojox.testing.DocTest");

Each doctest is listed as a single testcase inside a new test group which's name is "DocTest?: <modulename>"

comment:7 Changed 12 years ago by wolfram

Hi Alex,

so I talked to Neil, and if I understood him right he sees the DocTests? as rather a very low-priority feature. But as I understood, it also doesn't interfer with the examples in the comments either, though he even would prefer to reduce the amount of inline-comment to as little as possible.

Whereas I think DocTests? in the comments are a great help for seeing how a function ticks. And just for the records, DocTests? are not suited for all kinds of tests, and should also just be used where small one- or two-liners are valid tests (like dojo.string.substitute, dojo.trim, and alikes).

I just would like to sum the advantages:

  • if DocTests? are also generated into the api-docs you give the programmers valid examples at hand
  • if DocTests? are integrated in the default test suite you can even make sure that the exmples in the api-doc _always_work_
  • copying the tests from the FireBug? command line is just much less work than writing an extra test (and I guess a lot of functionality gets tested using FireBug?)
  • if one changes the code, the DocTest? is closer at hand and the chance it is adjusted to the changed behaviour of a function is much more likely than with external unit tests
  • DocTests? are tests right there with the code
  • they make it quicker to get an overview of what the function does

End of my plea :-)

Wolfram

PS: I also just learned to love DocTests? when I used them to learn by the Django's newforms tests, since then I love them :-)

comment:8 Changed 12 years ago by guest

+1 to getting this in upstream. The DocTest? approach is hard to let go of once you're familiar with it.

Wolfram, note that part of the strength of doctests is in writing self-testing tutorials and documentation. You might consider setting things up so you can run doctests at the click of a button, similar to Ian Bicking's self-testing documentation at http://svn.colorstudy.com/doctestjs/trunk/docs/index.html

Changed 12 years ago by wolfram

Attachment: DocTest.3.js added

Tiny fix, proper line numbering

Changed 12 years ago by wolfram

Attachment: doh_runner.2.diff added

Make it a little more verbose, use the comment on the first line as text-name

comment:9 Changed 12 years ago by wolfram

Summary: [patch] [CLI] Adding the module dojox.testing.DocTest[patch] [cla] Adding the module dojox.testing.DocTest

Changed 12 years ago by wolfram

Attachment: DocTest.diff added

Some bug fixes, add self testing

comment:10 Changed 11 years ago by alex

Resolution: fixed
Status: assignedclosed

(In [14046]) adding Wolfram's DocTest? module. Nice hacking Wolfram! Fixes #4651 !strict

Note: See TracTickets for help on using tickets.