Opened 15 years ago

Closed 13 years ago

Last modified 12 years ago

#2675 closed enhancement (wontfix)

Multiple updates for doc parser

Reported by: [email protected] Owned by: Neil Roberts
Priority: high Milestone: future
Component: Doc parser Version: 0.4.2
Keywords: Cc: [email protected]
Blocked By: Blocking:

Description (last modified by Neil Roberts)

The attached patch is made up of the following parts:

1) fixes to the tree walker to allow the parser to run on non dojo trees (also simplifies it)

2) Adds 'tags' as a special comment field

3) Allows filtering of what is included in the output files based on the tags, private members and can strip source.

4) Allows configuration of which paths, namespace prefixes and filters are run from the command line

Tags work as follows

        Add tags to doc parser. These could be useful for segmenting generated 
        documentation for different audiences. (eg. internal team vs clients, 
        contribs vs users)
        When the docparser runs it would produce a set of tags for each 
        1. They could be used to filter the parser output before it was used in 
        the ApiRef so there is no overhead of downloading stuff that is 
        never/should never be displayed.
        2. They could be used in the display of documentation in any form, or as 
        it is converted to another form (XSD).
        In source, tags are specified whereever you can have a summary. Comma or 
        whitespace separated. If we can get them added to other places easily 
        then cool.
        function foo() {
          //tags: internal, special_tag
          //summary: foo function
        In XML tags are added under different items as:
          ...  <tag>internal</tag><tag>special-tag</tag> ...
        This makes it fairly straightforward to use them in XSD or when parsing 
        with code.
        In JSON tags are presented as an array:
        { ... , tags: ['internal', 'special-tag'], ... }
        Use Case 1:
        In our map product we have a series of markermanagers that control how 
        markers are drawn. One thing they handle is whether and how markers are 
        clustered together.
        As a client, when you initialise the map you can specify a manager to 
        use. But apart from specifying it,
        the manager is internal and there is no client API for it. But because 
        there are a number of managers they have a documented API. Just not for 
        clients to use. So this could be tagged 'internal'.
        Use Case 2:
        Say use of an object or method is deprecated after V1. It should be 
        marked as such in the V2 docs and hidden by default. This item could be 
        tagged 'deprecated'.

Attachments (4)

dojo_doc_parser_patch (15.6 KB) - added by [email protected] 15 years ago.
parser_filters_diff (3.1 KB) - added by [email protected] 15 years ago.
Patch to add filter options to the parser
parser_namespace_diff (4.2 KB) - added by [email protected] 15 years ago.
adds support for user definable namespaces/paths
parser_tags_diff (4.6 KB) - added by [email protected] 15 years ago.
adds custom tags for filtering documentation

Download all attachments as: .zip

Change History (12)

Changed 15 years ago by [email protected]

Attachment: dojo_doc_parser_patch added

comment:1 Changed 15 years ago by [email protected]

Since some people have asked, I've added some brief docs on how to use the changes here. These should be added to if/when the patch is accepted. Comments and suggestions are welcome!

  • Parsing another namespace or set of code besides dojo. This will find all JS files under the specified path, and use the specified namespace. The default is to parse the dojo code. Once you've done this you can view the results immediately by opening /dojo/api/index.html.
    php5 -d memory_limit=100M -d error_reporting=2039 docparser.php --path=/path/to/mynamespace --namespace=mynamespace
  • Passing --filter_source will exclude any source snippets from the JSON and XML output.
  • Passing --filter_private will exclude any variables or methods named with an '_' prefix from the JSON and XML output.


Tags you can add to your code like this:

  // tags: mycustomtag, deprecated, internal_only
  // summary: Soon we will have enough treasure to rule all of New Jersey.
  // description: Or we could just get a new roomate.
  //    Look, you go find him.  He don't yell at you.
  //    All I ever try to do is make him smile and sing around
  //    him and dance around him and he just lays into me.
  //    He told me to get in the freezer 'cause there was a carnival in there.
  // returns:  Look, a Bananarama tape!

You can add them anywhere you can put a summary note currently (where that is we'll leave as an exercise for the reader). They can be letters, numbers and underscores and are case sensitive. These get added to a tags block in the JSON and XML output, and are currently unused by anything.

However, you can filter the docparser output using tags. For example:

  • to exclude all the items tagged 'deprecated': --filter_exclude_tags=deprecated
  • to include only the items tagged 'new': --filter_include_only_tags=new
  • specify more than one tag for these options separated by commas.
  • you can combine filter_exclude_tags and filter_include_only_tags.

comment:2 Changed 15 years ago by [email protected]

Note that this patch applies to HEAD after [7844] which fixes some issue with parsing dojo.declare() statements.

comment:3 Changed 15 years ago by [email protected]

Apologies if I'm being dense, but your patch cites (revision 656). That's obviously not a dojo SVN revision #. If I switch that to 7844, tortoiseSVN informs me the "The patch seems outdated" .. the file line and patch line do not match. I've got todays HEAD, but I've also tried against 0.4.2

comment:4 Changed 15 years ago by [email protected]

Possibly the patch was from our vendor branch, I did check it applied against head but who knows what happened, I have attached three new patches which separately add: tags, filters and namespace setting. These are definitely against head as of 3 mins ago [7906].

Changed 15 years ago by [email protected]

Attachment: parser_filters_diff added

Patch to add filter options to the parser

Changed 15 years ago by [email protected]

Attachment: parser_namespace_diff added

adds support for user definable namespaces/paths

Changed 15 years ago by [email protected]

Attachment: parser_tags_diff added

adds custom tags for filtering documentation

comment:5 Changed 14 years ago by dante

did this ever get considered? merged? deprecated?

comment:6 Changed 14 years ago by dylan

Milestone: 1.1

Neil, please decide what to do with this prior to 1.1 final.

comment:7 Changed 13 years ago by bill

Milestone: 1.2future

defer dormant enhancements to "future" (they don't look like they're getting fixed soon so might as well mark as such)

comment:8 Changed 13 years ago by Neil Roberts

Description: modified (diff)
Resolution: wontfix
Status: newclosed

New parser works differently than this patch. Will add stripping features if requested.

Note: See TracTickets for help on using tickets.