About Lectures Research Software Blog
Musical Site
MySpace
Facebook

Moods Blog

Dojo Shin Kaï

RSS Feed
Thank you!

XHTML 1.0 conformant
CSS 2.0 conformant
Didier Verna's scientific blog: Lisp, Emacs, LaTeX and random stuff.

Tag - Texinfo

Entries feed - Comments feed

Tuesday, May 10 2022

Declt 4.0 beta 1 "William Riker" is released

Today, after two years and a half of (irregular) development, and 465 commits, I have released the first beta version of Declt 4.0, my reference manual generator for Common Lisp libraries.

I seldom release official beta versions of software, let alone make announcements about them, but this one deserves an exception. Since version 3, Declt has been undergoing a massive, SOC-oriented, overhaul. The intent is to reimplement it into a clean 3-stages pipeline. Stage one, the "assessment" stage is in charge of gathering information. Stage two, the "assembly" stage, creates a documentation template specifying how the information is to be organized. Finally, Stage three, the "typesetting" stage, renders the template in a specific output format. Of course, the purpose is to eventually be able to support different kinds of templates and output formats.

Declt 4.0b1 marks the achievement of Stage one, which is now complete. It also contains a lot of new (and long awaited) features and improvements. The user manual has been updated to provide an overview of the new architecture, along with some details about Stage one of the pipeline. From now on, I'm also going to release new versions of Quickref to closely follow the evolution of Declt.

Apart from the aforementioned architecture overhaul, which really boils down to internals shaking and shouldn't affect the end-user much, a lot of new features and improvements are also provided in this release. They are listed below.

Backward Incompatible Changes

For the benefit of Separation of Concerns, but also for other reasons, a number of keyword arguments to the declt function have been renamed. :hyperlinks becomes :locations, :version becomes :library-version, :texi-directory becomes :output-directory, and :texi-name becomes :file-name.

New Features

Default / Standard Values

Declt now has the ability to advertise or hide properties that get default / standard values (e.g. standard method combination, :instance slot allocation, etc.).

Support for Aliases

Declt now recognizes and advertises "aliases", that is, funcoids which have their fdefinition, macro, or compiler macro function set manually to another, original, definition.

Support for New Programmatic Entities

Typed structures and setf compiler macros are now properly detected and documented with all the specific information.

Proper Support for Uninterened Symbols

Such symbols may be encountered on several occasions such as slot names (trivialib, for example, defines structures with uninterned slot names in order to prevent access to them). Definitions named with uninterned symbols are considered private, and denoted by the empty set package (∅).

SBCL 2.1.2 Required

The following enhancements depend on it:

  • short form setf expanders now get correct source information,
  • method combinations lambda-lists are now documented.

Domestic vs. Foreign Definitions

The concept of domestic (as opposed to foreign) definition has been extended to include those created in one of the sources files of the library being documented, even if the symbol naming the definition is from a foreign package. If the source file is unknown, then the symbol's package must be domestic. The general intent is to consider language extensions as domestic, hence, always documented. For example, a new method on a standard generic function (say, initialize-instance) will now always appear in the documentation.

This refinement is accompanied by a new option allowing to include foreign definitions in the generated documentation. Of course, only foreign definitions somehow related to the library being documented will appear in the documentation. Also, those definitions will in general be partial: only the parts relevant to the library being documented will appear. Continuing with the initialize-instance example, new methods normally appear as toplevel, standalone definitions in the documentation (and their parent generic function is simply mentioned). If foreign definitions are included however, there will be a toplevel entry for the generic function, and all methods defined in the library will be documented there (truly foreign methods won't appear at all).

Introspection Heuristic

Until now, there was a single heuristic for finding domestic definitions, which was to start from domestic packages and scan as many connections between symbols as possible. That heuristic is reasonably fast, but may occasionally miss some domestic definitions. Declt now has the ability to scan all symbols in the Lisp image instead of only the ones from domestic packages. This new option ensures that all domestic definitions are found, at the expense of a much greater computation time.

Supported Licenses

The Microsoft Public License has been added.

Info Installation Category

In other words, the value of Texinfo's @direntry command is now customizable (instead to being hardwired to "Common Lisp").

Improvements

Documentation Thinning

Packages reference lists do not point to methods directly anymore (as they can be reached via the generic function's reference). Also, only slots for which the parent classoid is named from another package are now referenced.

Files reference lists do not point to slots anymore (as they can be reached via the parent classoid's reference). Also, only methods for which the parent generic function is defined in another file are now referenced.

The readability of long references has been improved. In particular, they don't advertise the type of the referenced definitions anymore, when there is no ambiguity.

Slot documentation now advertises the slot name's package only when different from that of the parent classoid.

Non standalone method documentation now advertises the source file only when different from that of the parent generic function.

The rendering of EQL specializers has been inmproved.

The documentation of setf / writer methods doesn't render the "new value" argument / specializer anymore.

Merging

Generic definitions containing only reader / writer methods are upgraded to specific reader / writer categories, and definition merging is now only attempted on those.

Lambda Lists

Uninformative parts of lambda lists are now filtered out. This includes &whole, &environment, &aux variables, along with options / keyword variables and default values.

Method specializers in lambda lists now link back to their respective class definitions.

Method Combinations

The method combination discovery scheme has been upgraded to benefit from SBCL 1.4.8's enhancements (themselves following my ELS 2018 paper). The old code didn't break, but prevented unused method combinations from being detected.

Bug Fixes and Workarounds

ASDF

Better handling of missing / unloaded components or dependencies (this can happen for instance with feature-dependent conditional inclusion).

Cope with the lack of specification of the license information in ASDF systems by coercing to a string.

Fix several cases of system files documentation duplication. Declt automatically documents .asd files as special cases of Lisp files. However, some systems have the bad (IMHO) habit of mentioning them explicitly as components (e.g. static files). When this happens, Declt silently discards that definition and keeps its own (at the expense of having a slightly incorrect system documentation).

Anchoring

After years of vain attempts at providing human-readable yet unique anchor names (which was only really useful for Info, BTW), I finally got rid of my last bit of idealism. Anchor names now use numerical definition UIDs, and since Texinfo allows me to expand references with some more information than just the anchor, it's good enough. Besides, it fixes the last remaining rare cases exhibiting the fact that it's just impossible to have anchors that are both human-readable and unique.

Setf Expanders

Fix the computation of short form setf epxander lambda lists (which didn't correctly handle the presence of optional or rest arguments before.

Handle the potential unavailability of a setf expander's update function or short form operator. Document it if applicable. Also signal a warning when the expander is domestic.

Tuesday, November 19 2019

Declt 3.0 "Montgomery Scott" is released

I'm pleased to announce the release of Declt 3.0 "Montgomery Scott", my Texinfo reference manual generator for Common Lisp libraries.

This is a new major release of the library, although the changes may not be so visible from the outside. The main concern in this release has been to increase the robustness of the output, from three different angles.

  1. Many places from where Declt attempts to extract meaningful information are underspecified (ASDF system slots notably).
  2. The pretty-printing of Lisp items can be difficult, given the liberalism and extensibility of the language.
  3. Finally, several restrictions in the syntax of Texinfo itself (anchor names notably) get in the way.

These issues were described in a paper presented at the TeX Users Group conference, this summer in Palo Alto. This release goes a long way toward fixing them.

The new generated reference manuals look mostly the same as before, but some important things have changed under the hood, notably the names of hyperlinks and cross-references (backward-incompatible, hence the increase in the major version number). The output is now also much more robust with respect to the final output format: the generation of HTML, DVI, Postscript, PDF, and even plain Info should work fine and has been tested on all Quicklisp libraries (in fact, Quickref will also be upgraded in the near future to provide all those formats at once).

Those improvements do come at a cost. Unicode is now required (even for Info readers). To be honest, many Lisp libraries already had that implicit requirement. Also, this release depends on improvements and bug fixes only available in Texinfo 6.7, now a requirement as well. I have to thank Gavin Smith, from the Texinfo team, for his collaboration.

Apart from that, this release also has a number of bug fixes.

  • Some method specializers were not handled properly.
  • The manuals were missing documentation for some non-Lisp source files.
  • There were some glitches in the pretty-printing of unreadable objects.

Get it at the usual place, and enjoy!

Monday, October 16 2017

Declt 2.3 "Robert April" is out

I'm happy to announce the release of Declt 2.3. Declt is my reference manual generator for Common Lisp libraries.

The improvements and bug fixes in the last two releases are the result of running Declt against the whole Quicklisp world (around 3000 ASDF systems for 1500 libraries). See this post for more information.

New in this release:

  • Advertise file extensions in references.
  • Advertise the type of foreign definitions.
  • More robust display and indexing of, and with, lambda-lists.
  • Use UTF8 special characters to denote invisble ones.
  • More robust support for Texinfo brace escaping.
  • Handle modules sharing the same location.
  • Ensure output is done with standard IO syntax.
  • Fix potential duplication of some (non-lisp) files and document all static files.
  • Fix potential duplication of packages documentation.

From the 2.2 "Christopher Pike" release (not previously advertised):

  • Require a UTF-8 environment.
  • Understand ASDF's notion of inferred system, and also be more protective against ASDF extensions.
  • Support for improper lambda lists (e.g. destructuring ones).
  • Improve contact defaulting code.
  • Update support for SBCL's setf expanders introspection.
  • Accept ASDF system designators.
  • Various bug fixes in the areas of method combinations, accessor definition merging and setf expanders.

Find it at the usual place...

Saturday, August 24 2013

Declt 1.0 is out

After 15 betas, I'm happy enough with the current state of Declt to finally make a 1.0 release.

A lot of things have changed since the previous version, sometimes in a backward-incompatible way. Many more items are documented (including, as you have recently seen, method combinations). In addition to the reference manual, generated by Declt itself, there is now a real user manual.

Declt is still SBCL-only, requires ASDF 3 and Texinfo 4 but generates code that is compatible with Texinfo 5. Also, beware, I've deleted the old repository and moved the project to GitHub. Below is a more precise description of what Declt currently does.

Declt (pronounce "dec'let") is a reference manual generator for Common Lisp libraries. It works by loading an ASDF system and introspecting its contents. The generated documentation contains the description for the system itself and its components (modules and files), the packages defined in that system and the definitions found in those packages.

Exported and internal definitions are listed separately. This allows the reader to have a quick view on the library's public API. Within each section, definitions are sorted lexicographically.

In addition to ASDF system components and packages, Declt documents the following definitions: constants, special variables, symbol macros, macros, setf expanders, compiler macros, functions (including setf ones), generic functions and methods (including setf ones), method combinations, conditions, structures, classes and types.

The generated documentation includes every possible bit of information that introspecting can provide: documentation strings, lambda lists (including qualifiers and specializers where appropriate), slots (including type, allocation and initialization arguments), definition source file etc.

Every documented item provides a full set of cross-references to related items: ASDF component dependencies, parents and children, classes direct methods, super and subclasses, slot readers and writers, setf expanders access and update functions etc.

Finally, Declt produces exhaustive and multiple-entry indexes for every documented item.

Reference manuals are generated in Texinfo format (compatible, but not requiring Texinfo 5). From there it is possible to produce readable / printable output in info, HTML, PDF, DVI and PostScript with tools such as makeinfo, texi2dvi or texi2pdf.

The Declt reference manual is the primary example of documentation generated by Declt itself.

Wednesday, June 29 2011

Declt 1.0b12 is out

I've just released the next version of Declt, my reference manual generator for ASDF systems.

This release includes some fixes for the Info format target but most notably, support for documenting (generic) writer functions and methods. When it makes sense, the documentations for func and (setf func) are grouped together. Getting closer to an official 1.0 stable version...

Grab it here, and enjoy!

Tuesday, May 31 2011

Declt 1.0b11 is out

I've just released a new version of Declt, my Texinfo reference manual generator for ASDF systems. This release contains only one bugfix: when trying to create links to source files, Declt now checks whether the files actually exist or not.

Tracking this bug down had the side-effect of exhibiting a misfeature of SBCL's introspection facility: the COPY-<struct> functions (automatically generated by defstruct calls) have their definition source set to target-defstruct.lisp which is an SBCL source file. It would make more sense to set it to the file containing the defstruct call instead, as is already the case for constructors, predicates and accessor functions. Patch sent to the SBCL developers.

French Flag English Flag
Copyright (C) 2008 -- 2018 Didier Verna didier@lrde.epita.fr