Documenting Common Lisp Projects for GitHub

I want to be able to automatically create the README.md for GitHub whenever I create a new Common Lisp project.  I was impressed by the number of systems that exist for documenting a Common Lisp package.  However, I couldn’t find anything that was simple and that created Markdown output, like the README.md that GitHub uses.

I was in a hurry, so after a few minutes of Googling, I decided to create my own function for documenting packages for GitHub.  It worked.  Sort of.  It documents function only, not macros or other things.  Here’s the function:

(defun document-package (package output-filename)
  "Documents the Common Lisp package PACKAGE and writes that documentation to the file given by OUTPUT-FILENAME."
  (loop for function being the external-symbols of (find-package package)
     when (and (fboundp function) (documentation (symbol-function function) t))
     collect
       (list :function function
             :function-name (string-downcase function)
             :documentation (documentation (symbol-function function) t))
     into functions
     finally
       (return (loop for function in 
                    (sort functions #'string<
                          :key (lambda (x) (getf x :function-name)))
                  collect (format nil "## ~a ~a~%~a"
                                  (string-downcase (getf function :function-name))
                                  (replace-regexs
                                   (format nil "~s"
                                           (sb-introspect:function-lambda-list
                                            (symbol-function (getf function :function))))
                                   '(("\\s\\s+" " ")
                                     ("DC-UTILITIES::" "")))
                                  (getf function :documentation))
                  into function-docs
                    finally (spew (format nil "# ~a~%~{~a~^~%~%~}" package function-docs)
                                  output-filename)))))

That function, when called like this: (document-package :dc-utilities “~/README.md”) produces Markdown that looks like this:

# DC-UTILITIES
## alist-values (ALIST &REST KEYS)
Returns the values associated with KEYS in ALIST.  ALIST is an associative list.

## bytes-to-uint (BYTE-LIST)
Converts the list of bytes BYTE-LIST into an unsigned integer.

## change-per-second (FUNCTION-OR-SYMBOL &OPTIONAL (SECONDS 1))
Given the function FUNCTION-OR-SYMBOL, who's return value changes over time, or a variable who's value changes over time, with the change being unidirectional, this function computes the rate of change by calling the function, sleeping, then calling the function again, then computing the rate of change per second.  You can optionally specify the number of seconds to wait between calls.  If FUNCTION-OR-SYMBOL is a variable, then this function retrieves the value of the variable, sleeps, then retrieves the value of the variable again.

## create-directory (DIR &KEY WITH-PARENTS)
Works just like the mkdir shell command.  DIR is the directory you want to create. Use WITH-PARENTS if you want the function to create parent directories as necessary.

## cull-named-params (NAMED-PARAMS CULL-KEYS)
Given a value for NAMED-PARAMS like this one

    '(:one 1 :two 2 :three 3)

and a list of CULL-KEYS like this one

    '(:one :two)

this function returns a list of named parameters that excludes the names (and their values) that match the names in CULL-KEYS.  In the above example, the result is

    '(:three 3)

## directory-exists (PATH)
Returns a boolean value indicating if the directory specified by PATH exists.

.
.
.

Which looks like this on GitHub:

Is there a package out there already that can do this better?  If so, please leave me a comment.  If not, I need to find out how to get the documentation for a macro, for a method, for a special variable, for a class, and so on.  Any help would be greatly appreciated.

Other things I want this function to do:

  • Check the .asd file to see if there’s a license bit: :license "MIT License"If there is, the code should check to see if there’s a LICENSE file in the project.  If there isn’t, the code should determine if the name of the license is known.  If the code is able to get the text for the license, it should include it at the beginning of the documentation.
  • Optionally produce a table of contents.
  • Output other formats (aside from Markdown) as well, like Text or Emacs Org-Mode.

For now, the document-package function is just a function in the dc-utilities package.  However, once I flesh the function out a little, I will probably move it to its own repo.  If you get ahead of me with any of this, please send me a note.  I could really use this functionality.

Cleaning House by Open Sourcing

I’m going to start open sourcing some of the code that I currently have in private repos. It’s not going to be much at first, because I tend to collaborate with others when writing code, or I write it for companies that would never consider open sourcing their code. However, I’m going to start now and I’m going to try to talk collaborators into letting me open source stuff that I write in the future.

Open sourcing some of this code might encourage me to make it more usable.

I’m starting with some code that I use to tinker with neural networks: dc-ann

It’s disorganized right now, but, in time, I’ll put up some examples of how to use it, convert it to a quicklisp project, and make it easier to use. Furthermore, the code, as released, leaves out the transfer-function and weight-initialization changes that make the neural network a deep-learning neural network. I’ll get those in as soon as I clean up that code in the private repo.

Here are some other bits that I’m tossing into the open source community now:
dc-affinity (Lisp code to demonstrate the effects of measuring Euclidean Distance between trigram vectors)
Utyls (Perl utilities that I use frequently)

More bits that I plan to release soon include a ton of Emacs Lisp code and more Perl utilities.