A Deep Learning Rig That I Want To Build

Here are the specifications for a deep learning rig that I want to build.  I want to call it Mindrigger, and I want to make it available as a service to myself (and others) so that we can experiment with neural-network frameworks such as Torch, Tensorflow, Keras, Theano, and so on.


I already have most of these parts.  Most notably, the motherboard and CPU are missing.  I don’t have the internal or external storage devices, but I have 1.5 TB of internal SSD storage that I can use in the mean time.

If you’re interested in contributing any parts or providing some other kind of assistance, I’ll give you an account when the system is built and goes online.

Training Neural Networks To Help With Healing People

On Saturday, June 9, I’m hosting a Meetup on Neural Networks at my home.  (See the Contact section for the address.)

Last month, 150 do-gooders convened at the the Silicon Valley Artificial Intelligence Genomics Hack-a-thon to take on cancer.  Sebastian, who runs the Meetup that I’m hosting, and a member of the team that took the first place at the Hack-a-thon, is giving the talk.

I’m especially interested in learning what public data exists that can be used for machine learning and some ways in which neural networks can be trained to help with healing people.

If you want more information or if you’re interested in attending, please visit the Meetup event page.

Documenting Common Lisp Projects for GitHub

I want to be able to automatically create the README.md for GitHub whenever I create a new Common Lisp project.  I was impressed by the number of systems that exist for documenting a Common Lisp package.  However, I couldn’t find anything that was simple and that created Markdown output, like the README.md that GitHub uses.

I was in a hurry, so after a few minutes of Googling, I decided to create my own function for documenting packages for GitHub.  It worked.  Sort of.  It documents function only, not macros or other things.  Here’s the function:

(defun document-package (package output-filename)
  "Documents the Common Lisp package PACKAGE and writes that documentation to the file given by OUTPUT-FILENAME."
  (loop for function being the external-symbols of (find-package package)
     when (and (fboundp function) (documentation (symbol-function function) t))
       (list :function function
             :function-name (string-downcase function)
             :documentation (documentation (symbol-function function) t))
     into functions
       (return (loop for function in 
                    (sort functions #'string<
                          :key (lambda (x) (getf x :function-name)))
                  collect (format nil "## ~a ~a~%~a"
                                  (string-downcase (getf function :function-name))
                                   (format nil "~s"
                                            (symbol-function (getf function :function))))
                                   '(("\\s\\s+" " ")
                                     ("DC-UTILITIES::" "")))
                                  (getf function :documentation))
                  into function-docs
                    finally (spew (format nil "# ~a~%~{~a~^~%~%~}" package function-docs)

That function, when called like this: (document-package :dc-utilities “~/README.md”) produces Markdown that looks like this:

## alist-values (ALIST &REST KEYS)
Returns the values associated with KEYS in ALIST.  ALIST is an associative list.

## bytes-to-uint (BYTE-LIST)
Converts the list of bytes BYTE-LIST into an unsigned integer.

## change-per-second (FUNCTION-OR-SYMBOL &OPTIONAL (SECONDS 1))
Given the function FUNCTION-OR-SYMBOL, who's return value changes over time, or a variable who's value changes over time, with the change being unidirectional, this function computes the rate of change by calling the function, sleeping, then calling the function again, then computing the rate of change per second.  You can optionally specify the number of seconds to wait between calls.  If FUNCTION-OR-SYMBOL is a variable, then this function retrieves the value of the variable, sleeps, then retrieves the value of the variable again.

## create-directory (DIR &KEY WITH-PARENTS)
Works just like the mkdir shell command.  DIR is the directory you want to create. Use WITH-PARENTS if you want the function to create parent directories as necessary.

## cull-named-params (NAMED-PARAMS CULL-KEYS)
Given a value for NAMED-PARAMS like this one

    '(:one 1 :two 2 :three 3)

and a list of CULL-KEYS like this one

    '(:one :two)

this function returns a list of named parameters that excludes the names (and their values) that match the names in CULL-KEYS.  In the above example, the result is

    '(:three 3)

## directory-exists (PATH)
Returns a boolean value indicating if the directory specified by PATH exists.


Which looks like this on GitHub:

Is there a package out there already that can do this better?  If so, please leave me a comment.  If not, I need to find out how to get the documentation for a macro, for a method, for a special variable, for a class, and so on.  Any help would be greatly appreciated.

Other things I want this function to do:

  • Check the .asd file to see if there’s a license bit: :license "MIT License"If there is, the code should check to see if there’s a LICENSE file in the project.  If there isn’t, the code should determine if the name of the license is known.  If the code is able to get the text for the license, it should include it at the beginning of the documentation.
  • Optionally produce a table of contents.
  • Output other formats (aside from Markdown) as well, like Text or Emacs Org-Mode.

For now, the document-package function is just a function in the dc-utilities package.  However, once I flesh the function out a little, I will probably move it to its own repo.  If you get ahead of me with any of this, please send me a note.  I could really use this functionality.

Predicting a User’s Movie Preferences

One thing that online video distributors (such as Netflix) don’t do yet (and that they will most certainly do in the future–some of the tech is already in place) is to take into account the content of the movies, such that they can learn a user’s likes in a way that is independent from (and complimentary to) the likes of other users with similar preferences.

It’s been possible for while, for example, to automatically identify “types” of documents and to tag or cluster those documents with a high degree of accuracy. Early last year, using a euclidean-distance algorithm that worked on trigram-vector representations of text, I created an example of just such a thing, document affinity, using Wikipedia articles. In the example, you could choose a Wikipedia article and the system would list the other Wikipedia articles in order of similarity to the one you chose. Despite the fact that this method is rather well known, the results of that experiment were nothing short of astonishing, and provided a quality of links among Wikipedia articles that exceeds anything that exists today, much better even than the human-made links that are in the articles. Moreover, consider that the bag-of-trigrams method I just described is archaic by comparison to some of the algorithms that research groups are exploring today, like recurrent neural networks using word vectors (my favorite).

Imagine if, in addition to resorting to the standard preference algorithms, an online video distributor (such as Eros Now or Netflix) were to analyze the script of the movie (subtitles, for example), identify similar movie scripts, and account for that when choosing other movies that you might like. The distributor’s predictions would improve in a dramatic way.

The technology to analyze the text of the movies exists already. But, if you look into the future, you’ll find that soon computers will be able to analyze the visual part of the movie, the frames themselves, the motion, and the sounds. Computers will be able to combine that type of analysis with script analysis for a full-content analysis that will increase the preference predictions even more markedly. Google and others have already started to do some work in related areas, largely based on the contributions to deep learning that Geoffrey Hinton and his students have made in the last decade, including rectified linear units, dropout, ways of combining convolutional neural-network layers with fully-connected layers for higher abstraction, and word-vectors and phrase-vector training.

We’re going to see far better preference predictions in the near future, probably by smaller companies in the beginning. In the long run, this kind of prediction, and the more advanced ones that I described above, which encroach into the realm of human cognitive abilities, will become widespread.

My Wishlist for the Future

I’m am waiting for the following developments and contributing in any way that I can to their fruition:


  • All languages to converge toward a Lisp-like language
  • Trillion-fold increase in computer processing power
  • Superhuman-level artificial general intelligence
  • Human mind uploads
  • Indefinite human lifespans


  • Dissolution of all international borders
  • Fully distributed, internet-like, free-market, and non-invasive economy, with multiple distributed-ledger based currencies
  • Consolidation of all taxes, including income tax, sales tax, gas tax, local taxes, and property taxes into a single, monthly bill, at the local level
  • Completely open government process that collects and incorporates feedback from citizens, uses revision control, and uses open-source software to automate all aspects of government
  • Replacement of all government-issued business licenses (including medical licenses, legal licenses, engineering licenses, education licenses, and driving licenses) with commercial rating systems as well as legal and enforcement systems that can encourage honesty and deter wrongdoers
  • 1 year sunset on all regulations

I fully expect all of these developments to happen in my lifetime. They will bring tremendous wealth, health, equality, meaning, and worthiness to all of humanity.

Cleaning House by Open Sourcing

I’m going to start open sourcing some of the code that I currently have in private repos. It’s not going to be much at first, because I tend to collaborate with others when writing code, or I write it for companies that would never consider open sourcing their code. However, I’m going to start now and I’m going to try to talk collaborators into letting me open source stuff that I write in the future.

Open sourcing some of this code might encourage me to make it more usable.

I’m starting with some code that I use to tinker with neural networks: dc-ann

It’s disorganized right now, but, in time, I’ll put up some examples of how to use it, convert it to a quicklisp project, and make it easier to use. Furthermore, the code, as released, leaves out the transfer-function and weight-initialization changes that make the neural network a deep-learning neural network. I’ll get those in as soon as I clean up that code in the private repo.

Here are some other bits that I’m tossing into the open source community now:
dc-affinity (Lisp code to demonstrate the effects of measuring Euclidean Distance between trigram vectors)
Utyls (Perl utilities that I use frequently)

More bits that I plan to release soon include a ton of Emacs Lisp code and more Perl utilities.

API for a Mobile App in Common Lisp

Not too long ago, I released a mobile app called Lil’ Baby Names, for iOS and Android. I had a few friends help me. Mark Edelsberg designed the UI, Liu Xinjun built the Android front end, and Pritesh Shah built the iOS front end. I built the API that the devices use. What’s interesting about the API is that it’s written in Common Lisp. Why? Most of all because I love Lisp code. To my eye, the same algorithm written in Lisp or any other language always looks prettier in Lisp. But, I’ll get to that in another post. Here are some other reasons why I chose Lisp:

  • The code compilation process is fast and transparent
  • The code compiles directly into machine code
  • I can recompile functions in a running program, so I don’t have to restart the production server if I change any code
  • The REPL allows me to test pieces of my code as I go, providing continuous feedback on the work that I’m doing
  • Lisp macros and other lisp features make the code concise and easy to follow
  • The resulting service is really fast

For example, the Lil’ Baby Names API has a single endpoint and here’s how I define that endpoint:

  (get "/api/names" api-names))

Here’s another example. This code reads a small map of genders to gender IDs from the database:

(flatten (db-cmd *db* :query
                 (:select 'gender 'id :from 'genders)))

Note how I write my SQL in Common Lisp. That’s possible thanks to Lisp macros. This is not just beautiful, but also convenient: now, my editor can properly highlight, indent, and detect syntax errors in my SQL code.

As a final example, take a look at code that formats the results into JSON or plain text, depending on the value of the format variable:

(case format
    (ds `(:map :time ,(format nil "~ss" (read-time event))
               :total-matches ,results-length
               :page ,page
               :page-size ,page-size
               :regex ,regex
               :starts-with ,starts-with
               :ends-with ,ends-with
               :contains-string ,(get-parameter "contains-string")
               :contains-letters ,(get-parameter "contains-letters")
               :min-length ,min-length
               :max-length ,max-length
               :format ,format
               :sort ,sort
               :results ,(ds-list sorted-paged-results)))))
   (format nil "~{~a~^~%~}"
           (mapcar (λ (x) (gethash :name x))

Note the use of the Common Lisp format function, and how concise it can make the code. Also, notice how easy it is to transform a Lisp object or nested list into JSON.

Sure, there are some negative aspects of the language. For one, Common Lisp is a big language with a number of fairly powerful functions and macros (think format and loop). It can take a little time to get the hang of those. Furthermore, the best Common Lisp editor is probably Emacs, and who wants to learn that? However, for those brave programmers who choose to overcome those obstacles, Common Lisp provides a real glimpse into the future of programming.

Valentines for Nina and Lola

Nina and Lola,

Whenever I’m away from you and I spend a lot of time writing computer code, I think about the pretty patterns that your names make in binary code. Your names, Nina and Lola, are wonderful in so many ways that I could never describe them all. Here’s one of those ways.

If you use the following Common Lisp expression, you can find the graphical binary representation of your names in binary code.

(defun binary-name (name &optional (zero #\Space) (one #\O))
    (map 'string (lambda (c) (case c (#\1 one) (#\0 zero) (t c)))
         (format nil "~%~{~b~%~}" (map 'list #'char-code name))))

After defining a function like that, you can call the function with your names, like this, for example:

(binary-name "Nina")

Which returns the following pattern:

    O  OOO 
    OO O  O
    OO OOO 
    OO    O

That pattern represents your name! The letters are in rows. So, ‘N’ is ‘O OOO’ and ‘i’ is ‘OO O O’. Here’s the same pattern with letters in front of each row:

    N => O  OOO 
    i => OO O  O
    n => OO OOO 
    a => OO    O

One interesting thing is that, in your name at least, the capital letters start with ‘O ‘ and the lower-case letters start with ‘OO’.

Lola, here’s the pattern for your name:

    O  OO  
    OO OO  
    OO    O

Here’s the pattern for ‘Daddy’:

    O   O  
    OO    O
    OO  O  
    OO  O  
    OOOO  O

Notice how the pattern for ‘Daddy’ is bigger than the patterns for ‘Lola’ and ‘Nina’? That’s just because ‘Daddy’ has more letters.

Here’s the pattern for ‘loves’:

    OO OO  
    OOO OO 
    OO  O O
    OOO  OO

Here’s the pattern for ‘&’:

    O  OO

So, what do you think this means:

    O   O      OO OO   
    OO    O    OO OOOO     O  OOO               O  OO  
    OO  O      OOO OO      OO O  O              OO OOOO
    OO  O      OO  O O     OO OOO               OO OO  
    OOOO  O    OOO  OO     OO    O    O  OO     OO    O

Happy Valentines to you both! I love you more than anything else in the whole world and I’ll always be there for you.