We do have a small collection of audio books; the Baker Library at the Business School has the best collection. (A HarvardKey will also allow you to stream audio books from the Naxos Spoken Word Library.)
If you are looking for a particular title, search the Hollis Catalog and limit the results to format "sound recordings." To find many audio books to browse, search the Hollis Catalog for "recorded books," then limit to format "sound recordings" or form/genre "audiobooks." If you wish, you can also limit by genre/form "fiction" or by the holding library.
The best way to get a comprehensive list of all audio books cataloged in the Harvard libraries is through the "Command Search" screen in Hollis Classic. Type "wxa=I" to get all audiobooks at Harvard. If you want to retrieve records from specific libraries, you can search "wxa=I and wsl=wid" for Widener, wsl=bak for Baker, wsl=lam for Lamont.
The Cambridge Public Library has an excellent collection of audio books, and you can find audio books to download through the Digital Media Catalog. LibriVox provides free audiobooks from the public domain.