I’ve recently been quite interested in Graph languages, not for Graph Databases, but for representing information, in code that is essentially “things” and “relationships”. For example, one perspective on Organizations is that they are simply representable as people, systems and the information that flows between them. Those three building blocks are, in theory, enough to describe anything from a small business to a government, if you have an appropriate way to describe all the information, ensure the connections make sense, and report on it. If you further layer on the idea of systems of systems, you have the ability to describe different levels of government, business units, departments and so on. Public and private flows of information allow models that accurately represent the fact that no organization is fully meshed: the systems only expose certain information. The reporting on it is quite interesting, and led to be becoming interested in DOT as a way to describe reports resulting from queries. I didn’t find a small, embeddable Java library that enabled me to read and write DOT files, so I ended up writing DOT4J. The writing of DOT files, is trivial. The reading however, ended up requiring a lexer-parser since DOT is, essentially a programming language of its own. The grammar came from ANTLR, of course. DOT4J is here. The Github page has the Maven co-ordinates for published binaries.

I recently had reason to get to know OpenAPI at work, so I decided to become familiar with it. I have a HomeSeer home controller at home, which exposes a JSON API, so I decided to write a Java OpenAPI server to expose the JSON API over OpenAPI. The net result is hsOpenAPI. It exposes most of the HomeSeer API and uses hsClient, the Java wrapper I had previously built for the HomeSeer API.

I recently got a call from a friend who has a library of thousands of PDF documents. He wanted to know there was a such thing as a personal search engine that could index his library of documents, and allow full text search inside of them. I have a giant document library too, including PDF documents, Office documents, source code, email and so on. I decided I needed one too, and it should be based on Elastic Search. The net result of the weekends work is “kSearch”, a personal search engine. kSearch include a threaded file system indexer, automatic maintainence of the search index as the file system changes, and a web search UI. You can see screen shots and source code on the github project.

A couple projects I’ve been working on have a need for a REPL style console.  I had written a simple framework for creating REPL’s, which is open-sourced here.  

I recently had a reason to need a TELNET server and couldn’t find a Java library that worked for me.  After a weekend of hacking, I now have kTelnet, an Open-source, BSD-licensed, TELNET server.  

I’ve recently become very interested in blockchain, and that, naturally led me to Ethereum and Bitcoin.  From there, I got a little interested in online trading, but since I prefer not to have to think about trading, I started thinking about a trading bot.  Part of the work I needed to do was write a Java API for Qaudrigacx.  The API is here and is free to use under the GPL v3 terms.  

I was recently asked to release some of my code to Maven Central, and therefore had to figure out how to do it.  I’ve now released these khubla.com libraries: cBean Pragmatach antlr4test-maven-plugin OLMReader simpleIOC ParadoxReader The maven coordinates for each are documented on the github pages.  

I recently had a reason to parse a large data set, for another project.  I decided that an ideal “large data set” would be my Outlook mail saved archives.  Sadly, Outlook for Mac doesn’t output PST files, it outputs OLM archives, which are, essentially, giant zip files full of XML.  I was coding this all in Java, so I needed a Java library to parse OLM files. The resulting source code is here.  Schema for OLM XML is here.    

I’ve been interested in RIAK for a while, and ORM’s are nothing short of fascinating. I decided to try writing an ORM for Riak, and the results are here: https://github.com/teverett/cbean My ORM is not an ORM of course, because RIAK is not relational. However it is ORM-like; I can store POJO’s and retrieve them. The features I wanted for my ORM were those that I am accustomed to with Hibernate, or eBean. Ability to store POJO’s and retrieve them Ability to store object trees of POJO’s which contain POJO’s Support for Lists of POJO’s composed inside a POJO Lazy Loading In the end, I ended up with a ORM-like layer that can store data into any Key-Value store. There is a plugin which supports RIAK, and there is an emulated Key-Value store built on a filesystem, which is useful for development purposes. Theoretically cBean could work on any Key-Value store, such as MongoDB, but I haven’t built that support yet. Adding support for a Key-Value store is as simple as implementing the the interface KVService. Supported Types All java simple types (int, String, long, etc) All java wrapper types (Integer, Long, etc) Contained POJOs java.util.List of POJOs java.util.UUID java.util.Date Annotations Similar to Hibernate or eBean, cBean POJO’s must be annotated. I didn’t chose to use the JPA annotations, but instead defined my own. They are here.  There numerous cBean annotated POJO’s in the tests, here. @Entity Similar to JPA, the @Entity annotation simply marks the POJO as one which is of interest to cBean. @Id Every POJO must have an id field and it must be a String or UUID.  The @Id field is a little different in cBean than in JPA.  Inserting a new object with the same JPA @Id as one that already exists in the RDBMS is an error.  In cBean, you simply overwrite the existing object. @Property The @Property is used to indicate that a specific property (simple type, wrapper type, object type or list type) will be persisted.  POJO fields without the @Property annotation are ignored.  There are a number of properties which are valid on an @Property annotation. cascadeSave cascadeDelete cascadeLoad ignore nullable cascadeSave, cascadeDelete, and cascadeDelete are only relevant for POJO fields which are themselves POJOs, or lists of POJOs.  Setting “cascadeLoad=false”, naturally, indicates that the field is lazy-loaded. @Version An POJO can define an Integer field and annotate it with @Version.  cBean will increment the annotated field by 1 each time the POJO is saved. Referential Integrity Frameworks like Hibernate or eBean provide referential integrity because RDBMS’s provide referential integrity.  Key-Value stores, such as RIAK do not provide referential integrity, and therefore neither does cBean.  Therefore is it entirely possible to persist a POJO which contains a POJO, and have the contained POJO be deleted underneath the parent.  In a RDBMS this can be prevented with a foreign key; there is no such protection in cBean.  Therefore application code using cBean must be aware that POJOs in lists, for example, may not be resolvable when the list is reloaded. The strategy that cBean uses for handling broken “foreign keys” is two-fold: If a POJO contains a child POJO and the child is deleted, that Object will be set to null on reload If a POJO contains a list of POJO’s and one of the elements is deleted, the element Object will be set to null.  The List size() will remain the same as when it was persisted. Example Code There is a working example at https://github.com/teverett/cbean/tree/master/example.

I’ve tried a couple different mp3 taggers to tag my mp3 library, however, most seem to have trouble with large mp3 libraries.  So, after doing some reading about AcoustID and MusicBrainz I decided to quickly code up my own tagger, MusicBrainzTagger. MusicBrainzTagger is a command-line application which recurses a directory of mp3 files and tags each one, one by one.  This approach allows it to handle very large libraries; it only processes one file at a time.  File processing consists of reading any ID3 tags in the input mp3, and then calculating the Acoustic ID fingerprint.  The fingerprint is then resolved to a MusicBrainz ID which is used to look up the recording. MusicBrainzTagger then tags the file, renames it, and moves it to a new directory.