Yesterday I switched from using planner-mode to using Org-Mode, both emacs note-tasking and todo-task management apps. I’ve been using planner-mode for the last year and a half, and rely on it pretty extensively to track upcoming todo items, in both personal and work contexts.

I’ve read in the last couple of months, however, that Org-Mode is already in emacs-cvs, and will be part of the standard distribution of emacs-22. I decided to try it out, was quickly impressed, and transitioned over.

Both support the same basic data model: files of mixed notes and todo-items, where the items are simply specially-formatted lines in the file. Items have an optional due date and an optional priority (ABC). For example, in planner-mode, they appear:

A _ Write up planner-/org-mode post (2006.11.24)

B X Make faux-chicken dinner for Thanksgiving (2006.11.23)

For Org-mode:

** TODO [#A] Write up planner-/org-mode post SCHEDULED: <2006-11-24> ** DONE Make faux-chicken dinner for Thanksgiving SCHEDULED: <2006-11-23> CLOSED: [2006-11-23 Thu]

Both support the same basic execution flow: when requested, the package looks through the special files to find lines that fit the pattern, and constructs another file (planner) or a transient view (Org) over past-due and presently-due items. There are operations for creating items, scheduling them on a particular date, transitioning them closed, jumping from the item to associated notes, &c. As well, both support some form of unidirectional publishing of the content to HTML, ostensibly to serve as some public project-planning status.

Org-mode, I’ve discovered, goes far beyond planner-mode in terms of features.

• Core
• It is based on outliner-mode, and its documents are inherently foldable, hierarchical documents.

• It uses file extension (“.org”) for mode-selection, so planning document can live next-to or within projects.

• Todo

• There’s a bit more latitude where items can be placed in the notes documents.
• It supports — separately from the scheduled to-do time for a task — a deadline, which is brought up differently in the “now” view.
• Overdue items are indicated more clearly.
• Items can be scheduled for a range of time.
• Items can be tagged arbitrarily and searched/viewed by tags (“WORK”, “STORE”, “@LAPTOP”, …)
• Supports progress logging

• Supports hierarchical sub-tasks (yay!)

• Linking

• Explicit linking rather than CamelCase (though CamelCase is an option), which was the major problem I was having with planner-mode (it starts to become really slow for large documents).

• Patterned links (“bugs:1234″, “http://bugs/show_bug.cgi?id=%s” → “[...]?id=1234″) are supported.

• Other

• Has a table/spreadsheet editor. The table-editing part appears similar to table.el, but the spreadsheet functionality is just pure awesome.

Update Mon 2006-11-27: after being contacted by Carten Dominik, the author of org-mode, I’ve revised the comment at the end about the table/spreadsheet editor.

Were you the Telepathy project, what would you name your Oscar protocol handler subproject? Why, Wilde, of course. Yuck yuck yuck.

cacti_db.py is a script which will clone an existing data-template and graph-template for multiple (host,rrd_file,template) instances.

In the scenario it was created for, data is (externally) collected into a (directory) tree of RRD files. While there are only a handful of unique RRD file and graph types, there are a large number of instances of those templates. Good examples of this are:

• cpu load
• 1,5,15 minute averages
• 1 per machine
• disk util
• avail, used, free
• ~5 per machine
• cache stats
• (hit, miss, size) stats
• 2 .. 20 caches/”node”, depending type.

This collected data might exist in the following tree of RRD files:

/servers/hostA/cpu.rrd /servers/hostA/disk/root.rrd /servers/hostA/disk/usr.rrd /servers/hostA/disk/data/logs.rrd /node/nodeA/cache/foo.rrd /node/nodeA/cache/bar.rrd [...15 more caches...]

Cacti has the following templates manually configured:

/servers//cpu /servers//disk /node/*/_cache

The script, then, knows how to relate RRD files of the regex pattern r'''/servers/([^/]+)/cpu''' to the template(s) “/servers/*/cpu”, using the regexp group 1 as the name of the host; edit the ‘rrd_types‘ global to suit your scenario.

Instances are named as their RRD file name (yes, skipping Cacti’s own |template_formatting|).

The script will delete instances with the same name, allowing you to re-run it pretty liberally to pick up new rrd files, changes in templates, &c.

The process of cloning is as follows; see ‘execute_plan_alpha(...)‘ in the script for the gory details.

• find the host in table ‘host‘.
• data template instantiation
• find the data template id (by name) in ‘data_template‘.
• if an instance with the name already exists: delete it.
• insert into data_local, get new (autoinc) id for the instance.
• get the list of rrd DSes from the template (from ‘data_template_rrd‘)
• copy each, maintaining the instance -> template id map.
• instantiate the data_template itself (‘data_template_data‘)
• copy the RRAs as well (‘data_template_data_rra‘)
• graph instantiation
• get the graph_template_id (by name) from ‘graph_templates‘
• insert into ‘graph_local‘, get autoinc id.
• instantiate graph (‘graph_template‘).
• copy ‘graph_instance_item‘s, as this is where the graph-ds/rrd-ds mapping is stored, run the relevant ids through the map retained earlier.

For details on Cacti 0.8′s rather atrocious database schema, see Cacti Forums: Database schema… WTF? and Cacti Forums: Cacti data relationship diagram.

Ever since buying our home, we have been eager to replace both the (hideous) kitchen and dining room floors. While the dining room floor isn’t nearly as offensive, a couple of the tiles had already come up, and we were pretty lazy in covering it when we re-painted the room, so it was easily a candidate. As well, given that we’ve never installed a floor, and the dining room floor – unlike the kitchen — is basically square, we decided to start with that room to get an idea of the process.

We found a very good guide to installing a hardwood floor, which was quite useful. We certainly bought enough wonderful bamboo wood beforehand, but the tools were generally purchased and rented in the Friday before the weekend. In short, the following were used:

• pneumatic floor nailer (2: the first one jammed/failed just after the store it was rented from closed for the remainder of the weekend; Home Depot was still open, however)
• air compressor
• rubber mallet
• pry-bar
• miter box + hand-saw
• measuring tape, square
• normal hammer
• paint scraper
• finishing nailer

We started pulling up the existing floor on Thursday night with the two tiles that had already come up in the last year … trying to understand how hard that would be. Luckily, the original floor installers had used very little glue. The tiles came up without much effort, and left nothing behind. The sub-floor was in great shape, ready to accept a new floor.

The installer in the aforementioned guide did some nifty drilling to create well-countersunk nail wells and covering plugs for the initial row of flooring. We, uh, didn’t do this. After carefully aligning the first row, we tried in vain to nail some finishing nails straight through, but bamboo is 106% harder than Red Oak, the manufacturer proudly proclaims. At this point, for some reason, we also didn’t have the finishing nailer. So, we used the floor nailer directly applied to the surface, creating some unsightly blisters on the first row. However, that row was now solidly attached to the subfloor. We looked at the remainder of the unfinished floor, and put our first folly behind us.

The remainder of the rows slotted nicely into place. As mentioned, this room is simple, and thus has only 3 widths. As the planks were tounge-in-groove and end-matched, there was some thought-work required to make sure we had the appropriate cuts to create “in-nie” and “out-ie” pieces for the left and right sides as we alternated seams. There was a section where my measurement was a bit off, so we had 2 rows that ended in a 1 inch gap from the wall. We agreed that wide trim would hide a multitude of sins, and pressed on.

We were also lucky in that we only had to cut one (1) notched piece, around one of the heater vent covers. All the other transitions were “perfectly” aligned. There were multiple “will you look at that!” and “my word”. It was good.

Of course, the last row didn’t work quite so nicely, and needed to be “ripped” … as I now understand, a word meaning “to cut a thin piece of wood the long-way”. I started into the short segment with the hand-saw in the basement, but the bamboo – 106% stronger than Red Oak – did not like this idea. As well, just given the geometry of the saw and cutting through a long piece of wood, a saw toothed pattern emerged. We resolved that trim would be our friend, and pressed on. We did, however, call Home Depot to find out that they do in fact cut wood that you do not buy there, as the long segment was forthcoming.

On the 2nd Home Depot trip of that day (after getting the finishing nailer), I found out that they cannot rip pieces unless it’s plywood wider than 12 inches. Dejected, we returned home, having no other option but to saw our arms off.

After returning, and even after starting to saw, we looked for options. We settled on perforating the board with a series of closely-spaced drilled holes, which would reduce the volume of wood by a good fraction … at least half. This actually worked pretty well; at times, the hand saw cut through inches of board as like butter. With the last piece cut, and luckily fitting, we ended the second night with an un-trimmed but finished floor.

When we do the kitchen, we’ll make only a couple of changes: * have the finishing nailer at the beginning * rent a table saw

But, otherwise, we’re looking forward to another nice home improvement project.

And that damn kitchen floor being gone.

After much sporadic work, testing, updates, frustration and hacking, GnuCash 2.0.0 was released last night. Now the real fun can begin. :)

I’ve developed some patches to GATE to allow an application to work around memory issues resulting from using JAPE in a long-lived VM.

The problem stems from use of the GateClassLoader and the dynamic creation of classes in loading JAPE grammars. Generally, classes are loaded into the Permanent Generation section of the heap, and are only eligible for garbage collection when the classloader from which they were sourced is itself unreachable. As the GateClassLoader is never reset or reinitialized, over time the classes will accumulate and fill the heap, and cause OutOfMemory errors in VM. While a first attempt to resolve this would be to cache the loaded JAPE objects, there are still classes created on each use of that grammar, which will slowly cause the same problem.

The caller needs a facility to reset the class loader, as provided by the patches to src/gate/Gate.java (and the compiler classes). It is expected that the caller invokes Gate.resetClassLoader() periodically, either on a fixed frequency or after a set number of uses of the library; our application resets every 15 minutes.

One complication is the difference in time between the generation/loading of the classes for a particular JAPE grammar and the instantiation of objects of those classes. Obviously, the objects needs to be instantiated from a valid class, and in particular from the classloader from which they were created; if the classloader is reset in the middle, there will be instantiation exceptions thrown when using a different classloader. A change was thus made to src/gate/jape/RightHandSide.java to retain a reference to the relevant GateClassLoader. Note, that there’s still a race condition present, but the window of time is significantly less than before.

Our (application-level) reset operation resets both the Gate classloader as well as invalidating our JAPE cache: current threads will finish using their JAPE objects, references to the classloader are eventually lost (from gate.Gate via reset and the jape RightHandSide objects through reference), and the garbage collector is allowed to collect the whole shebang.

This patch is much more of a workaround than a fix for the underlying problems. I don’t care if these patches are applied, so much as they’re useful in demonstrating the problem so it can be more appropriately fixed. Really, though, JAPE shouldn’t need to create classes per use.

• Patches against SVN:
• /2006/06/gate-svn-classloader.patch

• /2006/06/gate-svn-jape-rhs.patch

• Patches against 3.0 (B1846):

• /2006/06/gate-b1846-classloader.patch

• /2006/06/gate-b1846-jape-rhs.patch

• Previous memory issues thread

Also, while I’m at it, here are patches to:

• fix the mangled end of prj.el
• replace the ad-hoc chained exceptions with java’s

Update, 2006-07-10 I was pressed for detail about my claim that JAPE creates classes on use, and found that I couldn’t. Here’s my reply to gate-users:

I went to reproduce this, and found I can’t. Now I believe I mis-spoke.

The classes I remember being created were…

japeactionclasses.postprocessNewLineActionClass[#] japeactionclasses.postprocessVBnegActionClass[#] japeactionclasses.postprocesssimpleJoinActionClass[#]

…which are from the tokenizer’s grammar. Thinking back to the period I was debugging the issue, I don’t believe I was caching the tokenizers, but I had instituted the other JAPE caching. As well, I didn’t remember that the tokenizer was implemented in terms of JAPE. As such, I tricked myself into believing that the simple use of JAPE grammars from a different processing step was causing the loading of the aforementioned classes, when really it was just not caching the tokenizers.

Attached to Gentoo Bug#133629:

-- shell-script --

Copyright 2006 Joshua Sled

Distributed under the terms of the GNU General Public License v2

inherit eutils java-pkg

MY_PV=${PV}final

DESCRIPTION=”SQL client” HOMEPAGE=”http://squirrel-sql.sourceforge.net/” SRCURI=”mirror://sourceforge/${PN}/${PN}-${MYPV}-src.zip”

@fixme: log4j, commons-cli, nano-xml

DEPEND=”>=virtual/jdk-1.4 dev-java/ant-core” LICENSE=”LGPL-2″ SLOT=”0″ KEYWORDS=”x86″ IUSE=”"

S=”${WORKDIR}/sql12″

src_unpack() { unpack ${A} cd ${S} }

srccompile() { cd ${S}/build ant jarsource compile_plugins }

src_install() { dodir /opt/${P} dodir /opt/${P}/doc

cp -R ${WORKDIR}/squirrel-sql-dist/squirrel-sql/core/dist/* ${D}/opt/${P}/ cp ${D}/opt/${P}/squirrel-sql.sh ${T} echo ${T} sed -e “s#%INSTALLPATH#/opt/${P}#” < ${T}/squirrel-sql.sh > ${D}/opt/${P}/squirrel-sql.sh makedesktop_entry squirrel-sql.sh “SQuirreL SQL ${PV}” /opt/${P}/icons/acorn.png “Utility” /opt/${P}/ }

Last night’s VAGUE meeting was dominated by the Key Signing Party, which went pretty smoothly. We were joined by a few fresh faces, which is always nice. This was my first key-signing event, and it was simple and painless; the pizza helped, I’m sure. I’ve already received a couple of signatures from others, and will start sending mine to various UID addresses this evening. Unfortuantely, a couple of people were unable to attend and there was a mis-recorded fingerprint, so a follow-up event might be in the cards during an upcoming meeting. I had intended to present a round-up of [application software that can make use of GPG](http://www.gnupg.org/(en%29/related_software/index.html), but it turns out that GnuPG already has a nice list of such software. So I punted and spent time on GnuCash instead.

After the KSP, we talked briefly about next month’s meeting. It will be on the 3rd Thursday of the month, which is Thu March 16th, and it’s very likely to be in the Mac lab (Waterman 113-T); Jim Lawson will schedule the room. With this meeting we’ll get some external publiclity, primarily by putting an announcement into Seven Days. Sam Hooker has volunteered to put the ad in next week.

At the same time, we didn’t come up with any agenda for the next meeting. We had talked briefly about discussing (the draft of) version 3 of the GPL. It seems like that should be a presentation followed by open discussion, and thus would require someone to develop a (or adapt an existing) presentation on the changes. But, even that is up in the air. So, content TBD…

I primarily do backend processing at work, often dealing with extended processing that doesn’t naturally have a user-interface. As such, I’ve been finding ways to provide quick and convenient interaction with the processes.

One simple obvious way is logging, but I found that text-based statement logging doesn’t get me quite the level of detail and structure I want and ultimately need to be productive. When I was playing with RDF more heavily earlier last year, I started to format my log messages as N3 pseudo-statements:

** 2006-01-25 12:34:56 INFO Thread-0 Processor: [ object "foo"; count 12; disposition "thumbs-up" ] ** 2006-01-25 12:34:56 INFO Thread-0 Processor: [ object "bar"; count 7; disposition "thumbs-down" ]

Of course, the “real” form of this would be to have the entire logging pipeline emit actual N3 statements, including the logging-provided (meta-)data:

[ # a log:Entry log:ts "2006-01-25 12:34:56" ; log:lvl log:INFO ; log:thread "Thread-0" ; log:class "Processor" ; log:msg [ :object "foo"; :count 12; :disposition "thumbs-up" ] ].

N3′s great because of its simple syntax for representing structured data, but I’ve recently started using JSON instead, which has a more straightforward syntax and semantics for loose, non-RDF data representation. If you’re unfamiliar with JSON, it is a subset of javascript representing dictionaries, lists, and the basic data types (integers, floats, booleans and strings). An example would be:

{“favorite int”: 42, “floating value”: 42.01, “this is an example”: true, “a list of things”: [42, 42.01, true, {foo: bar, baz: quuz}]}

For example, I’ve been developing a content parser, and running it against a fixed set of test cases to assess the impact of changes I make. Each test result consists of 4 parts:

1. the test-case label (the input filename)
2. the expected-and-produced pairs
3. the expected-but-missed cases
4. the unexpected-but-produced cases

Each test-case and result is on the order of 10 “things”, so they’re well suited to manual review. The test run outputs a set of log statements like:

** 2006-01-25 12:34:56 INFO Thread-0 Testing: {“case”:”/path/to/test/case/file.name”,”expectedProduced”:[[«expected object», «produced object»], «…»], “expectedMissed”:[ «...» ], “unexpectedProduced”:[ «...» ]} ** 2006-01-25 12:34:57 INFO Thread-0 Testing: {“case”:”/path/to/test/case/file.name”,”expectedProduced”:[[{"foo":"bar"}, {"foo":"bar"}], «…»], “expectedMissed”:[{"foo":"baz"}], “unexpectedProduced”:[{"foo":"quux"]}

The process of logging general objects is made much easier by using a reflection-based utility class I wrote that will json-stringify random objects thrown at it, but that’s for another post…

A nice feature of JSON – being a subset of javascript – is that it is trivially parsed in javascript with an eval statement. As such, it’s simple to write an HTML page that accepts pasted JSON and renders it as nested tables. As this accepts and evaluates arbitrary javascript, I won’t link to it, but instead present the (mostly) short page here:

json:

decoded: