Dezi search platform

This week I announced the initial release of Dezi, a new search platform based on Swish3, Apache Lucy, OpenSearch and Plack.

As of about 15 minutes ago, there are now PHP and Perl clients available.

File under projects/swish Sat Oct 1 21:47:46 CT 2011

REST resources

Read through the comments for some good further reading.

File under perl/ Fri Sep 23 09:58:18 CT 2011

libswish3 1.0.0 released

I am happy to announce the 1.0.0 release of libswish3:

http://swish-e.org/swish3/libswish3-1.0.0.tar.gz

libswish3 is at the core of multiple Swish3 implementations, and has reached a stable enough API that a 1.0.0 release seems appropriate.

From the README:



There are currently four different implementations available of Swish3.

• swish_xapian (C++ using libxapian, included in libswish3 distribution)
• SWISH::Prog::Xapian (Perl using Search::Xapian)
• SWISH::Prog::Lucy (Perl using Apache Lucy)
• SWISH::Prog::KSx (Perl using KinoSearch)

All the Perl implementations are available from CPAN. They each rely on SWISH::3 (the Perl bindings to libswish3) and the core SWISH::Prog project, a Perl rewrite of the swish-e 2.x C binary and accompanying helper scripts. The SWISH::Prog distribution includes a 'swish3' command line interface with options very similar to the swish-e 2.x command line tool.

Xapian, KinoSearch and Apache Lucy all offer robust UTF-8 and incremental indexing support, as well as the ability to scale to many millions of documents across multiple servers.

You can read more about Swish3 at the devel site.

UPDATE: Mailing list announcement here.

File under projects/swish Wed Sep 21 22:03:59 CT 2011

Montessori and Open Source Culture

My kids go to a Montessori school. As parents we have been very impressed by the philosophy of learning there, and our kids have thrived.

It should come as no surprise to me then that there is a strong link between the Open Source community and Montessori.

File under general/ Fri Aug 19 09:40:23 CT 2011

FabFi: wifi build out of trash

This is very cool.

File under general/ Sun Jul 17 20:31:34 CT 2011

Al Franken and Ari Discuss Fishing

File under general/ Wed Jun 1 00:00:00 CT 2011

I-94 Corridor and the East Side

I live on the east side of St Paul. These big changes to the I-94 corridor would happen within a mile of where I live.

File under general/ Sat May 28 22:39:50 CT 2011

Search::OpenSearch::Server with REST API

Just uploaded several modules to CPAN that together implement a full REST API for KinoSearch indexes, using Search::OpenSearch::Server::Plack.



The modules are:

Search::OpenSearch 0.11

Search::OpenSearch::Server 0.05

Search::OpenSearch::Engine::KSx 0.08

SWISH::Prog::KSx 0.17

SWSIH::Prog 0.49

File under projects/swish Thu May 26 13:56:43 CT 2011

Rubber Soul

I've loved the Beatles since the summer of 1983, when I was 11 and my aunt introduced me to their records (including some rarity vinyl iirc). The first record I ever bought was the blue greatest hits double record. I played the first side of the first record (Strawberry Fields, Penny Lane, Sergent Peppers, Day in the Life) over and over in 6th and 7th grade.

But it's taken me nearly 30 years to identify my favorite record. I've had lots of favorite songs, but Rubber Soul is now officially my favorite record. I realize no one but me cares, but it feels important somehow to have identified an actual studio record, rather than a greatest hits compilation, as my favorite.

File under music/ Sun Mar 20 19:53:57 CT 2011

Mounds Park

From my neighborhood email list:

File under general/ Fri Feb 4 16:38:47 CT 2011

Bug

My laptop started making a Very Bad Noise on Tuesday morning, the kind of whirring creak I usually associate with the death throes of a hard drive. It didn't sound exactly like a hard drive, which often clicks or knocks, and the whir was rather slower paced than the high RPMs of a hard drive. But as I didn't think this Macbook Pro had a fan, since I had never heard one, I just assumed the only moving part was the hard drive and so I ordered a new one online using my wife's computer.

When the drive arrived today, I opened the laptop to swap out the hard drive, and lo! there was a box elder bug caught in one of the two small fans on the logic board. Yes, my computer had a bug in it. I pulled out the lifeless, hard little black and orange-striped carcass, and put the cover back on. Started without problem or Very Bad Noise.

I had thought that the term "bug" used to describe a computer glitch was coined after someone found a moth in an early computer. But according to the all-knowing Wikipedia I was wrong. Nonetheless, I was relieved to discover this bug and to fix the problem so easily.

And now I have a spare drive for that time when my hard drive really does die.

File under general/ Thu Jan 27 20:16:25 CT 2011

False Laziness

One of the three virtues of programming is Laziness. Beware of false laziness. Andy Lester writes on the problem aptly when he describes an interaction with another programmer:

File under projects/ Thu Dec 23 03:40:32 CT 2010

Funny depending on whoami

Only funny if you're a programmer.

File under projects/ Mon Dec 20 21:25:40 CT 2010

The Interruptible Programmer

A brilliant and humane essay on changing work habits.

Excuse me while I get up and stretch.

I can vouch for the writer's experience, though for me it has been less about back pain (though I have that too) than eye strain (going on 7 years now). Biggest of all though has been having children and working from home: that is the interruption formula in a nutshell.

File under projects/ Fri Oct 15 10:43:06 CT 2010

CPAN test failures

SWISH::3 0.08_04 is passing all tests all over the CPAN testers universe, so that is encouraging.

However, some reports (notably on FreeBSD) report false failures because of a Wstat issue.

I've posted about it at PerlMonks and hope someone out there has an easy fix.

Update: finally found a fix for this. The problem is that Perl has its own my_setenv() function that interferes with the native setenv() called by libswish3.c. The fix was to set the magic Perl var PL_use_safe_putenv as shown here. This took many hours and googling to track down. Glad to be done with it (I hope!).

File under projects/swish Mon Oct 11 00:37:31 CT 2010

Ingrid Sequoia

Our new daughter arrived yesterday. We are all home, doing well.

Update: her middle name is inspired by this Wendell Berry poem.

File under general Thu Aug 5 10:38:41 CT 2010

Seduction and Betrayal

This is a really good op-ed. Mayhill Fowler gets just the right amount of "philosophical and epistemological" in her reflections on the recent McChrystal blow-up.

The storyteller changes reality, because the story changes our memory, personal and collective. It's always that way. It's the great lesson of the Deconstructionist school. All narrators are untrustworthy -- or rather, trust is not the same as fact. And faith is an act of trust, not fact.

File under ideas/ Fri Jun 25 09:12:59 CT 2010

Open Source Business

opensource.com has an interesting thread on the open source business model(s).

File under general/ Wed Jun 9 08:35:33 CT 2010

Swish3 progress report

There's been a ton of work on Swish3 in the last year. I've actually started planning a 1.0 release, after 5 years of work.

Lately I've been focusing on three things: (1) making the Perl bindings easier to install; (2) indexing of compressed documents; and (3) supporting XInclude of document fragments. The first is accomplished: you can install the entire library via CPAN. The last two are aimed at large doc sets where I want to keep the XML compressed on disk for space reasons, and where I want to re-use subsets of the document collections in building multiple indexes.

File under projects/swish Tue Jun 8 23:33:18 CT 2010

Searching for Jesus in the Gospels

Adam Gopnik writes a cogent and articulate essay about current biblical scholarship and the life of passersby in last week's issue.

File under new yorker/ Tue May 25 20:42:33 CT 2010