peknet :: an eddy in the bit stream         
about peknet
peknet is Peter E Karman musing on technology, politics, religion, books, beer and parenthood.

navigate

credits

Brighter Planet's 350 Challenge

St Paul Minnesota Yellow Pages

Powered by Swish-e

Valid CSS!

proud member of the
Open Source Community

© 2005 peknet dot com

syndicate this site

iPhone 5

So frustrated with my iPhone 3 (which I bought refurbished 2.5 years ago) I finally broke down and bought new iPhone 5. Wow. Wow wow wow. So this is the Mobile Experience everyone keeps talking about. I can actually read a web page and use the maps app and Do Things. Reminds me of the first Mac G3 I got after using a LC for several years (yes, I am that old).

File under general/ Fri Apr 5 20:24:24 CT 2013

DuckDuckGo dot com

I've been keeping tabs on DuckDuckGo for about three years now. It's a web search site that doesn't track users like Google does.

A nice write-up on the DuckDuckGo architecture today. Nice to see some technologies I recognize.

File under projects/ Mon Jan 28 22:13:16 CT 2013

Two Hard Things in Programming

There are two hard things in computer science: cache invalidation, naming things, and off-by-one errors.


via http://martinfowler.com/bliki/TwoHardThings.html

File under projects/ Tue Jan 8 21:20:34 CT 2013

Poem #1110

The death throes of the mouse under the sink, in the trap I laid, evoke neither sadness nor pity. I turn Nick Drake up a little louder.

File under general/ Sat Nov 10 22:58:09 CT 2012

Dezi 0.2.4 released

Happy to announce release of Dezi 0.2.4 with spelling suggestion feature.

File under projects/swish Wed Oct 17 22:42:28 CT 2012

Charlotte' Web Day

I am reading Charlotte's Web, the classic E. B. White children's novel, to my kids for about the 10th time. I don't know if it is the 10th time or not. After about four readings I lost track. Let's just say it is as familiar to me now as any book I have ever read.

I mention this book today because tomorrow, September 7, is a significant day in the story. It is the day when Wilbur wins his medal, when he cajoles and bargains with Templeton to save Charlotte's egg sac, and he leaves Charlotte alone, at the Fair, where she will die.

The book is 60 years old, the best-selling children's book of all time (according to Wikipedia), and though I have now read it more times than I can keep track of, I do not tire of it. Garth Williams' illustrations still amuse and enlighten. White's prose still sparkles. I still cry at the end, which my children did not understand the first few times but they do now. Malcolm swears off bacon for a few weeks after each reading.

So I hereby declare September 7 to be Charlotte's Web Day.

Here are some quotes I love.

"I was just thinking," said the spider, "that people are very gullible."
"What does 'gullible' mean?"
"Easy to fool," said Charlotte.
"That's a mercy," replied Wilbur, and he lay down in the shade of his fence and went fast asleep.


"Have you heard about the words that appeared in the spider's web?" asked Mrs. Arable nervously.
"Yes," replied the doctor.
"Well, do you understand it?" asked Mrs. Arable.
"Understand what?"
"Do you understand how there could be any writing in a spider's web?"
"Oh, no," said Dr. Dorian. "I don't understand it. But for that matter I don't understand how a spider learned to spin a web in the first place. When the words appeared, everyone said they were a miracle. But nobody pointed out that the web itself is a miracle."


The crickets felt it was their duty to warn everybody that summertime cannot last forever. Even on the most beautiful days in the whole year--the days when summer is changing into fall--the crickets spread the rumor of sadness and change.


"We can't tell what may happen at the Fair Grounds. Somebody's got to go along who knows how to write."
-- Charlotte


As my friend Eric likes to quote:
It is not often that someone comes along who is a true friend and a good writer. Charlotte was both.


Here's to you, Charlotte.

File under books/ Thu Sep 6 20:43:38 CT 2012

Python Dezi Client

Finally got back to the Python Dezi client I started several months ago. All tests passing now so I'm calling this the 0.001000 release.

File under projects/swish Fri Aug 24 13:46:37 CT 2012

Service Oriented Architecture

For my own keeping track of things: Yegge's SOA Google-vs-Amazon rant.

File under projects/ Thu Aug 16 21:19:58 CT 2012

Is Weak Typing Strong Enough?

I found this Yegge article via this Yegge article. If you write code, they are both worth reading. If you don't write code, don't bother.

File under projects/ Thu Aug 16 21:00:35 CT 2012

Scary Smart

*sigh*

File under general Thu Jul 26 20:02:07 CT 2012

Dezi, now with Xapian backend

Read all about it.

File under projects/swish Mon Apr 30 21:39:13 CT 2012

Sleep

When I've got a lot going on in my life, I often don't sleep well. That's not unusual, I realize. Now I know that eight hours of sleep a night is a socal invention just like everything else.

File under general/ Tue Feb 28 09:05:25 CT 2012

Dezi search platform

This week I announced the initial release of Dezi, a new search platform based on Swish3, Apache Lucy, OpenSearch and Plack.

As of about 15 minutes ago, there are now PHP and Perl clients available.

File under projects/swish Sat Oct 1 21:47:46 CT 2011

REST resources

Read through the comments for some good further reading.

File under perl/ Fri Sep 23 09:58:18 CT 2011

libswish3 1.0.0 released

I am happy to announce the 1.0.0 release of libswish3:

http://swish-e.org/swish3/libswish3-1.0.0.tar.gz

libswish3 is at the core of multiple Swish3 implementations, and has reached a stable enough API that a 1.0.0 release seems appropriate.

From the README:

libswish3 is a document parser compatible with the Swish-e 2.4 -S prog API. libswish3 is a C library for parsing documents into a data structure that can then be stored and searched with a variety of IR backends.


There are currently four different implementations available of Swish3.
  • swish_xapian (C++ using libxapian, included in libswish3 distribution)
  • SWISH::Prog::Xapian (Perl using Search::Xapian)
  • SWISH::Prog::Lucy (Perl using Apache Lucy)
  • SWISH::Prog::KSx (Perl using KinoSearch)


All the Perl implementations are available from CPAN. They each rely on SWISH::3 (the Perl bindings to libswish3) and the core SWISH::Prog project, a Perl rewrite of the swish-e 2.x C binary and accompanying helper scripts. The SWISH::Prog distribution includes a 'swish3' command line interface with options very similar to the swish-e 2.x command line tool.

Xapian, KinoSearch and Apache Lucy all offer robust UTF-8 and incremental indexing support, as well as the ability to scale to many millions of documents across multiple servers.

You can read more about Swish3 at the devel site.

UPDATE: Mailing list announcement here.

File under projects/swish Wed Sep 21 22:03:59 CT 2011

Montessori and Open Source Culture

My kids go to a Montessori school. As parents we have been very impressed by the philosophy of learning there, and our kids have thrived.

It should come as no surprise to me then that there is a strong link between the Open Source community and Montessori.

File under general/ Fri Aug 19 09:40:23 CT 2011

FabFi: wifi build out of trash

This is very cool.

File under general/ Sun Jul 17 20:31:34 CT 2011

Al Franken and Ari Discuss Fishing

File under general/ Wed Jun 1 00:00:00 CT 2011

I-94 Corridor and the East Side

I live on the east side of St Paul. These big changes to the I-94 corridor would happen within a mile of where I live.

File under general/ Sat May 28 22:39:50 CT 2011

Search::OpenSearch::Server with REST API

Just uploaded several modules to CPAN that together implement a full REST API for KinoSearch indexes, using Search::OpenSearch::Server::Plack.

% curl -XPOST http://localhost:5000/foo \ -d '<doc><title>bar</title>foo</doc>' \ -H 'Content-Type: application/xml' [response:] { "success":1, "doc":{ "orgs":[], "places":[], "people":[], "topics":[], "summary":"", "title":"bar", "author":[] }, "total":"21581", "code":"200" }


The modules are:
  • Search::OpenSearch 0.11
  • Search::OpenSearch::Server 0.05
  • Search::OpenSearch::Engine::KSx 0.08
  • SWISH::Prog::KSx 0.17
  • SWSIH::Prog 0.49


  • File under projects/swish Thu May 26 13:56:43 CT 2011

    Rubber Soul

    I've loved the Beatles since the summer of 1983, when I was 11 and my aunt introduced me to their records (including some rarity vinyl iirc). The first record I ever bought was the blue greatest hits double record. I played the first side of the first record (Strawberry Fields, Penny Lane, Sergent Peppers, Day in the Life) over and over in 6th and 7th grade.

    But it's taken me nearly 30 years to identify my favorite record. I've had lots of favorite songs, but Rubber Soul is now officially my favorite record. I realize no one but me cares, but it feels important somehow to have identified an actual studio record, rather than a greatest hits compilation, as my favorite.

    File under music/ Sun Mar 20 19:53:57 CT 2011

    Mounds Park

    From my neighborhood email list:

    For all of us interested in what being/been proposed/planned for Indian Mounds Regional Park here is a link to "City of Saint Paul – Indian Mounds Regional Park Master Plan" page. It includes an overview, meeting agenda, notes, presentation along with the Indian Mounds Master Plan Concept.

    Link:

    http://www.stpaul.gov/index.aspx?NID=4036

    File under general/ Fri Feb 4 16:38:47 CT 2011

    Bug

    My laptop started making a Very Bad Noise on Tuesday morning, the kind of whirring creak I usually associate with the death throes of a hard drive. It didn't sound exactly like a hard drive, which often clicks or knocks, and the whir was rather slower paced than the high RPMs of a hard drive. But as I didn't think this Macbook Pro had a fan, since I had never heard one, I just assumed the only moving part was the hard drive and so I ordered a new one online using my wife's computer.

    When the drive arrived today, I opened the laptop to swap out the hard drive, and lo! there was a box elder bug caught in one of the two small fans on the logic board. Yes, my computer had a bug in it. I pulled out the lifeless, hard little black and orange-striped carcass, and put the cover back on. Started without problem or Very Bad Noise.

    I had thought that the term "bug" used to describe a computer glitch was coined after someone found a moth in an early computer. But according to the all-knowing Wikipedia I was wrong. Nonetheless, I was relieved to discover this bug and to fix the problem so easily.

    And now I have a spare drive for that time when my hard drive really does die.

    File under general/ Thu Jan 27 20:16:25 CT 2011

    False Laziness

    One of the three virtues of programming is Laziness. Beware of false laziness. Andy Lester writes on the problem aptly when he describes an interaction with another programmer:

    This person was one of those programmers who tried for the premature optimization of saving some typing. He forgot that typing is the least of our concerns when programming. He forgot that programmer thinking time costs many orders of magnitude more than programmer typing time, and that the time spent debugging can dwarf the amount of time spent creating code.

    File under projects/ Thu Dec 23 03:40:32 CT 2010

    Funny depending on whoami

    Only funny if you're a programmer.

    File under projects/ Mon Dec 20 21:25:40 CT 2010

    The Interruptible Programmer

    A brilliant and humane essay on changing work habits.

    Excuse me while I get up and stretch.

    I can vouch for the writer's experience, though for me it has been less about back pain (though I have that too) than eye strain (going on 7 years now). Biggest of all though has been having children and working from home: that is the interruption formula in a nutshell.

    File under projects/ Fri Oct 15 10:43:06 CT 2010

    CPAN test failures

    SWISH::3 0.08_04 is passing all tests all over the CPAN testers universe, so that is encouraging.

    However, some reports (notably on FreeBSD) report false failures because of a Wstat issue.

    I've posted about it at PerlMonks and hope someone out there has an easy fix.

    Update: finally found a fix for this. The problem is that Perl has its own my_setenv() function that interferes with the native setenv() called by libswish3.c. The fix was to set the magic Perl var PL_use_safe_putenv as shown here. This took many hours and googling to track down. Glad to be done with it (I hope!).

    File under projects/swish Mon Oct 11 00:37:31 CT 2010

    Ingrid Sequoia

    Our new daughter arrived yesterday. We are all home, doing well.

    Update: her middle name is inspired by this Wendell Berry poem.

    File under general Thu Aug 5 10:38:41 CT 2010

    Seduction and Betrayal

    This is a really good op-ed. Mayhill Fowler gets just the right amount of "philosophical and epistemological" in her reflections on the recent McChrystal blow-up.

    The storyteller changes reality, because the story changes our memory, personal and collective. It's always that way. It's the great lesson of the Deconstructionist school. All narrators are untrustworthy -- or rather, trust is not the same as fact. And faith is an act of trust, not fact.

    File under ideas/ Fri Jun 25 09:12:59 CT 2010

    Open Source Business

    opensource.com has an interesting thread on the open source business model(s).

    File under general/ Wed Jun 9 08:35:33 CT 2010

    Swish3 progress report

    There's been a ton of work on Swish3 in the last year. I've actually started planning a 1.0 release, after 5 years of work.

    Lately I've been focusing on three things: (1) making the Perl bindings easier to install; (2) indexing of compressed documents; and (3) supporting XInclude of document fragments. The first is accomplished: you can install the entire library via CPAN. The last two are aimed at large doc sets where I want to keep the XML compressed on disk for space reasons, and where I want to re-use subsets of the document collections in building multiple indexes.

    File under projects/swish Tue Jun 8 23:33:18 CT 2010

    Searching for Jesus in the Gospels

    Adam Gopnik writes a cogent and articulate essay about current biblical scholarship and the life of passersby in last week's issue.

    File under new yorker/ Tue May 25 20:42:33 CT 2010

    But Enough About Me

    A brief history and reflection on the (un)popularity of the memoir. I liked this part in which the author talks about the effect changes in technology have had on the outpouring of personal narrative:

    So if we're feeling assaulted or overwhelmed by a proliferation of personal narratives, it's because we are; but the greatest profusion of these life stories isn't to be found in bookstores. If anything, it's hard not to think that a lot of the outrage directed at writers and publishers lately represents a displacement of a large and genuinely new anxiety, about our ability to filter or control the plethora of unreliable narratives coming at us from all directions. In the street or in the blogosphere, there are no editors, no proofreaders, and no fact-checkers--the people at whom we can at least point an accusing finger when the old-fashioned kind of memoir betrays us.

    File under new yorker/ Tue May 25 19:30:13 CT 2010

    Open Source Handbook

    Open Source Handbook reviewed.

    File under projects/ Tue Mar 16 23:03:03 CT 2010

    make test

    Invoking
    make test
    in a project and watching as 1000s of successful tests scroll by, culminating in the
    All tests successful.
    message, gives me the same thrill of satisfaction as when I used to paint houses, and having finished a long day of sweaty labor at sanding and chipping old paint off, I could stand back and survey the structure, primed and ready for a fresh coat of paint. It's the anticipation that thrills, in the same way that a trip to the grocery store and a full fridge, or several loads of clean laundry folded and stowed safely away in drawers, thrills me. The knowing that I am prepared, belt cinched tight, all tests successful.

    File under projects/ Wed Mar 3 21:56:14 CT 2010

    The Patch

    John McPhee's Personal History piece is poignant and flashes like a fish in sunlight. Reminded me of the best of Annie Dillard.

    File under new yorker/ Tue Mar 2 23:29:47 CT 2010

    Non-Stop News

    My work colleagues and I just spent an intense day and a half effectively locked in a room, talking about our work together and vision for where we want to be. I was reminded of this piece by Ken Auletta on the current state of the media vis-a-vis President Obama. A lot of what he has to say about the impact of the internet, the pace of the news cycle and the breakdown of the 20th century business model around journalism is part of my daily grind.

    File under new yorker/ Fri Feb 26 20:44:56 CT 2010

    History in smells

    David Owen's piece The Dime Store Floor is a bit of nasal nostalgia. The sense of smell is a vivid memory evoker. A couple of summers ago I walked into a lumber yard's warehouse and had a sensory hit so vivid that for a moment I was 8 years old in my great-grandfather's woodshop/garage next door to the house where I grew up. Something about the old wood and sawdust and heat. The force of that memory surprised me. Owen's piece is like that too.

    File under new yorker/ Fri Feb 26 20:32:30 CT 2010

    Yahoo! Learning to Rank

    Yahoo! has announced a learning to rank contest and has put up some actual dollars in addition to some data sets.

    File under search/ Fri Feb 26 19:55:02 CT 2010

    Duck Duck Go

    Found out about Duck Duck Go via Benad's Blog. I'm hoping to experiment with LSI at $work in the coming weeks.

    File under search/ Mon Feb 22 15:29:16 CT 2010

    Trailhead

    E. O. Wilson's fiction piece in the New Yorker reads like a National Geographic article, not the kind of fiction I expect from the New Yorker. But then, that makes it the kind of thing I expect to read in the New Yorker, which is a wide-ranging publication. I liked the piece.

    File under new yorker/ Sat Feb 13 20:15:18 CT 2010

    The Apple Store

    I was at the Apple Store just now getting a bad RAM chip replaced in my MacBook. All in all it was a very pleasant experience, and aside from the inconvenience of having to drive 40 minutes round-trip for a 20 minute errand, pretty painless.

    I took the bad RAM chip, which I had identified and yanked from my machine a couple of weeks ago, in an anti-static bag I had in my desk drawer. My desk is full of them, along with spare parts and adapters and such, many for machines that haven't been manufactered or supported for over a decade. I'm a packrat for old computer junk, though to my credit I have tossed/recycled lots and lots of old "beige" computer parts in the last few years, especially now that the city/county has good recycling for that kind of thing.

    Anyway, when I handed the bag with the bad chip in it to the young man at the Apple Store, I didn't think anything of it, but on returning the bag to me he joked that it was a vintage piece. I chuckled and replied, Well, I'm feeling kind of vintage these days.

    The bag had the original label attached: 32MB Apple Quadra and Centris Series.

    The chip I had replaced was a standard-issue 2GB size, roughly 1000x more memory than the bag had originally held.

    You know you're getting old in this business when you can distinctly remember the thrill of a 32MB chip of RAM and how much pure computing power it held.

    File under general/ Tue Feb 9 14:57:54 CT 2010

    Frozen Perl 2010

    It's been a long week, culminating today in Frozen Perl 2010, a Perl conference for and by Perl hackers, here in the Twin Cities. I gave two talks at today's conference, one on Swish3 and the other on Devel::NYTProf and Search::Tools. Both talks seemed well-received.

    In the process of preparing the talks I also released a few new, related modules to CPAN this week:
    Search::OpenSearch
    OpenSearch server glue for KinoSearch and Swish-e 2.x via SWISH::Prog. There's a demo Plack app and ExtJS, using both search engines as part of the slides for my Swish3 talk.

    I think OpenSearch is very cool and look forward to doing more with that spec, including adding more features (e.g. facets) to Search::OpenSearch.
    Search::Query
    Search::Query now has support for SQL and SWISH Dialects. I hope to add KinoSearch and Xapian dialects soon. The Search::Query::Parser now has (undocumented and experimental) support for range queries, so that you can say:
    foo=( 1..4 )
    and that'll be expanded to
    foo=( 1 OR 2 OR 3 OR 4 )
    when the Dialect query object is stringified. Handy for things like ranges of dates, which is how I am using it as $work.
    Search::Tools, SWISH::API::*
    New releases of these older modules as well, with some bug fixes and refactoring to support the Search::Query.
    So, yes. A busy week.

    I enjoyed hearing other folks' talks today at Frozen Perl. There was a good variety: pack/unpack, Unicode, i18n and best practice-related presentations. I met some new people, renewed friendships with folks I already knew, and drank lots of free coffee. The cookies were good too.

    File under projects/swish Sat Feb 6 23:27:19 CT 2010

    Dave Rawlings on NPR

    You can watch/listen here.

    File under music/ Thu Feb 4 20:35:33 CT 2010

    The Vendor-Client Relationship

    So I don't surf youtube very much. Or rather, only when my kids are wanting to watch Wallace and Gromit trailers. So I'm always waaaay behind the times. That said, this video is a riot.

    File under projects/ Sat Jan 30 21:42:00 CT 2010

    Terminal Color

    For the last ten years I have used the color #E3BF70#fddc8e (hex) as my terminal background color. It's a darkish amber color that is very easy on the eyes. I'm recording it here because every year or so I have to set up a new system and always have to eyeball the settings till I get something close to what I am used to.

    Update: 26 Jan 2009 Here's my .Xdefaults file for my xterm under X11 on OS X.
    XTerm*background: #fddc8e
    XTerm*foreground: black
    XTerm*faceName: monaco
    XTerm*faceSize: 10
    XTerm*saveLines: 10000
    XTerm*scrollBar: true
    XTerm*rightScrollBar: true
    XTerm*jumpScroll: true
    XTerm*geometry:100x40+0+0
    

    File under projects/ Tue Jan 26 20:13:41 CT 2010

    I like Plack

    Plack is a Perl Web Server written by miyagawa.

    File under projects/ Tue Jan 19 10:56:20 CT 2010

    CQL

    Contextual Query Language is defined by the Library of Congress. I discovered it via CQL::Parser. Brian Cassidy is involved, so it must be good.

    I immediately thought "oh shit. Now my new Search::Query module feels late-to-the-party." But on further reading, I think a CQL dialect in Search::Query makes some sense.

    Search::Query is a SQL::Translator-like module for free-text search. I coded it up this week after brewing the idea for some many months. I'm imagining it now as a next-generation Search::QueryParser::SQL, for contexts beyond SQL. Example: I have a query string that works with Xapian and want to convert it to one that works with Swish-e 2.x or KinoSearch. Just parse it with Search::Query::Parser and assign it a target dialect and then call $query->stringify to get the translated version out.

    File under projects/ Thu Jan 14 22:46:01 CT 2010

    A Friend of a Friend

    Saw David Rawlings and Gillian Welch in concert just before Christmas. Heard this interview just now. I like the record even more after hearing Dave talk about it.

    File under music/ Sat Jan 9 23:29:40 CT 2010

    Perl6 and Perl5

    I know the people who read this blog generally do not care about Perl at all (hi Mom!) but I spend a great deal of time writing code in the language and talking with other members of the Perl community about our common projects, and so like anyone who has lived in the Perl world for any length of time, I have an opinion about Perl6. For those not in the know, Perl5 is the current version of Perl and has been around for over 10 years. Perl6 is the next major version evolution, but it has been in development for nearly the same length of time. The problem is that 10 years is a long time for a computer language release to gestate and many folks whose opinions count (i.e. managers) see that lack of a release as a sign that Perl Is Dead and not a good choice for their next programming project. So (the argument goes) Perl6's vaporware status makes it hard for Perl5 programmers to find jobs, because the "if it ain't new it ain't sexy" ethos of technology counts for more than it should with those making the money decisions.

    The real problem isn't that Perl6 hasn't been released. The real problem is the name Perl6. Perl6 is not a single executable "thing" like Perl5 is; it's an umbrella for several different projects. Right now I can sit down at just about any modern Unix-like computer and type 'perl' and write some code that runs. Perl6 doesn't work quite that way. It's a whole new language, not just a major revision to an existing language. So the version number 5 vs 6 is misleading. That's the problem. Perl is alive and well. Perl5 continues to be maintained and developed. I get lots of work done every day using it.

    Matt Trout writes a nice piece about this topic, aimed at the Perl community. I applaud it.

    File under projects/ Mon Dec 7 10:03:13 CT 2009


    Past entries: 2004 . 2005 . 2006 . 2007 . 2008 . 2009 . 2010 . 2011 . 2012 . 2013 .