an eddy in the bitstream

Author: Peter Karman (Page 6 of 76)

Maker of lunches.
Teller of stories.
Singer of songs.
Crafter of code.
Kicker of darkness.

Elasticsearch development

Over the last few years I have done a lot of development against Elasticsearch, especially using the Ruby libraries. The Elasticsearch::DSL is very powerful, but anything beyond a simple query can take hours of debugging to get the syntax just right. I always learn the hard way that it is better to handcraft the JSON first to get the logic correct, and then translate it to the Ruby DSL code.

I don’t do it often enough to remember, so after my latest struggle with ES, I thought it useful to document.

The curl command is:

curl -X POST localhost:9200/myindex/_search \
-H 'Content-Type: application/json' -s \
-d '@q-test.json' | json_xs  | jq .hits.total

where q-test.json is a file with the DSL JSON.

18FX

Just finished my last day at 18F. Dropped off my #FedEmployeeInABox at UPS. Feeling all the feelings right now, but mostly gratitude for the opportunity to use my technical skills to benefit so many people.

I wrote this Medium essay about my experience.

(Edit 2020-01-24: added a copy of the Medium article here for posterity)


Leaving 18F

Two years ago, when I accepted the opportunity to serve my country within the US government as part of 18F, the digital consultancy within the General Services Administration, I had some unspoken expectations. Now that my two-year term is ending and I am leaving government, I have reflected on how my experience did and did not meet those expectations, and some new expectations about what I’m going to do next.

I expected career government employees to resent and resist the technical hero narrative.

And they would be right to do so.

All the press I read about 18F and the U.S. Digital Service and the Presidential Innovation Fellows before I started suggested that everyone was a Silicon Valley Mensa member with a big-name tech company pedigree, and that all these experts (who seemed overwhelmingly younger than me) were swooping in to government to fix things and rescue the befuddled civil servants from the dragons of Byzantine regulation and bureaucracy and bad technology.

Those stories are false. They perpetuate a larger false narrative that the government is a monolithic, faceless force, bent on making people miserable, made up of “them” and that all the rest of “us” are powerless to affect it. That story is just wrong. The government is us. Its effectiveness and illumination, darkness and implacability, are our own. Our Constitution says it clearly: “We the people” — not us the people and them the government. I’ve not ever felt that quite so clearly as while working within the government.

The truth is that most 18Fers are not from Silicon Valley and do not come from big-name tech companies. Our team lives all over the country, and our distributed-first ethos allows us to span many geographies, including my own, here in Kansas. Sure, we have some experience doing technology at a faster pace and with a newer toolset than government is used to. But most of us had very little experience with how government operating authorities, regulations and procurements work. 18F is not made of up of experts in government. So there’s learning and negotiating in both directions.

Whenever 18F has been effective as civil servants with technical skills, it has been due to active and engaged partnerships with folks already in government, who were already working hard at reforming technical attitudes and processes long before 18F existed. We are not heroes or saviors. We are partners, in a long process of reimagining how technology can help improve people’s experience of digital government services.

I expected my spirit to be slowly ground into dust under the harsh stone wheel of government bureaucracy.

Based on my previous experiences doing technology within large, bureaucratic organizations, I expected to barely manage the two years of my term before I ran screaming from rules and procedures that made little sense to my personal mental model of the Way Things Should Be. Imagine my welcome relief at finding that my 18F colleagues shared my feelings and had already done so much work to make onboarding and getting questions answered downright pleasant.

I won’t lie. There are things about government that make me want to pull out what’s left of my hair. But most of them are the same things that any large organization suffers from: processes to mitigate risk (and blame), rules and traditions whose original motivations are now opaque if not completely lost, and the kind of small-p political squabbles that constitute fairly normal human interactions. (There are large-P Politics too, which I’ll get to later.)

I am grateful for my fellow travellers in government, and to the Slack channel #scream where catharsis was always a keyboard smash away. The quality of good humor is not strained.

I expected my fellow 18Fers to be really smart.

I was not disappointed. Everyone is really smart.

Better, everyone is really fun. I’ve had so many delightful conversations with folks while waiting for meetings to start. Our Slack channels are emoji-filled rivers of humanity and brilliance and sympathy and insight and puns and fascination.

Better yet, there is a culture of empathy and learning and humility and humor that is a joy in which to participate. I have been personally stretched and challenged by the hard work of the diversity guild, the “hey guys” Slackbot response, and the community of engineering peers who take documentation, testing and code reviews as seriously as any code, and I am a better person for it all.

Best of all, the lack of ego makes the air feel lighter, like hiking above the smog. Folks take the work very seriously, far more seriously than they take themselves. I was inspired, every day.

If you know me, you know I’ve loved The West Wing since it debuted nearly two decades ago. My favorite character is Charlie Young, who in this scene joins the president’s staff. I’ve thought of this scene often during the last couple years because of the final exchange between Charlie and Josh Lyman.

Charlie: “I’ve never felt this way before.”

Josh: “It doesn’t go away.”

I expected to do the minimal two-year term of service and no more.

It is possible to work for 18F for more than two years. Many of my peers serve a second two-year term, working the maximum of four years. I knew of that possibility when I started, yet based on what I knew about my own capacity for working within bureaucracy, I expected to be quite ready to leave after two years.

So I was surprised, in the fall of 2016, a year into my term, that I could imagine serving another term. I was working on a very interesting project (login.gov) and feeling hopeful about the work we were accomplishing.

Then the election happened.

We don’t talk about candidates or political parties at work. An essential part of the culture and ethics of the civil service is the non-partisan nature of our work. Our oath is to the Constitution and we serve the people, not a party or an administration. Still, it’s not an exaggeration to say that every morning since Election Day has been a challenge to summon the will and focus necessary to do the work. It is demoralizing to work within a system headed by an administration that does not respect the system. I believe my colleagues feel this too, not just at 18F but throughout the civil service. The reason is obvious: the current administration does not respect the civil service and is actively working to dismantle it.

I am not registered with a political party. I believe in people organizing themselves to improve the welfare of our species and the planet we share. That’s the tl;dr of the preamble of the US Constitution.

So that’s what I’m going to keep doing: helping to organize to improve the welfare of our species and the planet we share. For now, I’m going to do that outside of government.

I expect to be sad for awhile.

A few weeks ago someone asked me what I look for in an ideal co-worker. Without hesitation, I said someone just like the people I work with at 18F. I spoke quite passionately about all the virtues of the people I have worked with for the last two years. Some of those words probably made it into this essay.

That person responded, wow, you must be sad to leave. I had not, until that moment, realized the depth of my emotional attachment to my colleagues and to the work and to the institutions which we serve. I will indeed be sad for some time about my decision to leave, and I apologize to all my future employers and co-workers right now, because 18F has set a high-water mark for me in terms of what I expect from a workplace.

In a few days I will no longer be an 18Fer. I’ll be an 18FXer, an alum. It is one of the proudest professional badges I will ever wear.

I expect to focus more on the process of building things, and less on the things.

The real product of 18F is not software or websites or improved procurement policies. The real product is the process by which we make those things: focused on the people who use the things we help build, in open, agile collaboration with the partner agencies we serve. The product is the process. The artifacts we produce are the means, not the ends. We don’t have an enthusiastic commitment to open source because it necessarily produces superior code (though it often does). We do open development because the process of working in the open is a good exercise for government. Like voting, it reminds us that the work belongs to all of us.

It took me a year to realize all that about the 18F product. Once I did, it became far easier to trust that we were delivering it, every day. Of course, producing great websites is a requirement for this process-as-product. If we failed to deliver great websites, it would be hard to trust that the process really has the quality we claim it does.

My own epiphany during these last two years has been that it takes deliberate, conscious action on the part of individual team members to perpetuate the culture of open process. Openness doesn’t just happen by itself. Someone needs to remember the disciplines: daily standups, retrospectives, sprint planning, documentation, agile methodologies, adherence to team practices, working in public. Like democracy itself, open process requires daily practice, permission to fail and be messy, reflect and try again.

I expect to experiment more, in direct conversation with the people who use the things I build.

Before I started at 18F, I knew intellectually the value of user-centered design and user experience research and iterative development. Now I know it in my bones. I’ve never worked at a place where design and user experience professionals numbered as many as software developers. It has changed forever my appreciation of the varieties of human experiences, of suspending as best I can my assumptions about how things are supposed to work, of trying things out to validate hypotheses, of changing my mind and trying something else. Learning as I go, listening to others’ experiences of the things I build, and trusting that their experiences are not a value judgement of me, but that instead we are working together to build a thing that benefits us all.

These obviously aren’t new ideas, or even new to me. What 18F did was give me an opportunity to practice those ideas as a technologist every day by treating them as core virtues, not nice-to-haves.

I expect to be politically involved.

Political engagement is a continuum, from alive and breathing on one end, to running for office and active leadership on the other. I have never been particularly politically active. In my cynical 20s I voted only in presidential elections, reluctantly, and even then doubted the effects. Back then I believed that my vote did not matter, and that organizing to affect political change was not very effective. What I’ve come to believe is that my individual vote doesn’t change the election. My vote changes me. It moves me along the continuum, and turns me into someone who cares about the outcome because I have invested myself in the process. It connects me to “we the people.” With all the obstacles to voting that exist in this country, it’s an experience being denied to large groups of people. That’s not right, and it’s not good for our democracy.

One thing I have learned from my time in government is that the long, slow, difficult repentance from cynicism looks a lot like showing up, doing the work that needs to be done, and trusting that it matters.

So what I’m doing next is going to be a lot of listening to people who have been politically active for a long time, trying to learn from them, and using my technical skills to improve the systems that power the democratic process at the local and state level. Make it easier for people to register and vote. Make it easier for people to run for office. And make it easier for the candidates who reflect my values to win those elections.

If the 2016 election had turned out differently, would I be leaving government now? Probably not. I’d probably be content to do another term. Am I leaving because I don’t want the efforts of my labor to reflect well on the current administration? Partly, but that’s a pretty minor reason for me. The American people benefit from improved services far more than any administration does, and I respect my colleagues who are staying and continuing the work. Mostly I’m leaving because, while the unjust parts of the American story sometimes threaten the whole, I still love this place more than ever. I respect the history and integrity of the office of the president and what it stands for, and I don’t ever want to feel again like I’ve felt for the last six months.

Because here’s the thing: our current president disgraces the office and undermines the trust necessary for democracy to work. And by its daily silence and denial, the party that claimed and elected him disgraces itself and the democratic institutions of our country. I believe how we fix that starts with our local school boards, and our city councils, and our county commissions, and our state legislatures. It starts with deepening the bench of folks, inclusive of all genders, ethnicities, colors and classes, who feel empowered to run for higher office. So that’s where I’m headed next.

If you have specific ideas about how to do any of those things, I’d love to hear from you.

Bart Campolo

[This is a letter I just sent to the New York Times magazine editor. They won’t print it, so I am posting it here.]

I was not surprised to read about Bart Campolo’s move to secular humanism, nor that the evangelical community “has barely noticed.” American evangelicalism has always been less about what you believe and more about how you believe. Evangelical theology functions more as a shibboleth than an orthodoxy.

Bart’s father knew this. I will always remember hearing Tony preach 25 years ago in my midwestern college chapel service. In the heat of his sermon he would say the word “shit” and observe immediately that we were more disturbed by hearing that word than by the poverty and hopelessness he was describing. Words signify which camp we belong to as much as anything.

Those of us who leave the religious traditions of our childhoods don’t so easily leave the psychological patterns those traditions imprint. The toilet paper sticks to your shoe even after the bathroom door swings shut. Noticing that about yourself, and freeing yourself from the endless repetition, is a lifetime’s work. I’m glad if Bart has found a certain relief in his new belief system. He still smells like an evangelical to me.

« Older posts Newer posts »

© 2024 peknet

Theme by Anders NorenUp ↑