it’s probably a good thing, but it’s at least a little scary

Today’s coordinated action in opposition to SOPA/PIPA by the internet community is something that’s both heartening and inspiring.  Its volume–despite a lack of coverage that should deeply embarrass the traditional media–and the speed with which it’s prompting legislators to reverse position are both impressive.  It could easily turn out to be remembered as the moment when “the internet” emerged not just as a political force but as a political constituency.

Still, those pointing out that today’s site blackouts are corporate speech are not wrong.  And this community’s ability to call what amounts to a general strike is more than a little worrisome, at least in principle.  The country is ever-more reliant on web companies to conduct normal business, but it has gotten into this position without entering into contractual arrangements with those companies.  The nature of information technology’s ability to scale means that the decision to deny web service will often rest in relatively few hands (with the exception of explicitly communal projects like Reddit and Wikipedia).  This all makes me at least a little bit uneasy.

In practice I suspect it’ll all be fine: there aren’t a lot of issues with the positive optics of speech rights; and the mass net movement around SOPA/PIPA really has originated with the grassroots.  I don’t think this kind of campaign would be thinkable for web companies without some genuine democratic underpinnings to support it.

Still, all of this positive activist energy would be much less charming if it were being directed toward sales tax amnesty or another one of the web startup community’s harebrained political priorities.

UPDATE: Some similar thoughts over at GigaOM.

what would a data scientist do for a campaign, anyway?

I take it that this is what the Obama campaign was hiring “data scientists” to do.  It’s an interesting project, but I think the Slate writeup fails to make a few important connections.

First: how likely is it that the campaign actually needs a new method of gauging supporters’ opinions and priorities? Not very, I’d say. Surveys and focus groups are well-developed methodologies that don’t suffer from the obvious selection effect problems that a “share your story” feature on a website does.

Second: pay attention the the bits of the article recounting Rayid Ghani’s resume. The work he did for Accenture sounds legit — applying classic machine learning techniques to product classification and sales projections.  But how much of it dealt with expressed opinion (“share your story”) versus observed behavior (“was this product purchased?”).  The answer: lots of the latter, none of the former.

And this is important. By themselves, personal expressions are not terrifically useful. People don’t know their own minds all that well. But personal expressions can be a useful signal when correlated with objective observations.

I think “[this project is] about us being better listeners” is an pretty generous gloss on what’s being done here. The odds of this being an input to the platform or priorities of an incumbent president’s campaign seem certain to be low.

I expect that the campaign is using this corpus of personal essays to classify supporters into better-segmented subgroups, so that more specifically customized appeals can be made to them. It’s conceivable that this will be deployed for GOTV efforts, but that’s a tricky experiment to run since you only get to do a trial once every few years.  More likely this is about fundraising, with classic A/B testing used to hone the model and refine the system of segmentations and corresponding appeals. If that approach delivers better results than existing methods, they’ll try to use whatever demographic characteristics are available in the voterfile to segment potential supporters who haven’t helpfully supplied personal essays (“he’s a middle-aged white guy from Detroit–put him in the ‘the bailouts worked’ email group”). Whether you can generalize from someone engaged enough to contribute a “share your story” entry to someone who isn’t seems like a big if, but hey, it’s worth a shot.

That’s my guess, anyway. None of this is nefarious; it’s just what campaigns do.  But I think it’s important for people to understand that, rather than a triumph of democracy-enabling technology, this is just a quantitative approach to squeezing even more money out of voters and into our political system.  It’s necessary work, but I wouldn’t say it’s exactly noble work. If you want your political opinion to shift the policy needle, you should probably start by calling or writing your representatives, not by pouring your heart out to barackobama.com.

ok, this is the last LED Christmas lights video

I promise. Probably. I probably promise.

progress

this battlestation is… more operational

GE Color Effects Lights logic in Python

I’ve been playing with these holiday lights recently. They’re very neat: each bulb has RGB LEDs embedded in it and is individually addressable (albeit with only 4 bits of resolution per color channel). Some wonderfully talented folks have reverse-engineered the protocol that the light controller uses; others have written Arduino code making it possible to snip the light’s control line, connect it to an Arduino, and begin programming your own animations. Still others are doing various neat and unusual things with the result. Personally, I just want to have some interesting light displays for New Year’s.

These other folks’ work makes this possible, but not as convenient as it could be: developing new animation routines on the Arduino is a pain in the butt. To do so you write your code in the Arduino’s irritating development environment, then flash it to the chip and hope that everything worked right. There’s a small risk of damaging the lights, and a large chance of things going wrong.

So! I wrote some bridge code that moves the light control logic into Python, which can run and be manipulated on a regular ol’ computer. The state of the lights is then sent to the Arduino many times per second (though not as many as I’d like — still, only about 1FPS slower than the string’s theoretical max) and the Arduino dutifully updates the state of the lights. This should make development of animations easier, and make it simpler to trigger or modulate them in response to network events or other things that the computer can detect.

If you have any use for this code, you can find it here. For me, there are two challenges remaining: creating some interesting animations, and connecting a second string to the Arduino.

This latter issue is a bigger problem than you might think: the lights’ address space is only 6 bits — too small to simply connect the two strings together and retain the ability to control individual lights (bulbs can share addresses, but I’d rather not do that). So the two strings need to be treated as individual entities. That’s easy enough on the Arduino side. But the Arduino is going to live at one end of this double-string, not in the middle. That means that the signal to the far string will have to be transmitted along the length of the near string. This is too much distance for a fast serial connection to traverse without being spread out into illegibility.

The solution, I’m told, is to shift the serial signal to RS-422, a higher-speed and more ethernet-like standard, then back to serial at the light string (it will travel along a twisted pair of conductors that I’ll run along the already-too-heavy first string). I have the chips to do this, and it all *looks* pretty simple. Fingers crossed…

well, that was much harder than it needed to be

I just spent some time banging my head against this problem; let me save people wandering in from Google similar trouble.

So: you’re trying to use pySerial to speak to your Arduino. It’s not working. Or sometimes it is, but only when you have the Arduino Serial Monitor window open! This makes no sense.

Here’s the deal: the Arduino’s fancy “you can flash new programs onto me without pressing my reset button!” functionality works by resetting the damn thing whenever a new serial connection is made. With the serial monitor open, the connection’s already open, so no reset happens and your script might actually work. When the monitor is closed, the Arduino resets when Python connects to it — and the Arduino might still be busy booting up when your script begins shoving bits across the link.

So! You can disable this auto-reset behavior (you’ll have to ask the Arduino people about that), but it seems simplest to just make your python script long-running and wait patiently for a beat or two after opening the serial link. Here’s a dead-simple example:

1
2
3
4
5
6
7
8
9
10
import serial, sys
from time import sleep

SERIAL_PORT = '/dev/tty.usbmodem621'
SERIAL_RATE = 9600      

ser = serial.Serial(SERIAL_PORT, SERIAL_RATE)
sleep(2)
val = int(sys.argv[1])
ser.write(chr(val))

Here’s the Arduino sketch. It just blinks the LED on pin 13 however many times you sent as a byte (I’m working on sending binary-ish data over the serial link; hence the use of raw byte values):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
byte inByte = 0;

void setup()
{
  Serial.begin(9600);
  pinMode(13, OUTPUT);
}

void loop()
{
  if (Serial.available() > 0) {
    inByte = Serial.read();
   
    for(int i=0;i<inByte;i++) {
      digitalWrite(13, HIGH);
      delay(500);
      digitalWrite(13, LOW);
      delay(500);
    }
  }
}

There! That was irritating.

it’s a divide all right, but it’s not all that digital

This is more or less why I don’t spend a ton of time at work worrying about the digital divide.  It’s tempting to tell yourself that the reason politically marginalized populations don’t avail themselves of their representatives’ attention is that they lack access to the necessary technologies.  If only you could write your senator through an SMS interface instead of needing an expensive, Javascript-capable smartphone!

But of course this is nonsense.  To a first approximation every American can afford to use the telephone and the post office. And in urban and suburban areas, at least, libraries make computers widely available.  What marginalized people lack is time, attention and education.  Maybe they don’t have a cellular data plan either, but first things first.  Besides, to the extent that technology access is a real problem, falling electronics prices are likely to do much more to address the issue than even the most conscientiously designed interfaces (on this score, the news on smartphone penetration is very encouraging).

Still, just because projects that concentrate on technological deficits are focusing on a small and relatively unimportant part of a dauntingly large problem doesn’t mean that such efforts aren’t laudable.  There’s always room for making a difference at the margin.  And there’s something to be said for principle.  Designing our work to be inclusive and truly democratic is important — sometimes more important than maintaining a depressingly realistic appraisal of our chances for coding our way to egalitarianism.

(I should probably add that I think creating accessible interfaces for politically engaged populations that have difficulty using technology — the disabled, seniors, etc — is a very different endeavor from the broader/gloomier digital divide issue)

yet more a la carte

Matt and I have gone back and forth about this before, and I’d be lying if I said I felt completely confident in my response to the licenses-for-parks metaphor.  But I do feel confident that he’s wrong about this:

[...] Once the cable company has built the infrastructure to deliver cable to your house and given you the box, it doesn’t cost them any more money to give you 100 channels rather than 10. To be sure, building infrastructure capable of delivering more channels costs more money. But once the infrastructure’s built, it can deliver what it can deliver and there are no incremental savings to achieve by not using all the capacity. I think it’s completely true that cable television has become a questionable value proposition—it’s extremely expensive and though the infrastructure to deliver hundreds of channels to the home is impressive, it’s not actually useful since nobody watches that many channels. What we actually need is faster broadband internet. But the infrastructure’s already built. Refusing to use it doesn’t reduce the suppliers’ costs and wouldn’t save customers any money.

Two things. First, the cable company buys its content from various aggregators and producers of video product. I’m sure that the terms of these purchases vary considerably, and to some degree must be tied to consumer demand for that content. But there is marginal cost to adding a new channel even if there isn’t marginal cost to distributing it. There’s necessarily cross-subsidization going on, which means some viewers are getting a raw deal. Probably most of them, I’d argue: given the lack of choice in the market, there are strong incentives for the cable company to add new channels in order to widen their customer base; they can simply add the new features to their existing customers’ tabs. iTunes, Amazon and Netflix can be understood to be competing with cable and satellite largely on the basis of offering a non-venti option to consumers (to borrow Matt’s analogy) and they’re having a lot of success with it because many people rightly intuit that they’re not consuming $100 worth of video product every month.

Second, nearly all cable systems are now or soon will be digital, which means they’re packet-switched, which means that, to varying degrees, broadband capacity and video capacity are fungible. The actual details are, I’m sure, quite convoluted and complex, and we’re not going to be able to hash them out here without help from someone who understands the relative merits of different versions of DOCSIS. But it’s safe to say that there is one copper conductor entering your house that handles both internet and TV, and that the portion allocated to each can be changed through configuration choices made by your cable vendor.

the primal mud run

me, exhausted and covered in mud

Somehow Charles talked me into doing the Primal Mud Run on Saturday. It’s true that I’ve been adopting more of his crazed fitness strategies over the last year, but this was still an entirely new thing for me.  I’m not a very good runner, and making my 5k debut with an event that also involved mud and obstacles was daunting.  Today I’m sore and my lungs seem to still be reconfiguring themselves to a level of functional ambition they had never before considered. But I’m glad I did it.

One part was particularly interesting. The most unpleasant obstacle involved wading into a very cold pond and swimming under some barrels.  You know how when you hit your thumb with a hammer or burn your hand on the stove, and say “Ow!”, there’s some sense of volition tied to the exclamation? I mean, it’s not a deliberate act, exactly, saying “ow”.  But you could suppress it if you wanted to.

Well, each time I came up from under those barrels I was surprised to hear myself emit a sort of pathetic howl/whimper, not dissimilar to the ones tough-minded Jack Bauer might have drawn from a sinister interrogation subject.  It was just completely automatic and beyond my control. It’s always interesting and horrifying when you discover some new thing your body does without asking your permission. I trudged through the rest of the event in a kind of animal stupor, and when I was done Charles had to show me where I’d left my jacket.

But this was all pathetic and ludicrous. A hundred yards away, trucks were selling beer and barbecue sandwiches. Other participants were running in halloween costumes, tutus and thrift-store suit jackets.

All of which just goes to show that I would have made a terrible Marine. And the next time you hear me speculating about strategies for surviving the collapse of civilization, you should probably roll your eyes even harder than you normally would’ve.

UPDATE: Charles found some photos.