Tim points out that Vint Cerf’s advocacy for network neutrality doesn’t mean that every venerable internet expert agrees with it.  True!  Google for “Dave Farber” or “Bob Kahn” and “net neutrality” and you’ll find plenty of articles… from 2007.

I think this debate has stagnated.  People are continuing to act as if the FCC’s plan is a complete unknown. This enables the lazy “evil corporation/incompetent government” frame that we all know and love. At one point this was at least semi-appropriate.  People — including those like Cerf, Farber and Kahn — made guesses about what was likely to happen, largely based on their own position along the regulation/markets ideological spectrum. “The government is trying to outlaw QoS!” “The ISPs are going to charge for Google!”  Both of these positions were hyperbolic.

But look, there’s a difference now: the FCC has announced its NN principles, to thundering… well, there hasn’t been much reaction, actually.  Everyone’s just pretending that nothing happened and sticking to their guns.  “The principles are too vague!” they cry, even though those principles say that allowances will be made for necessary network management, that throttling heavy users will be okay, that malware can be filtered, that copyright law can be enforced, and that managed services can be developed and sold.  That wipes out a significant swath of the ISPs’ objections.

Yet no one seems to have noticed. Here’s Farber responding to the new principles — he doesn’t particularly object to them, he just thinks they’re too vague and that they might make ISPs wary of trying to innovate (as if the research costs to network management are so great that the potential for work product to be shot down makes it impractical; please, this is software).  He also says they’ll lead to a bunch of lawsuits, though he simultaneously says we should instead rely on the DOJ and FTC to handle this (in a rule-free environment!), which makes just about no sense to me.

I digress.  Look: the FCC has put some of its cards on the table.  Not all!  Rulemaking still needs to occur (if it didn’t, opponents would be even more outraged).  But the agency has signaled that it’s not actually going to do the incredibly idiotic things that neutrality opponents claimed it would. Those people need to acknowledge this fact, and come up with a clarified set of objections. What is it, exactly, that the FCC’s stated plans are going to stop you from doing? Right now I’m just seeing a bunch of now-irrelevant FUD; some talk about how great a monopolistic IPTV market would be; and some hand-waving about “new business models”, which I take to be a synonym for “rent-seeking”.  It’s time to admit that the FCC isn’t the one being vague.

It’s hardly worth noting, but this is a lousy editorial.  A couple of points, one of which I buried in the last post about net neutrality:

• Comcast’s reversal of its decision to fiddle with Bittorrent seems to be getting bandied about as an example of the market taking care of things. People should be less sanguine: it was not market pressure from clients but rather PR pressure from activists that forced Comcast to back down.  And if the net neutrality issue hadn’t been looming at the time, the outcome could easily have been different.  Mustering that level of coordinated outrage is not a sustainable model for policing ISP behavior.  Other ISPs offer evidence of gloomier scenarios: in Canada, Rogers has been throttling Bittorrent — and, in fact, all encrypted traffic, since it can’t distinguish encrypted BT from encrypted anything-else — since at least 2005.  Similar difficulties face customers of some Australian ISPs.  And there are other policies that have been in place so long that people have stopped complaining: unless something has changed, I’m still prohibited from running a server for personal use off my home connection (in principle a server of any type; in practice, port 80 is the only one that will be caught). And it was years — years! — before Comcast disclosed or even acknowledged the existence of its secret bandwidth caps.  Neutrality opponents are mistaking a single victorious battle for the absence of a war.
• The Post piece refers to the broadband market as “a vibrant and well-functioning marketplace”. In truth, it’s stagnated: in North America, prices remain steady — at a rate above what much of the developed world pays — while ongoing improvements in speed show little hope for of catching up with the networks of countries significantly poorer than the US.  If the Post wants to argue that net neutrality will make the situation even worse, fine. But they don’t even seem to realize that by global standards our domestic broadband marketplace is an underperformer.

Surely any Redskins fan will agree that foremost among Dan Snyder’s sins is his selection of a broadcast outlet that only provides its audio stream in Flash.

See, I’m going to be heading back from Philly this Sunday, and many Sundays besides, and I’d rather not miss the whole goddamn football season.  If WTEM provided an MP3 stream this would be no problem — the FStream iPhone app would let me listen to the game on stretches of I-95 well beyond the anemic reach of 980 AM.

That’s not the case, though.  WTEM uses a Flash-based streaming system, apparently through a vendor called streamtheworld.com.  That’s fine for the web browser, but the iPhone’s too stupid to deal with Flash.

So I set out to provide an alternate stream.  Amazon offers on-demand virtual servers through its EC2 service, and my first thought was to fire one of those up for each game.  At ten cents an hour it would be affordable, and I could use the more-competent server to transcode the Flash stream into a more iPhone-compatible MP3 stream.

I got far enough along this path that I might as well share my work.  Here’s a quick recipe for setting up a VLC-transcoding-capable server on EC2:

1. Launch the following Ubuntu AMI: ami-ed46a784
2. Make sure that you’ve opened port 8080 in the relevant security settings.
3. Execute these commands:
   

   apt-get -y update
   apt-get -y install libmp3lame-dev
   apt-get -y install ffmpeg libavcodec-unstripped-52
   apt-get -y install vlc vlc-plugin-esd mozilla-plugin-vlc

4. Execute something similar to, but not quite this:
   

   sudo -u nobody vlc -vvv “http://208.80.52.80/WTEMAM” –sout ‘#transcode{acodec=mp3,ab=64}:standard{access=http,mux=asf,dst=}’

This is close.  It’s really close.  If you run this and then load up VLC — the amazingly useful media player — and connect to the AMI on port 8080, you’ll get a 64 kbps transcoded MP3 stream of the WTEM flash audio stream.  It works well!

Unfortunately, I don’t know the specific incantation that makes this MP3 stream compatible with FStream on the iPhone.  I tried various playlist formats.  I tried various audio formats.  I tried various container formats.  There doesn’t seem to be any decent documentation on the internet telling me what to do.  It just says it’s connecting, forever.

But while searching for a solution I discovered something interesting: there’s an iPhone version of VLC!  True, it’s 13 megs, and yeah, it’s only available on jailbroken phones (via the Cydia package manager).  But it works great!  I pointed it at http://208.80.52.80/WTEMAM and it started spitting out incomprehensible sports radio babble almost immediately.  Admittedly, that was over wifi.  But I see no reason to expect it to perform any worse over 3G than an equivalent MP3 stream would.  I’ll be giving it a shot on Sunday during the Giants game. And while I’m sorry to not be providing an MP3 stream for other fans, I’m glad to offer another reason to jailbreak your phone.

Tim wades in to the debate about robots replacing low-skilled workers. I don’t think I made my case as strongly as I could have, and consequently I can’t resist responding, if only briefly.

1. I didn’t react to the “cost of materials and energy” argument as forcefully as I should have. I really, really, really don’t think this is going to be an issue. Machines are much more efficient than humans — that’s sort of the whole point. It’s not at all a subtle difference, but rather a massive leap (they don’t call the first sweeping wave of automation a revolution for nothing). The price of technology continues to fall. And the energy costs we’re talking about are slight: keeping a Roomba charged takes about a third as much power as illuminating the lamp on your nightstand — and that’s if the lamp is using a compact fluorescent bulb.
2. Tim’s discussion of our hardwired preference for other humans is an interesting one. But I’m not sure this is a problem that will be as resistant to engineering as he thinks. Consider Paro, the robot seal. By all accounts he works pretty well at soothing nursing home patients. Tim’s probably right that people would prefer — and pay for — a human R.N. rather than Rosie the Robot in nursing scrubs. But if a suite of relatively simple assistive technologies is cheaper than either — partially automated showers, biomonitors, pill reminders and the like — they’ll probably settle for that.

Human labor remaining competitive with robots seems to assume that one of two things will happen.

First, wages for unskilled labor could be pushed even further down. Presumably this is supposed to happen as average income continues to rise. I’m not sure how this can be considered feasible, either practically or ethically — we’re basically talking about reinventing the serf.

Second, robots could become dramatically less economical due to some sort of resource or energy shock. But given robotic efficiency (and the reality of their still-falling prices), it’s hard for me to envision this happening except in the sort of catastrophic scenario in which maintaining human employment levels would be the least of our worries.

Although it is arguably as much a business innovation as a technological one, there’s no doubt that the iPhone has radically altered the mobile device landscape, and, in the process, profoundly changed the way that Americans think about sitting on the toilet.

In fact, its uses extend beyond this crass but foremost example.  You’ve also got your standing-in-line downtime, the crucial waiting-for-coffee-to-brew period, and of course miscellaneous hours spent idly fiddling while in meetings. If I’m not mistaken, the Bureau of Labor Statistics is on record as saying that high scores from Sudoku games played on mobile devices account for most of the productivity gains of the past decade.

Personally, I like to use the device to catch up on my online reading.  But this crucial electronic-fucking-around is slowed by the need to enter URLs.  If you’re like me, typically you’ll have some long-form article open in a neglected browser tab: say, 2000 words from Sasha Frere-Jones on why your implicit acceptance of the latest top 40 hit means you’re more — and less! — racist than you thought.  It’s something I mean to read, but which the demands of the workday have made me put off.  As I leave my desk to pursue less productive activities (food acquisition; interaction with humans), I frequently find myself wishing I had that article loaded on my handset.  But the only way to move it there without tedious typing is to email myself the link, then check my email, then click on the link.  If only there was an almost perfectly equivalent but slightly faster way!

Well, you’re/I’m in luck!  I twittered this ridiculous first-world complaint the other day, and @tbridge and @jroo helpfully pointed me toward Prowl, a $2.99 iPhone app that takes the 3.0 firmware’s push notification capabilities and wraps them in a simple API.

This opens the door to creating a bookmarklet that grabs your browser’s current URL and pushes it to your phone.  The phone will buzz, you’ll click the “view” button, and you can then follow the link.  Easy!  Here’s the bookmarklet.  Just follow these steps (which, I should note, have only been tested in Firefox 3.5):

1. Buy Prowl.  Open it on your iPhone — you’ll need to register with its parent site and give the app permission to display notifications.
2. Using your newly-created credentials, log into the Prowl site. Go to the settings tab and create an API key by clicking the appropriate button.
3. Paste the API key into the form field below. Click the “Create Bookmarklet” button. NOTE: the customization of the bookmarklet is done in client-side Javascript, entirely within your browser — don’t worry, you won’t be sending me your API key.
4. Drag the newly-created button into your browser’s quicklaunch bar.

Simple!  Now when you click that bookmarklet a new window will be briefly opened.  It’ll submit a request to the Prowl site that contains the current URL that you’re looking at.  Shortly thereafter you should get a message on your phone with the link.

As for the new window: I admit, it’s inelegant to spawn a popup.  But the Prowl API only accepts POSTed requests, which, barring a sudden and deeply unwise decision on their part to host third-party scripts, rules out a more seamless AJAX solution.  The popup works well enough, although on especially slow connections the window may wind up closed before the request goes through.  A better solution would involve a timer that checks the spawned window for when its location.href property suddenly becomes inaccessible due to cross-domain security policies (indicating that the page it contains is now being served by the Prowl domain).  But my first crack at that didn’t work, so for now you guys are stuck with this.

Nextbus stops! Served by Google App Engine! Displayed on the iPhone!

Of course:

• The app crashes as soon as you try to do anything after the map loads.
• There are way too many pins. And I don’t want them to be pins anyway. And the stops aren’t likely to change often so I should actually move this operation off the network and onto the iPhone, which might mean reimplementing geohash, which I don’t yet fully understand (but know to be neat!).
• I haven’t written code to query Nextbus for predictions. This will be fairly easy, but exactly how I want to handle caching is an open question.

Still: wheels, motion, etc.

P.S. Objective C suuuuuucks

It seems like the Iranian cyberactivist movement has marked the entrance of the term DDoS into our culture’s shared vocabulary. There’s a big DDoS attack going on right now, in fact: the Post has coverage (they’re among the targets), and I heard a piece about it this morning on NPR. Governments are among the victims. People are starting to notice that the agencies responsible for dealing with this stuff don’t have the power to do much more than write whitepapers. Governments are going to start doing things about it.

I think this will likely take two forms:

• Legislated standards for ISPs. In particular, I imagine we’ll see ports for IRC and SMTP, if not all non-http ports, move to a default-closed state. I’m not sure how you’ll opt into opening them, but SMS verification of an online request seems to be increasing in popularity (my bank does this; Google does, too, when you start to use their App Engine service). There’ll also likely be incentives or mandates put in place for things like server-side antivirus scanning of email attachments. The details matter quite a lot — the possibility of a corporate power-grab that constrains citizens’ ability to interact securely is real — but I think such attacks can be fought off, and the result will probably be a good thing on the whole.
• These ISP standards could conceivably get written into trade agreements, at which point a more politically interesting possibility will arise: the imposition of economic or network sanctions against nations judged to have an out-of-control internet. Is there a damaging DoS attack coming from South Korea? Is that country’s government judged not to have adequate standards in place to fight or prevent it? Cut ‘em off the network, or throttle their traffic. There’s a constituency for this: the WIPO people would be only too happy to have this capability.
  
  
  This opens up other interesting possibilities. For one thing, the mutually destructive nature of sanctions would become immediately clear to internet users in a way that isn’t always obvious in the comparatively sluggish realm of trade. And for another, online false flag operations would become a more serious concern. A hacker group could conceivably blackmail a small, cloutless nation under the threat of eliciting an international network crackdown.

It’ll be interesting (and no doubt horrifying) to see deep packet inspection debated on the House floor. I think we’re headed that way, though.

Oh yeah: let me plug this article, which is the more entertaining narrative about a DoS attack that I’ve come across.

GGW points me toward some pretty discouraging news about how NextBus has treated other transit app developers. They assert their copyright on arrival times; they demand licensing agreements, yet refuse to do so on a small scale; and, perhaps most astoundingly, they say that free iPhone apps count as commercial use of the data because apple sells the iPhone on the strength of the app store’s offerings. Ugh.

On the bright side, the technical aspects of my work on a map-based NextBus iPhone app are proceeding well. At the moment there are no obvious obstacles that need to be overcome. That’s not to say those obstacles don’t exist — they almost certainly do — but a seemingly clear path ahead is about the best you can hope for in a technical project.

Oh, and I had another look at the GTFS/NextBus stop match-up. The news is still bad. I’d hoped that the average distance between stops was being pulled up by a few bad matches, but that doesn’t seem to be the case. In fact there are only about two dozen matched stops closer than 10m from one another. I’ll have another look eventually (maybe NextBus is using a different projection system?), but for now the NB dataset is sufficiently complete that I can rely on it for my work.

No word yet from WMATA, but I did end up writing a script to grab NextBus’s routeconfig data (download here). Then I tried to match each NextBus-defined stop with the closest one in the GTFS dataset. Some stats*:

• NextBus’s dataset tracks 711 unique stop IDs. GTFS has 10,380.
• Using this function to measure distance, the average space between matched stops is 164 feet. The smallest is 11 feet. The largest is 9/10ths of a kilometer.
• 218 NextBus stops wound up sharing the same GTFS stop.

All in all, pretty bad — this level of data quality is clearly unusable. My GIS skills are weak; this may be my own stupid fault. I’ll consult with some experts and see what I might be doing wrong. But the basic distance-matching idea is pretty straightforward, so I’m not terrifically optimistic. It’s possible that data quality is just going to really, really stink — to be sure, this is not particularly encouraging. Here’s hoping we can get a proper lookup table out of WMATA or NextBus. Otherwise I don’t see a great alternative to manual intervention.

* These numbers ignore the routes that NextBus tracks but which GTFS does not; those are B99, F99, L99, NH1, P99, REX, S80 and S91 (they appear to be shuttles and the like). I haven’t yet identified the routes that are in GTFS but not tracked by NextBus.

Tim was nice enough to write a tweet endorsing my article about the potential downside of Twitter’s emerging political importance. But he noted that I didn’t say much about what the alternatives are — fair enough! As I said to him in response, I was only too happy to have word limits save me from having to propose a solution. Even though I think the situation is unfortunate, at this point I suspect that there isn’t much to be done about Twitter’s rising political relevance.

But, y’know, time heals all wounds. I am convinced that Twitter’s import as a cultural hub will decline. Twitter won’t go away entirely, mind you — it’s a genuine medium unto itself — but I think its true legacy is likely to be a frankly unbelievable extension, evolution and popularization of the capabilities represented by SMS. Multicast? Common use of symbolic delimiters like @ and #? Widespread institutional adoption? Two years ago, if you’d asked Verizon when SMS would be used this way, they’d have laughed in your face. Now the marketplace is going to demand this functionality — if not from Twitter, then from someone else.

But as I said, I think the conversations happening on Twitter will become less relevant, and the medium less vibrant. Actually, I’m beginning to think that this is an iron law of online mediums. This post (via Megan) helped focus my thinking a lot.

Here’s how it goes. First, a network achieves viability — enough people are using it to send non-”hello world” messages that the community can sustain itself. Next, users experiment, publishing and republishing content that they find compelling. The system amounts to a collaborative filter, and the quality and novelty of the results are surprisingly good. At this point people begin to notice and discuss the potential for the network to have greater relevance — and, inevitably, those who don’t understand that participation in the filtering activity is non-negotiable begin whining about taking the medium seriously when they see so much trivial content on it. Despite this carping, more users join the network and its value and potential importance begin to be more widely understood. At this point users change how they identify content worth publishing or republishing: rather than the first-order “how compelling is this?” they begin using the second-order “how compelling will other people find this?” Although they were excellent and determining what they thought was interesting and appropriate, they’re comparatively terrible at determining what other people will like. Quality declines (“I blogged: del.icio.us links for 2009-07-02″). Worse, as users continue to try to shirk their collaborative filtering responsibilities, experimental uses of the medium are discouraged or otherwise become less viable. The system ossifies, and soon enough everyone is sick of having to check Facebook. Time for a new no-pressure medium for goofing off with your early-adopter friends. Rinse, repeat.

I don’t want to oversell the preceding — I’m pretty sure that Clay Shirky accidentally scribbles more profound sociological observations about the internet during the course of searching for a working ballpoint pen at the bank. But this is my understanding of the situation, and by now I think we have enough data points to conclude that most, if not all online social networks achieve viability, blossom and stagnate (it may still be entirely possible to run a viable business during the stagnation phase, I should point out).

It’ll happen to Twitter, too — it is happening. So let’s start talking now about what conditions we should demand of the next medium-of-the-moment before we start moving our political institutions onto it. My suggestion for a place to start: open, free, and likely to remain so.

UPDATE: This is somewhat related — it’s an example of what I mean by people withdrawing from the collaborative filtering process.