Way back when, I wrote a Google Maps application for DCist that overlaid the DC Metro system on the usual GMaps tiles. People found it useful — me especially, since I think it helped me land a job at EchoDitto.  Its only real innovation was some simple, hacked-up geometry that would horrify a cartographer, but which allowed me to make an attractive map that recalled the more stylized WMATA map.  It wasn’t rocket science, but I still occasionally get emails from developers asking me how I did it (which is slightly bizarre, given that the code is right there for them to see).

In 2007 the GMaps API got an update, and I converted the project into something called a mapplet. I had to rewrite a few things, but it was more or less the same.  The main difference was that mapplets were used through the maps.google.com interface — you could add a bunch at the same time, but you could still use Local Search and permalinking and comments about businesses and other Googly innovations from within the interface.  I didn’t have to implement any of that stuff!  Instead, users could simply have their polished Google Maps experience supplemented by my modest mapplet.  Handy.

Unfortunately, over the last few weeks I’ve started receiving reports that the mapplet’s behaving weirdly.  Load the mapplet, then do a search for something — the station markers will disappear, and sometimes some of the lines that are supposed to connect them will, too.  It looked to me like an event handler had started working differently, so I went to investigate.

Alas!  It turns out that v2 of the API has been deprecated.  They’re on to v3 (not so bad) and they’ve discontinued the mapplet platform entirely (bad)!

I can still make the lines appear on a Google Map.  But I don’t think I can do it on the maps.google.com interface.  This is a drag: I don’t think the thing’s half as useful as a standalone product as it is when it supplements search functionality.  And I really don’t want to reimplement the entire maps.google.com interface (even though, yes, they expose the API for their local search stuff).

So! Developers! Anyone out there dealt with this? I’m not eager to dump a huge amount of time back into this project — a project that’s increasingly unnecessary thanks to Google Transit and the addition of transit stations to the GMaps tileset, but which is still useful when you’re working at a modestly wide zoom level.  But it would be nice to get things working again.

Kevin Drum elects to take security advice from Microsoft. This is not a good idea!

In a nutshell: MS says that users ignore warnings about unsigned encryption keys, which makes those warnings useless.  Some browsers, like Firefox, make it really difficult to ignore unsigned keys, but that’s annoying, and MS says we should abandon such efforts.

This is wrong.  Those warnings are saying: “The URL you entered means that you’ve asked for a snoop-proof connection, so okay, your connection to this server is encrypted; however, I can’t verify that this server is who it’s claiming to be.”  Your conversation with the server will be private, but you could be subject to a so-called man-in-the-middle attack, whereby someone hijacks the local network segment you’re on and starts speaking on behalf of, say, bankofamerica.com.  The magic of the certificate authority system means that they can’t do this without generating a warning.

There is one caveat: if they simply don’t try to use encryption, the warning won’t be generated.  This is one of the reasons why you’re supposed to check for https:// in the URL whenever you submit sensitive information; not just so that your password isn’t available to everyone else on the Bolt Bus (though that’s a good reason, too), but also so that anyone pretending to be a server they’re not will get caught.  Unfortunately, people aren’t very good at checking for that little s in the URL or that little key icon or whatever other little security indicator your browser provides, so the bad guys just direct their victims to unsecured sites.

That’s too bad, but it isn’t a sign that warnings about unsigned keys are bad ideas.  Actually, the fact that phishers have drifted away from that link in the security chain means that it’s working properly.  Weakening it isn’t any kind of solution.  MS security researchers would be better-served by spending more time thinking about how to get people to notice when they ought to be using a secure site.

The American Prospect on Trivium!  Worlds colliding!  It’s great!

I read the article, though, and found myself disagreeing with its author, Marisa Meltzer.  How can you provide an accounting of Tumblr’s success and not use the word “Gawker” once?  The service was launched in New York in the right kind of scene, was pitched in the right kind of way, attracted the right kind of writers, and grew from there. Like so many successful products, you can’t simply run Tumblr’s featureset backward through the perfectly-deterministic mechanism of the imagined market and arrive at a proper accounting of why it succeeded and its competitors failed.

I agree that it’s worth talking about the subtle mechanics of the site — how there’s a thumbs-up mechanism but no thumbs-down; I think the consolidation of reader and blog into a single form is a genuinely interesting idea. But Meltzer’s account seemed to me a bit like explaining the success of the iPod via a discussion of its clickwheel.

Kriston and I built Lady Ada’s Game of Life kit over at HacDC a while back, and it roused the interest of a few folks. Sommer asked that I be on the lookout for another kit that might be a good candidate for learning to solder. I eventually suggested another Lady Ada kit: the MintyBoost, a simple circuit that lets you top up your USB gadgets’ batteries through the use of a pair of AAs. By the time I got my act together, six(!) people had expressed interest in building the kit, pushing supplies of my tools (somewhat scarce) and electronics knowledge (decidedly meager) to their limits.

But it went surprisingly well! Everyone’s kit worked immediately; in fact, they even seemed to work with the iPhone 3GSes that were on hand — something that the MintyBoost can’t necessarily be relied upon to do.

I think that my favorite part was watching everyone mess around with the Dremel. That’s some advanced nerdery for you.

I think that folks had fun; we might do this again, perhaps with more of an Arduino bent. If you want in, let me know.

I made my first EAGLE schematic!  I’m sure it’s horribly broken, but I’m still feeling pretty good about it.

If you’re interested in getting the source file, head over to the post on Sunlight Labs.  And if you do, please be gentle.  Advice on what I’ve gotten wrong would be welcome, though.

ALSO: Thanks to the diligent outreach efforts of Sunlight’s own Nicko Margolies, the original post has now been picked up by Hack-a-Day and MAKE. Neat! And better still, educational: I’ve already had a number of revisions suggested to me by the Hack-a-Day commenters.

ALSO ALSO: Engadget, too! Though I’ve gotta say: man are the commenters there morons.

The wages of internet success, I suppose.

AAAAAND: Gizmodo. That’ll just about do it, I think. Their commenters are mean. Which I like. I want the apparent EE to offer some more information, though.  I think he’s at least partly wrong.

Earlier this week my old boss, JP, sent me a note saying that the full-text RSS script I’d written was being shut down.  EchoDitto was nice enough to continue hosting it on their servers, but it had been consuming an increasing share of resources, and finally it needed to be killed.  Sad.

Then, horrifying.  JP had mentioned it idly, but until I saw the Google Alert for this I didn’t quite realize what had been going on.  The self-described black-hat search engine optimization crowd — the folks who assemble sites peppered with ads that are designed to attract search engine traffic, aka “link farms” — had been using my script to steal other people’s content and republish it on their own sites.  Using this sort of genuine content helped them snag traffic more effectively than they could with the gibberish that you sometimes see in spam, so they were sorry to see the script go.

Well, I appreciate the attention, but it’s not exactly what I had in mind for my work when I released it.  So I explained the situation, and bid them adieu:

I was at least somewhat aware of this danger, and consequently didn’t open-source the code (thank goodness).  It was still a bit of a shock to see the reality of what I’d wrought, though.

I still believe what I wrote in that initial post: it’s pointless for online publishers to try to control how their readers consume content.  If you wish to publish digitally, you need to accept the realities that come with doing so.  Pretending otherwise is just going to inconvenience your readers and slow your business’s necessary evolution.  Prolonging that process seems likely to increase the painfulness of the process, not eliminate it.

For those curious, the algorithm was exactly what many in comments had guessed.  I used some regular expressions to build a hierarchical structure representing all the <div> and <td> elements in each page associated with an RSS item.  These tags are the ones most often used to provide a page’s layout, and the full text of an entry can usually be found in a single instance of such a container.  I then traversed this structure, looking for the text excerpt from the original RSS item.  Once found, the rest of that container’s contents could be pulled out.  It’s a simple idea, though the realities of HTML — and the difficulty of preserving byte offsets between a sanitized working copy and the original — made the actual implementation require quite a bit more cleverness (and caching) than it may sound like.

The result worked pretty well. Still, there were a few problems with the approach.  For one thing, comments were frequently included in the same container as the main entry.  For another, the script would fail if the RSS entry text was a summary of the item rather than an excerpt. I think that both of these are surmountable problems: a better approach would examine the “textiness” of each container using a variety of scoring metrics.  Something similar could be used for detecting the start of comments (which tend to be peppered with timestamps, quotations of the original text, and occur after a big <h[1-6]> containing the word “comment”).  I took a stab at a new, Pythonic implementation using Adrian Holovaty’s templatemaker and a few other tools, but a lack of immediate success (and much higher computational demands than the original script) made the project fall by the wayside.  Now that I know how it might be used, I’m even less likely to pick it back up.

But I’d still love to see my algorithm adapted by the people who make RSS readers, and would be happy to talk to any interested and qualified parties about making that happen.  Those SEO morons don’t appear to be particularly technically proficient — I’m not too worried about them managing to steal content via a client-side app (though certainly some thought should be given to the matter before giving the store away, say via Applescript).

Tim points out that Vint Cerf’s advocacy for network neutrality doesn’t mean that every venerable internet expert agrees with it.  True!  Google for “Dave Farber” or “Bob Kahn” and “net neutrality” and you’ll find plenty of articles… from 2007.

I think this debate has stagnated.  People are continuing to act as if the FCC’s plan is a complete unknown. This enables the lazy “evil corporation/incompetent government” frame that we all know and love. At one point this was at least semi-appropriate.  People — including those like Cerf, Farber and Kahn — made guesses about what was likely to happen, largely based on their own position along the regulation/markets ideological spectrum. “The government is trying to outlaw QoS!” “The ISPs are going to charge for Google!”  Both of these positions were hyperbolic.

But look, there’s a difference now: the FCC has announced its NN principles, to thundering… well, there hasn’t been much reaction, actually.  Everyone’s just pretending that nothing happened and sticking to their guns.  “The principles are too vague!” they cry, even though those principles say that allowances will be made for necessary network management, that throttling heavy users will be okay, that malware can be filtered, that copyright law can be enforced, and that managed services can be developed and sold.  That wipes out a significant swath of the ISPs’ objections.

Yet no one seems to have noticed. Here’s Farber responding to the new principles — he doesn’t particularly object to them, he just thinks they’re too vague and that they might make ISPs wary of trying to innovate (as if the research costs to network management are so great that the potential for work product to be shot down makes it impractical; please, this is software).  He also says they’ll lead to a bunch of lawsuits, though he simultaneously says we should instead rely on the DOJ and FTC to handle this (in a rule-free environment!), which makes just about no sense to me.

I digress.  Look: the FCC has put some of its cards on the table.  Not all!  Rulemaking still needs to occur (if it didn’t, opponents would be even more outraged).  But the agency has signaled that it’s not actually going to do the incredibly idiotic things that neutrality opponents claimed it would. Those people need to acknowledge this fact, and come up with a clarified set of objections. What is it, exactly, that the FCC’s stated plans are going to stop you from doing? Right now I’m just seeing a bunch of now-irrelevant FUD; some talk about how great a monopolistic IPTV market would be; and some hand-waving about “new business models”, which I take to be a synonym for “rent-seeking”.  It’s time to admit that the FCC isn’t the one being vague.

It’s hardly worth noting, but this is a lousy editorial.  A couple of points, one of which I buried in the last post about net neutrality:

• Comcast’s reversal of its decision to fiddle with Bittorrent seems to be getting bandied about as an example of the market taking care of things. People should be less sanguine: it was not market pressure from clients but rather PR pressure from activists that forced Comcast to back down.  And if the net neutrality issue hadn’t been looming at the time, the outcome could easily have been different.  Mustering that level of coordinated outrage is not a sustainable model for policing ISP behavior.  Other ISPs offer evidence of gloomier scenarios: in Canada, Rogers has been throttling Bittorrent — and, in fact, all encrypted traffic, since it can’t distinguish encrypted BT from encrypted anything-else — since at least 2005.  Similar difficulties face customers of some Australian ISPs.  And there are other policies that have been in place so long that people have stopped complaining: unless something has changed, I’m still prohibited from running a server for personal use off my home connection (in principle a server of any type; in practice, port 80 is the only one that will be caught). And it was years — years! — before Comcast disclosed or even acknowledged the existence of its secret bandwidth caps.  Neutrality opponents are mistaking a single victorious battle for the absence of a war.
• The Post piece refers to the broadband market as “a vibrant and well-functioning marketplace”. In truth, it’s stagnated: in North America, prices remain steady — at a rate above what much of the developed world pays — while ongoing improvements in speed show little hope for of catching up with the networks of countries significantly poorer than the US.  If the Post wants to argue that net neutrality will make the situation even worse, fine. But they don’t even seem to realize that by global standards our domestic broadband marketplace is an underperformer.

Surely any Redskins fan will agree that foremost among Dan Snyder’s sins is his selection of a broadcast outlet that only provides its audio stream in Flash.

See, I’m going to be heading back from Philly this Sunday, and many Sundays besides, and I’d rather not miss the whole goddamn football season.  If WTEM provided an MP3 stream this would be no problem — the FStream iPhone app would let me listen to the game on stretches of I-95 well beyond the anemic reach of 980 AM.

That’s not the case, though.  WTEM uses a Flash-based streaming system, apparently through a vendor called streamtheworld.com.  That’s fine for the web browser, but the iPhone’s too stupid to deal with Flash.

So I set out to provide an alternate stream.  Amazon offers on-demand virtual servers through its EC2 service, and my first thought was to fire one of those up for each game.  At ten cents an hour it would be affordable, and I could use the more-competent server to transcode the Flash stream into a more iPhone-compatible MP3 stream.

I got far enough along this path that I might as well share my work.  Here’s a quick recipe for setting up a VLC-transcoding-capable server on EC2:

1. Launch the following Ubuntu AMI: ami-ed46a784
2. Make sure that you’ve opened port 8080 in the relevant security settings.
3. Execute these commands:
   

   apt-get -y update
   apt-get -y install libmp3lame-dev
   apt-get -y install ffmpeg libavcodec-unstripped-52
   apt-get -y install vlc vlc-plugin-esd mozilla-plugin-vlc

4. Execute something similar to, but not quite this:
   

   sudo -u nobody vlc -vvv “http://208.80.52.80/WTEMAM” –sout ‘#transcode{acodec=mp3,ab=64}:standard{access=http,mux=asf,dst=}’

This is close.  It’s really close.  If you run this and then load up VLC — the amazingly useful media player — and connect to the AMI on port 8080, you’ll get a 64 kbps transcoded MP3 stream of the WTEM flash audio stream.  It works well!

Unfortunately, I don’t know the specific incantation that makes this MP3 stream compatible with FStream on the iPhone.  I tried various playlist formats.  I tried various audio formats.  I tried various container formats.  There doesn’t seem to be any decent documentation on the internet telling me what to do.  It just says it’s connecting, forever.

But while searching for a solution I discovered something interesting: there’s an iPhone version of VLC!  True, it’s 13 megs, and yeah, it’s only available on jailbroken phones (via the Cydia package manager).  But it works great!  I pointed it at http://208.80.52.80/WTEMAM and it started spitting out incomprehensible sports radio babble almost immediately.  Admittedly, that was over wifi.  But I see no reason to expect it to perform any worse over 3G than an equivalent MP3 stream would.  I’ll be giving it a shot on Sunday during the Giants game. And while I’m sorry to not be providing an MP3 stream for other fans, I’m glad to offer another reason to jailbreak your phone.

Tim wades in to the debate about robots replacing low-skilled workers. I don’t think I made my case as strongly as I could have, and consequently I can’t resist responding, if only briefly.

1. I didn’t react to the “cost of materials and energy” argument as forcefully as I should have. I really, really, really don’t think this is going to be an issue. Machines are much more efficient than humans — that’s sort of the whole point. It’s not at all a subtle difference, but rather a massive leap (they don’t call the first sweeping wave of automation a revolution for nothing). The price of technology continues to fall. And the energy costs we’re talking about are slight: keeping a Roomba charged takes about a third as much power as illuminating the lamp on your nightstand — and that’s if the lamp is using a compact fluorescent bulb.
2. Tim’s discussion of our hardwired preference for other humans is an interesting one. But I’m not sure this is a problem that will be as resistant to engineering as he thinks. Consider Paro, the robot seal. By all accounts he works pretty well at soothing nursing home patients. Tim’s probably right that people would prefer — and pay for — a human R.N. rather than Rosie the Robot in nursing scrubs. But if a suite of relatively simple assistive technologies is cheaper than either — partially automated showers, biomonitors, pill reminders and the like — they’ll probably settle for that.

Human labor remaining competitive with robots seems to assume that one of two things will happen.

First, wages for unskilled labor could be pushed even further down. Presumably this is supposed to happen as average income continues to rise. I’m not sure how this can be considered feasible, either practically or ethically — we’re basically talking about reinventing the serf.

Second, robots could become dramatically less economical due to some sort of resource or energy shock. But given robotic efficiency (and the reality of their still-falling prices), it’s hard for me to envision this happening except in the sort of catastrophic scenario in which maintaining human employment levels would be the least of our worries.