Last night I made a progressively enhanced dinner. In deference to my vegan, and vegetarian friends I don’t think I would call my dinner “enhanced”. I don’t think it means that my meal was better then theirs. I guess it was a progressively differentiated meal, but the idea’s of Progressive Enhancement were core to my meal making.

It started with pasta. I have been making pasta, and my own pasta sauce for a number of years now. I even created a video once that covers the basics, I really haven’t come that far from this video, although I have learned a weird beard is a bad idea. I have a few new tricks up my sleeve, but mostly around adaptability.

My the pasta, is always a semolina, but I have a friend who is sensitive to gluten. They brought their own rice based pasta. That is where its starts though, by having two pastas, I have already created two branching points.

The most Spartan meal that could be main would be the cooked, unsauced pasta, and it would be good. That I believe is the core of the meal. Though, for more advanced people, I support a more rich experience.

For those picky people, who dislike mushrooms, I made a mushroom, onion, and garlic thingy separate from everything else, that was vegan. The mushroom thingy could be added to any pasta bowl, by it’s very nature.

Other then the mushroom thingy, I made two base sauces. One was a fully vegan tomato suace made from canned whole tomatoes, and another sauce was a sausage based sauce also made from canned whole tomatoes.

Both sauces start the same. Browning some onions, and garlic in a pan with some form of fat, olive oil for vegans, a little butter for non-vegans. Then I added the sausage in the non-vegan pasta, cook until browned. At this point both pots have built a pretty good fond. Naturally I added some liquid to absorb the fond into the sauce. Since a couple of the vegans are also straight edge I try not to cook with alcohol. I deglaze the vegan sauce with strained tomato juice. The non-vegan sauce I almost always just throw in a bit of whatever I am drinking, at least if it’s close to simple. No Guinness, or wine spritzers, but it could be beer or wine, it doesn’t matter.

At this stage, I have let the tomato juice boil down a bit, and then throw in the whole tomatoes and break them apart a little, and then let the sauced stew on a low heat for 20 minutes, or so.

At this point you can build many dishes, I think this is a factorial operation. I have 5 basic ingrediants(semolina pasta, rice pasta, mushrooms, vegan sauce, sausage sauce). 5 factorial is a 120 sepereate dishes. From simple pasta, to overly sauced rice-pasta, mixed with semolina pasta, that is for adavanced people. Just like only advanced browsers can support advanced web page building tasks.

I have spent the last 3 years, or so, working at Yahoo!, you must raise your voice at the end of the hoo, and it has been awesome. The opportunities to learn, and grow were huge, but it’s time for me to move on. In any organization you can pick it apart, and I have, but my move isn’t about that. If anything, it’s about getting a variety of experience.

That is why I am proud to say I am going to an awesome little company called Mixed Media Labs, and their product Picplz (more info here). I am excited to apply what I have learned from building large websites, for millions of people, to building fast, sleek mobile apps. It doesn’t hurt that the entire company sits in one space. I can’t even imagine how refreshing that will be.

Thank’s Yahoo! don’t let them keep you down.

For some reason I was afraid of the Twitter Streaming API, but it turns out the streaming API is super simple. If you have ever wanted to implement a twitter search bot, or just wanted to play around with a large amount of twitter search data, or statuses the streaming API is the way to go.

The Streaming API is a little tricky, but ultimately easy. You are just issuing a request to twitter, which doesn’t close for a long time. Over the length of the request Twitter will continue you pass data down the pipeline.

This is a little tricky to understand at first, it’s actually quite counter to how a lot of programing is done. It’s event/loop based, versus sequential. If you are used to how things work in Javascript this actually might come in handy.

As a matter of fact an environment like NodeJS does a really great job at handling the streaming API. In less then 10 minutes I was able to understand how the streaming API works because of twitter-node. With code like the following you get notified of every new status from group of people.

var TwitterNode = require('twitter-node').TwitterNode, 
sys = require('sys') 
twit = new TwitterNode({ 
    user: 'username', 
    password: 'password', 
    follow: [8038312,40289924,68938254] 
});
twit.addListener(
    'tweet', 
    function(tweet) {
        sys.puts("@" + tweet.user.screen_name + ": " + tweet.text);
    }
);

Once you run this code, and then wait for a while, you will start to see tweets pop on the screen as they come down the pipeline.

I am really interested in using Node more, but I had already created a project in Django and I wasn’t interested in writing SQL to talk to my mysql database to store all the new data I was going to get from the Streaming API. This is a lossy version of what I did, but it’s close enough to give you a handle on whats going on. Tweepy has support for the streaming API, but they don’t have the docs yet so I had to dig through there code.

from tweepy.streaming import StreamListener, Stream
import simplejson
from datetime import datetime
import time
import locale

# Parses twitter dates stored in json # from twitter-python
def parse_datetime(string):
    # Set locale for date parsing
    locale.setlocale(locale.LC_TIME, 'C')

    # We must parse datetime this way to work in python 2.4
    date = datetime(*(time.strptime(string, '%a %b %d %H:%M:%S +0000 %Y')[0:6]))

    # Reset locale back to the default setting
    locale.setlocale(locale.LC_TIME, '')
    return date


# You need to subclass the StreamListener
class MyStreamListener(StreamListener):
    """docstring for MyStreamListener"""

    def on_data(self, data):
        print "starting on data call"
        data = simplejson.loads(data)
        return True


    def on_timeout(self):
        print "we got a time out"

    def on_error(self, status_code):
        print "we got an error %s" % (status_code)
        return False

mylisten = MyStreamListener()
mystream = Stream("voidfiles","hacker",mylisten,timeout=30)
mystream.filter(follow=[])

With something like that code I was able to save tweets that were from a group of twitterers. Because it’s python I was able use django’s ORM to store the tweets as they came.

I have been really impressed with the coverage of the javascript community over at DailyJS. So, when they were asking for guest bloggers I jumped at the chance. I chose to write about an idea that has started to surface which is mobile data sync. Sync can be tricky especially when you mix in the fact that mobile devices can be offline for long periods of time. I tried to find a solution, but couldn’t find anything. That is why I wrote the article I wanted to see if could kick up some dust and create a conversation. Check it out.

I just rewrote my web site Wacchen in python with Django. I converted it from a rails project. It’s easy to say something like rails sucks, but clearly people have been very successful while using rails, so I am not going to say that. I am going to say that I love Django, and the process of building my project from the ground up in Django was fun. But, rails ain’t all bad, they do have a cousin called Capistrano.

Capistrano isn’t actively developed anymore, but it still kicks ass. It helps you build a one line deployment script, which is really-really important. For years, I would start to build a script like that, and then at some point I was like, fuck it, I can do it later. What I didn’t realize was that I was throwing one of the best getting shit done tools out of the window. Once you taste one line deploys, a lot of mental blockage fades away, and you can iterate faster. I had never realize how powerful the thought of trying to deploy was. It was actually stoping me from want to hack. Now though, I converted to python land, and I needed something like Capistrano. Django doesn’t have as tight a relationship with Fabric as Rails does with Capistrano, but it’s been mentioned in more then one place.

• Fabric, Django, Git, Apache, mod_wsgi, virtualenv and pip deployment
• Deploying Django with Fabric

That is what I chose. I started with the above, and an assortment of blogs posts, but fabric has changed a lot since many of these posts were written. For instance, it’s not config anymore it’s env. Also you can’t use the fancy text replacer syntax any longer. This run('cd $(path); mv releases/current releases/_previous;') now becomes this run(‘cd %(path)s; mv releases/current releases/_previous;’ % env)`.

I am glad to say that it works, and it’s been quite plesant. I have learned a couple things along the way that I can share though.

There is no package deal

Borrow, cheat, look around, but you aren’t going to be able to use someone elses script verbatim, deployment is personal. This was a huge stumbling block for my self in the beginning, but like I said capistrano was like crack, so I got myself over the hump.

Use git

The ability to have git just dump a tar of your current directory is super useful. Then all you need to do is take that tarball and put it on the server. This was the extent of my putting.

This get’s me using git too, which is a nice added bonus. I use git, but I often forget at the very beginning to use it.

Use your fabfile as a scratch pad

Because there is no cannon for fab files, and you can’t just use someone else’s I found it best to create new deployment steps as I went. I would ssh to the server, mess with the command until it was right, and then immediately write it into my fabfile.

Chunk it up

Don’t try and write big fab targets, use lot’s of little ones, that way as your deployments get longer, you can start to target smaller portions of the deployment. I have an app server, and celery server, and I didn’t want to deploy too both servers all the time.

Those are my quick notes for now. I also ended up using a lot of other new tools like gunicorn, nginx, dreamhost ps, celery, and rabbitMQ, all of which have their own set of notes.

If you haven’t checked out anything from Jason Santa Maria, go check his blog out. He has been around for a long time, I think. The fact that I don’t know much about him is what impresses me the most. I have read a few of his blog post, but each one has been incisive, and powerful so that they stand on their own. I don’t need to know who he is to understand the he gets it. He understands the types of conversations that need to happen in order to advance the web as a medium.

I loved watching this video with him speaking in it, it’s been sitting in my queue for quite sometime, and finally tonight, screaming baby in tow, I took the time to watch it.

Its quick like 12 minutes, but he breaks down an important discussion. Why isn’t the visual design of print get translated online. He gave two good examples from Wired Magazine. He is right, the magazine layouts were beautiful, and the web versions were not. Not only that but they were robbed of there visual interesting-ness. He goes on to show how on his site he is attempting the ability to give each piece, I hazard to call them blog posts because they are so stunning, a distinct visual layout. He is succeeding in his goal to bring visual identity for individual pieces to the web.

This is where the discussion divides. Websites, unlike print, can be consumed in so many different manners I think unfair to attempt to give the digital form style in the first place. It’s worthwhile if it matters to yourself, but how can you translate the visual identity into a blog reader, or across twitter, or in pull quotes on other peoples websites. The web is in effect formless.

Twitter is a great example, each tweet has a permalink, a webpage that displays just that tweet, but is that it’s form? I don’t think so, tweets are more often then not consumed someplace else. I believe this has the effect of disinter-mediating tweets from form.

Medium as the thrust of the discussion, is why I am so excited about the video. In talking about webpages as a collection of information he contrasted them with physical collections of information. A magazine is a stack of articles. A magazine has a defined form, it’s only as big as it. Our brains can look at that stack and gauage how much information is in there. All this is being lost in digital translation. There is an answer though.

This being a medium discussion means that we can talk about how things can’t be translated from one medium to another. The fact that the web is formless frees us from trying to form it physically, but that doesn’t mean we can’t collect it. We can still create windows into the massive amount of information, but it is all done through other people, through curators.

Jason Kottke is a curator. He has a website, which has it’s own formless stream of content like all others, but I know this one. I know that I will get x number of posts a day, generally, I know what size it will be. I know that I will be delighted by a large amount of it. All these are starting to look like the very things we get from books, and magazines. The people around us will provide the form for our content, and the window in the massless, formless pile of content.

Design will still be a huge part of the web though, we still need visually interesting stuff, and enjoy how layout can change the meaning of a story, I just don’t see it being huge past helping people read the content better.

HTML5 is a loaded term that covers a range of ideas, but here is an attempt to explain it. Explicitly, all that HTML5 stands for is the next version of HTML. After this version there will be no more versions of HTML. Instead they will evolve the spec slowly over time instead of using major revisions. The next gold standard was going to be XHTML2, but it got dumped because it was built on an unrealistic view of the web. The view was that the web could be a place that used perfect syntax if only you forced it to. A bunch of standards makers broke away from the W3C, and created the WHATWG which gave birth to, among other specs, HTML5. It’s goal was the re-think how to derive standards, and it chose to look at how the web worked currently, versus what it could be. For that reason HTML5 has a lot of elements that we have today, but they have been codified some how. One example is progress bars, currently they are implemented as a hack, in HTML5 there is a progress bar element.

Besides the standards way of looking at HTML5, it is also an umbrella term for the next evolution of the web: Web 2.0 meets HTML5. Besides HTML, the markup language, HTML5 is a movement that represents all that is new in Javascript, and CSS3. It even encompasses new distribution channels like TV, and mobile. HTML5 is more of a platform, and less of a presentation language.

The bottom line is, HTML5 is going to be broadly accepted, and broadly distributed. It’s time for web developers to cheer, and it should make developers of other platforms weak in the knee’s. If a device has a HTML rendering engine it’s going to have HTML5: computers, phones, tablets, TV’s, and, yep, even cars. Some may dismiss HTML5 as a toy, but they are being shortsighted. For every nearsighted prognosticator there are a dozen web developers, who don’t care about the platform wars, and instead just want to make apps. HTML5 is the logical next step for them. Even billion dollar companies are gearing up to invest big in HTML5. Wherever you fit in your organization, lone wolf, or CEO, HTML5 is what’s next.

I don’t know about you but at some point in time I had to start asking my self why am I working at Yahoo! I am still to this day filled with joy that I ever had the chance to work here in the first place. When I was 12 I can remember thinking how cool it would be to work in Silicon Valley, never once did I think that I could actually do it, and now on regular basis I drive past the Yahoo! sign on my way to work. That’s cool, and will always be cool.

Past the warm fuzzys I get working at Yahoo, there are some fun technical challenges. Getting to work on websites of this scale will always be something that is challenging. Learning how to get work done in a beaurocracy is also a good thing.P ast that, why do I work at Yahoo, over say a startup. I have looked around, investigated working at startup, I have even been offered positions in startups, but turned them down. The biggest reason has been job security. I have two kids, and a wife, and Yahoo! is seemingly enough secure, but that’s not enough for me.

When you think of the reasons why people leave Yahoo!, which is complete speculation on my part, it might be place, you may not like Sunnyvale, you may live far away. It might be pay, but more then likely it’s not. Maybe that chance to strike it big, possibly if you are a very early employee. But, I think it rest’s mostly on one thing, the work. The work of startups seems to be more fun, at least from my outside looking in perspective.

Startups scan the horizon of innovation, pick a problem and attack it. They find really smart people, give them a salary, and hope that those smart people can solve the problem. The other part of there innovation, is there ability to turn on a dime, if there main product fails, they start over on something else.

So these two things innovation, and agility are something that I personally admire about startups, but I am here at Yahoo.

What can I do about it.

Well, we need to take what is awesome about the startup, and put it in Yahoo, and this change needs to start with us, the employee’s of Yahoo. I can’t do it alone. And here is how I think we can start.

Well, hold on, I am low in the chain here, for technical Yahoo’s there is a 1 to 8 scale, 8 being the “best”, and I am a 2. So, I can speak for no one but my self. Okay on with it.

This is a suggestion, it is a couple ideas I think we can use as individuals to get better at what we do.

One last note, I say hacker, but I don’t mean it has to be code hacking, just hack at whatever you want. Get better at hacking, food, or movies, or whatever. The important part is that you don’t let the corp dictate your innovation, do it on your own time.

Innovation

What that hell is innovation? Is it something you can plan for, is it something you can work towards? I thin, yes, you can, but it can’t be a line, or a box. You need to sneak up on it, kind fool it with misdirection, play hard to get. There isn’t a straight line from here to innovation. My bet is that if you can think of one idea right now that is innovative, it’s not, no matter how great it is, because if it’s innovative today, and you haven’t built it, if so you are getting passed by. My theory is that you don’t aim at innovation, but that you fall backwards into it. The only way you can do that is if you build a place where it can happen. I think this “place” starts with “hacking”.

4 ideas, 4 people, 4 steps - How we can be the innovation at Yahoo

I have 4 ideas I want to share, all are related to “hacking”, practically, and metaphorically if you will. These are ideas that I think have been guiding people for a long time, they aren’t incredibly new, but now they just so happen to have famous people talking about them.

The Medium is the message

If you have known me for any length of time, then I have brought this dude up, Marshall_McLuhan. This dude blew my mind with one sentence, “the medium is the message”.

The story on yesterdays front page doesn’t matter, its that the frontpage of the news paper is important that matters, its matters that we have this thing called a news paper that matters.

Think about how big The Beatles are, think about how they got here. Now take away from our world newspaper, TV, radio, records, and telegraph for good measure. Where would the The Beatles be now. My guess, a really great band in Liverpool.

How the hell does this relate to us, what it tells us all is that the medium matters. If we aren’t breathing, and sleeping our medium then we won’t be able to understand what the medium means. I don’t think we are in short supply here, but the next time you think you are wasting time watching a youtube video, or using facebook, start asking your self why, what about this experience is awesome, why do I sit here? This also means that we need to start being the master of our domains, which means hacking. We need to take these mediums that we use, and that we are expected to build and use them, we don’t have to use them like we are told either, that isn’t the point, but the point is can you hack this. Being a hacker is more then just writing code. We need to be hackers.

Step 1: We need to be hackers

Charlene Li

Charlene Li wrote Open Leadership: How Social Technology Can Transform the Way You Lead, along with another book. Charlene Li, is talking right now about how to use open, and not just transparency open, like we get to know how the company is doing open, or our CEO talks to us on a platform at all hands, but open like letting people use social media with out restrictions, letting the wave wild inside companies, and how that can help companies get better. But that isn’t the idea I want to talk about here, she has this great example of a pyramid.

Her idea is that we all start down at the bottom as a consumer, and work our way up to producer, but you don’t just leap from consumer to producer, you slowly work your way up the pyramid. Maybe you start by curating letting other people know about awesome stuff, having a link blog it doesn’t matter. Then you start be come a meta contributor, making comments on other peoples posts, and building relationships with other people on there way up the pyramid, then finally you get to producer.

The pyramid matters, because producers are the ones who drive innovation. The people out there in the wild, driving the conversation are the ones who are innovating. We need to be those people.

Step 2: We need to be producers

step 2 on our way to innovation domination is to become producers. Being a “producer” in our field is being a hacker, someone who plays with technology, someone who can help sort through all the different options in a knowledgeable way. They get there because they are hacking, and sharing there ideas about hacking. This gives them credibility, and a platform.

Clay Shirky

Clay Shirky, is quite possibly my favorite thinker about the internet. This dude has written more papers that helped me to understand social media, then anyone else. To really understand why tags are awesome read his paper Ontology is Overrated: Categories, Links, and Tags

In his latest book Cognitive Surplus he investigates such things as LOL Cats, and this thing called Ushahidi, you can watch his TED video, and it will be more awesome then what I am going to say but, here is the baseline. We watch 200 billion hours a year of television, which boils down to 20 hours of televisions per person in america. It as much as a part-time job. The amount of time we watch TV has grown steadily since TV was invented, until now. Young people are the first generation to watch less TV then there parents. Question #1, what are they doing with their time. Well they are making LOL Cats of course, but really they are using Facebook, they are watching youtube videos, they are building this thing called Ushahidi, which is a crowd-sourced online collector for reports of violence, that originated in Kenya during the election unrest a couple years ago.

What are you doing during that time. I have tried to tame this down, because I don’t want people to feel like harassing them, but really think about what you are doing during this time. I have two kids at home, and after I cook a meal for my family, and play with my kids, I want to hack on something. Mostly it’s code, but now it’s writing, blogging, tweeting, and curation.I take that extra time I have and I hack, you can all hack. You have the time, we as 20th century americans have some of the most free time on the entire planet, we should harness that time to help us work on awesome projects.

Step 3: Use a piece of your free time to hack

Ira Glass

Ira Glass is the host of “This American Life”. In each episode they have a theme, and then they have certain number of acts all related to that theme. That’s it. Within those rules, they make one of the most well-informed, well-thought, provocative pieces of media today. It never fails to disappoint. Mr. Glass has spoken, and written outside of the show, and has created meta narrative about the making of the show. One of my favorite ideas of his is that no matter how good you are at something, it still takes a shear act of will to make something great. He believes there is the law of nature that everything wants to be mediocre. This is the reason so much of what we see, hear, and watch is mediocre, because it’s really hard to make something awesome, you have to really work hard.

Hacking isn’t always going to be awesome, we aren’t always going to end up with a piece that even works, but that is not the point, the point is that we need work hard, and fast to try and make something great. If you keep hacking, and you build projects faster, and you fail faster, you will start to understand what is good, and what isn’t. Not just from a technically stand-point but holistically understanding if what you are working on can lead towards to innovation.

Step 4: Iterate, and work: Harder, Better, Faster, Stronger

Honorary mention: Time

None of these steps can happen overnight. That is why I love the pyramid idea, it takes time to move up the pyramid, but at some point in time you have to head towards it if you want it. It’s like watching TV, and forgetting the remote control in the other room, you might just sit there for awhile, the channel may not be so bad, but eventually you will want to control what you watch, and you might be tired, and you have to talk your self into getting up 3 or 5 times, but once you do it you are going to be happier.

I started actively reading blogs 4 years ago. It took me a good two years before I started to want to share what was finding, another year before I was doing what I would call active curation. Really, trying hard on a daily basis to find some great stuff, and writing good headlines for the stuff I found. In the last 6 months too a year, I decided that I wanted to start writing, I wasn’t sure what about, but I took the time to put some stuff together, with out being paid, and found that I liked it, and over time opportunities started coming to me, I thought I might want to write a book, I had some ideas. Finally out of the blue I had the opportunity to work on a book proposal, while the first publisher turned me down, I found one eventually, and I am going to write a book. I didn’t, 4 years ago, tell my self that in 4 years you are going to write a book about HTML5 Apps, what I did was set myself up to fall into it.

I think that if we focused as a team, and as individuals we could start to make some noise, and with noise comes opportunity, so you know lets see what we can do.

OpenAppMkt showed up in my stream today, thanks to Daring Fireball. It’s a great idea, and well executed. They even support selling HTML5 Apps. One key feature is the ability to “route around” Apple.

Daring Fireball is a great blog, and one that I respect, but Gruber made this statement today, “ iOS’s support for mobile web apps — totally open, no gatekeeping — is by design.” (source).

All due respect, but this seems like basic misunderstanding of open. Gruber has argued forcefully that Apple has every right to rule the App Store with an iron fist, and I don’t disagree with that, but you can’t then also say that it is open.

What OpenAppMkt does is route around the App Store, not iOS. iOS has great HTML5 support, but that is not what is in question.

When you look back at history, the means of anything, production, communication, distribution has always been a bigger deal the the individual pieces.

Right now we are seeing the open shots being fired in the battle for control of App distribution.

OpenAppMkt is absolutely routing around Apple, and forcing a fight, that I totally support, about the control of the distribution channel.

here is the story

My favorite part besides the book was that I started to read it as if it was a dudes blog post about working in a 24 hour book shop. I thought the whole thing was real.

The story was based in San Francisco too, and I was going to go to the store and check it out. Near the end of the story though something happened that isn’t real, and I realized it was a short story, and not a blog post, although it was written in the style of a longish blog post.

This must be like a newish genre of literature, who knows maybe I am just talking out my ass, i am not a lit guy, but I like to read. The story included bits and pieces from the real world, and not just the whole world, but my world of computers. It was well well written as well.

The kicker is that it had a really great ending, and illustrated a major point about a current debate that is happening in society right now, the trend of books decreasing in numbers.

It was awesome, I suggest you give it a read if you have the time.