I can almost always find something useful in a most popular list. For a long time I thought, if the list is properly regulated it can provide all that you would ever need when searching for content, but as time passed I found more evidence to the contrary. Now curated streams of content are turning out to be way better at giving me content I actually want to consume. Popularity lists, can be a great tool, and for a long time I thought it was a panacea of getting the best content. If a list is properly regulated, and is a minority list. Such as top 50 indie bands, versus top 50 all around artists, it was useful to me. I found lots of great stuff using most popular lists. But, now that I have been building a system for curation, I am slowly realizing, that they aren’t the best system, curated streams of content are the best.

Take music as an example. I like popular music, music off-the-beaten path is nice to. It would be a bad idea for me to say that any label placed on music, external, or otherwise, is what make the music interesting to me. What matters more is if I like it or not, I for one, can’t keep up with labels, but my taste in music will always be able to tell me if I like something or not. This is why top 50 lists worked for such a long time, labels didn’t really matter, I just new that I would like some of it. I seem to have an ear that likes popular music. I like that it changes, I like the narrative of pop music. People who know me think I am kind of a sap, for liking popular music, that’s fine. But this has led to the fact that popularity lists have always been a good starting point for me when it comes to finding new music that I like.

While I can get by with the billboard top 50, it doesn’t churn enough for my liking, so as music blogs rose, I relished sites like hypem.com that tracked the most popular music on blogs. I have been using the hype machine for years now. Recently something grabbed my attention, whenever you “love” a song they show you other users who “loved” that song. They have had this feature for awhile now that allows you to follow other users, but for whatever reason I didn’t really use it. Until they showed me inline the other users who loved songs. I started following users who had loved the same songs that I did. Now, even with a very small network, like 5 or less, I am really enjoying the stream of songs. On average, I like more songs from the stream then I do from the most popular list.

I still listen to the popular list, but I think it only works well because the hype machine limits the blogs it follows to a subsection of the total blogs.

Now, as I look to the future, and see video coming down the pike. We need services to help us slice, and dice video content. One such site is nowmov, they show you the most popular videos on youtube. It just goes from one to the next, like TV, wink wink. Its good, and there are probably many people who are looking for this type of experience, one like TV. But, this is just a popularity list, they will probably build in some better algorithms, and it might get much better, like pandora for video. But, I am going to bet that curated streams will beat nowmov, on a personal level, it’s not a zerosum game.

**Update 1/30/11: **They guys over at emedly who have done some awesome work in the last few months now ship ruby-specific gems to help with oEmbed. Check it out https://rubygems.org/gems/embedly Wacchen, trys to figure out what the best way to embed a users selected video into the web page is. Right now the best way I have is oembed. oembed is a great, simple standard for determing the best embed method for a give url. You just ping the service provider, ie youtube.com, with a url, and it will tell you the best way to embed the content from that url. There is alot of good python libs, and there are a few ruby libs. - oembed_links - based on the idea, that you will want to transform links inside text - OhEmbedr - ruby-oembed

I started out with OhEmbedr, but quickly I realized the downfall of all of the libs was that they had relativly few built in supported service providers. Even the fallback mechanism, http://oohembed.com/, was running into instance limits at google app engine. A couple weeks ago though, I remembered seeing that django-oembed, had been pumped full of new urls from Embedly, a new sort of meta oembed. Hoping they had done the same thing for ruby, I shot them an email, but no dice, as Sean Creeley said, “we are a python shop”, So, embedly didn’t have was a good ruby implementation, so I knew I had to do it my self. I forked a copy of ruby-oembed. And essentially, I took an awesome JSON file from embedly, that lists a bunch of simple regular expression urls they support. I was able to quickly load that file into ruby-oembed, and then I added some more tests, while not good tests, but tests. So, even though Embedly didn’t have a ruby lib, everytime I had a question Sean, was very quick with an email back. I am not entirely sure why Embedly is so cool, they don’t seem to do anything right now other then help people embed stuff, and that is awesome they do it really well, but they must have something super secret under the hood. Now, wacchen, has support for like 20+ videos providers out of the box. Embedly was a huge help. Next up, figuring out how to embed videos, found not on the original webpages. Like youtube videos, embedded in blog posts.

Rails like I said is super magical, and so far building websites has been an upfront, and forward kind of thing. Its nice to know that the ideas I have been working on, models, views, templates. routes, tasks. Are transferable from one framework to the next. I think there truly is a web vernacular when it comes to building websites. This helps all of use to have a common language.

If you just want to check out the code you can find it at github http://github.com/voidfiles/Watch-It-Latter, while I am calling my web site Wacchen, I am going to call the open source project Watch It Later. I am going to put a MIT, or BSD style license on it.

So far, I have been developing on a cool stack.

I host at Dreamhost, lots of people hate Dreamhost, but to me its like I spend 7 bucks a month and I get what?, thats awesome. I have always been a little disappointed that they don’t support things like django/python better, and now I know where all there time has been spent; on ruby.

Using Dreamhost I am able to get up 70 request per second, while maintain a sub-second page load time. Thats pretty good for Dreamhost. When setting up Drupal on Dreamhost, it takes something like Boost, a static page cache, to get anywhere near 70 rps

But, there are always differences, sometimes good, sometimes different, and sometimes bad. I haven’t run into anything that is bad, like makes me shudder bad, but things are definitely different from Django

I am using devise / warden for handling authentication. Super useful, easy to customize to a point. There views generator is broken I think, and I am not sure if rails, like Django allows you to overide just the template portion of an addin.

For embedding the videos I use oembed, there are a couple oembed libraries for ruby. Right now I am using ohembedr, but I think I am going to use something else in the end, what I really want is someone to come along and just setup embedly mappings in an existing library, so I won’t have to, but I think in the end I will probably have to do the leg work my self on this one.

Routes

Routes aren’t much different from Djangos url dispatcher, if anything rails routes has more functionality. That is my problem though, I don’t feel like there is a consistent API for routes. I have the basics down, but I am not sure I have more then that. Of course, this is a learn more thing, but my understand of Djangos routes, has been almost the same since I started using Django. They have a small api for urls, so it’s easy to get handle on all of its complexities quickly.

Deployment, and Capistrano

Deployment is really easy on rails, its part of the magic I feel. I especially like the idea, of having a public folder. Django, being a much more libertarian framework kind of leaves that all up to you. I am slightly torn on this one, its one more place for a newbie to screw up, but its one more way that Django doesn’t shepherd you into any specific way of doing things.

I think this idea is missing from Django, because there was no plan in the begging to ever server static files from the application server. The Problem being that, that is exactly what a lot of newbies do, just to get things working.

The ability to pull in all the dependencies into the source tree is helpful. So far this has been like the inverse of django. Python has lots of great libraries, but its not clear right off the bat how to use things like virtual_env, and pip to make your work flow easier.

Capistrano, for some reason getting it to work with python not so easy. Tried Fabric. Disclaimer, tried this multiple times before, I think each time it gets easier, this time may have just been the breaking point where it all made sense.

Working In Public

Still not entirely happy with the work flow of creating open source and then deploying to a private server, just so many possible gotchas. There has to be a better way then maintaining two completely independent source trees maybe I could do it with branches. Have like a public branch, and then a private release branch.

So far, I just manage this with gitignoreing files, that have sensitive data in them, but then I need to create template files for the ones that are missing in source tree.

I also had to add some extra hooks for deploying these private files, using Capistrano because the files aren’t where Capistrano expects them.

If you have never dealt with pythons awesome feedparser, Universal Feed Parser, you should check it out. This bad boy can take some of the crapiest, most mal-formed feeds atom, or rss, and give you back manicured, well formed python objects. I have used the project many times over, and am indebted.

There current security model is so tight that it removes all object, and embed tags. This is very secure, and for most people probably a good idea. But, while working on a recent project, I ran smack into this, because I am processing feeds with videos in them, and the videos are the most important part of the feeds. So, I needed to allow object, and embed tags

I thought mayby I would have to maintain my own version of feedparser, but that just had many problems written all over it. Then I remembered that python, being the awesome language that it is, allows you to monkey patch classes. It works because everything is an object, including classes.

This is my quick tip feedparser tip if you need to extend feedparser to include object, and embed tags

import feedparser\ feedparser._HTMLSanitizer.acceptable_elements = feedparser._HTMLSanitizer.acceptable_elements + [“object”, “embed”]

Thats it now when you kick off a feed parser object it will whitelist object, and embed tags

Ruby is curious, playful, and eccentric. These are all adjectives I would use to describe my daughter. In general I get a real sense of discovery while learning to use ruby. Things like its for loop

@names.each do |name|
    puts "Hello #{name}!"
end

I just haven’t seen a for loop like this before. Its cool.

Then other things like checking for things with a question mark

if @names.nil?
    puts "..."
elsif @names.respond_to?("join")
    ...
end

I am not kidding, when I saw that question mark syntax, the first thought in my head was how my daughter asks questions.

The other day, my wife went to the doctors. When my daughter woke from her nap, she wanted to know where her mom was. I told her she was at the doctors. She looked at me, leaned her head forward, and said “Mommy, Doctors?” (think little kid voice though). As she neared the end of the sentence, her eyes widened and her voice went up. It’s an exaggerated way of asking a question, but all the key elements are there.

This is what Ruby does too. We all have code that does boolean, true, false assertions, and they are kind of boring like. user.is_authenticated, all we are looking for is a yes, no. But, in ruby you know that its a question. Its like they have humanized code in some regard.

These aren’t new. I think there is a lot of mutual respect between Rails, and Django. On more then one occasion I have seen Jacob Moss give kudos to some part of rails.

I never thought there was much of a difference. I thought ah, cool, yea if I ever had to I could pickup rails. While, its not been as easy as picking it up, it has been easier then going from QBasic, to assembly.

In a funny turn of events, I feel as if I have made a huge clerical error in not at the bare minimum, jumping in. It left a hole as big as a jumbo jet open in my iron curtain of knowledge. So, to correct that, I have decided to go for a swim. I am going to post my code up on github soon, and the site will be up soon too. I am not sure what I want to call it yet.

Basically the idea, is that I want an instavideo, see instapaper for more context. I think instapaper is just a kick ass way to consume longer reads, and it gives me warm fuzzies inside. When I talk to other people who like instapaper, you can see there eyes light up. Using instapaper in some sense means you are interested in reading, and not just casually. This helps to buffer a larger point about people on the internet still do want to read, but I want to talk about video. Video, right now is as frantic as blogs were a couple years ago.

I feel the video world is ready to start seeing well crafted channels of video coming out over the web. There is boxee, kicking the pants off all its competitors, when its not drinking to much and coming into work late. The iPad is has all the big players, rushing html5 video out the door. So, video, is getting ready to bust out, and I need a complement to instapaper.

So, I am building in ruby, its a twofer. Build something I want, and learn something new in the process.

A friend of mine once complained on twitter about how bad the Django Documentation was. I thought he was crazy, but now the script is flipped I wonder if its just a cultural thing. I am going to find out, but right now, the documentation sucks. I am finding hard to dig deep into how the magic works.

That’s another thing, I feel in some sense that Rails, just has a ton of magic baked in. I am assuming that a lot of this magic is due to the power or ruby to be so expressive. I think I just am waiting for that paradigm shift moment, where all of a sudden the light bulb goes off.

Other then that, I am liking the migrations out of the box. Django didn’t have a strong sense for this, things like south came along, but I did what all the other developers did and just wrote my own SQL migrations. When you sort of force the user to think about migrations, then migrations make more sense. Then the notion of rollback becomes as simple as issuing a command. I really like this idea. I hope I don’t find away around migrations, and make sure I flex that muscel as much as possible.

Especially when you start looking at continuous deployment, which is something I am looking at, making sure you can roll everything back becomes super important.

For those of you who came to my presentation. I just realized that I talked about a few thing that won’t be in my content, and I wanted to provide you guys with some links.

• the cluetrain manifesto
• Louis Gray - great inside baseball kind of blog about social media
• daringfireball.net - John Gruber great little blog, covers a lot of mac news related stuff.
• kottke.org - a liberal arts blog. Quite possible the best curation blog out their.
• instapaper - a great way to consume longer form written pieces. Also and offline iPhone, and iPad app.
• TweetChat - Was brought up while we were talking, whoever suggested this, if you get in contact me I would like to give you credit.

I am at the Drupal core developers summit right now. Lots of smart people in one room. I though this would be a great way to get into the mood to present on Monday, and hopefully meet some people before I actually present.

Its super real now, I am in the program.

Just after I finished installing mysql on my mac book pro, for the umpteenth time. I realized quite possibly my favorite thing about Django is the built in dev server, mixed with sqlite. This combination allows you to go from idea, to mock in like 15 minutes. Drupal just doesn’t have that kind of speed when people just want to test some ideas out.

So, I have done some initial investigation, there is a php webserver out there, but I haven’t figured out how I can just hook it in. Anyway here is my deck, Google doesn’t allow me to embed my font, chunk, so its just whatever was stock.

Photo Credit: Daniel Gies