Doing a Pixel Ping without a server

Content is slippery stuff now days, everyone, and your grandma wants to know how to track it. Even more so now that content can meander all over the internet.

There are some technical hurdles to this problem. First, a classic way of keeping abreast of your stats is to use a javascript method. This works well on the source so, or on sites that you control the domain, but as content makes it’s way into the community. You can’t bring JS with you.

For this reason some people have turned to using small unobtrusive images, that allow them some insight into where there content is going. A while ago a nice piece of content tracking was released, by a fledgling news org.

There solution is great, but you need to own a server, but there is another way, and you don’t need a server. Mixpanel is a new player, there whole company is about tracking data. If you dig through there docs you can find that they offer this “pixel ping”, as apart of there API.

If I wanted to do this for say, this article I would start by signing up, and getting an account. Then you get a token. Mine is, 15de388dc6d39118b914db428a8975de, you will see what sharing this with you isn’t a bad thing in a moment. Our goal is to create a JSON string, and the base64 encode it. What I want is to track a content-view. This is incredibly inaccurate, but gives you some amount trackiness. Here is my string.

{
    "event": "content-view", 
    "properties": {
        "token": "15de388dc6d39118b914db428a8975de",
        "url": "http://alexkessinger.net/my-story"
    }
}

In python I can create something like the above

import base64
import simplejson

item = {
    'event':'content-view',
    'properties': {
        'token': '15de388dc6d39118b914db428a8975de',
        'url': 'http://alexkessinger.net/my-story'
}

data = base64.b64encode(simplejson.dumps(item))
url = 'http://api.mixpanel.com/track/?data='
url = url + data
url = url + "&img=1"
print url 
>>> http://api.mixpanel.com/track/?data=eyJldmVudCI6ICJjb250ZW50LXZpZXciLCAicHJvcGVydGllcyI6IHsidXJsIjogImh0dHA6Ly9hbGV4a2Vzc2luZ2VyLm5ldC9teS1zdG9yeSIsICJ0b2tlbiI6ICIxNWRlMzg4ZGM2ZDM5MTE4YjkxNGRiNDI4YTg5NzVkZSJ9fQ==&img=1

Now I can take that url and embed it in an image tag

<img src="http://..." width=1 height=1 > 

And if I embed that into this article I can know how many times it’s been viewed, although not necessarily read.

The other thing to keep in mind is that just because it looks like gobbeldy gook doesn’t mean it’s secure. People just have to base64 decode to see the string, so it shouldn’t be counted upon. While if Mixpanel did want to get serious they could do some form of encrypting with a private shared secret and make it harder to tell what data was being sent.

image

One facet of idea generation, capture, and organization

Idea generation, capture, and organization is a major part of any business. As you might assume every business does this differently as well. I have had a couple different experiences with idea generation.

Sometimes you just start kicking around an idea with friends. It’s fast, its mostly talk. In the end though you will be the one making this thing. There is no need to communicated anything, with anyone. It can be fun, but it’s hard to get an idea on it’s legs, and you are prone to not getting any feedback. Going down a long tunnel before you see daylight can happen quickly, and you end up with a lot of work, and nothing to show for it.

Yahoo, and possibly other large companies, have another method of idea capture. Ideas were okay, but only if they fit into your box. Ideas across the full spectrum were for someone else. It might sound like the responsibility for cross-product communication was somewhere up the chain, but in reality I had no idea where my projects came from. I had this murky understanding that they were usually reactions to market, and that we needed to get it out fast. To be fair, because of that murkiness, you supposedly could do almost anything you wanted. It comes with a catch though you need to convince some one with power to back your idea. We programmers had ideas, and would talk about it often, but there was no capture, because no one was listening. The most recent way has been at a startup. One goal of mine is to try, and understand why companies change as they grow. I really want to understand if it’s possibly to combine the power of startups with large companies. The idea generation thing has been an interesting part of my new job. At first, I just listened. I wanted to make sure I understood how things worked, what was the flow. After awhile I realized that idea generation was organic, ideas would bubble up any time. Sometimes they would get posted to IRC, or just mentioned out loud. Some ideas would be said out loud, and receive no response, others would receive, an oh yea we should do that.

This part of the equation isn’t all that different from the lone wolf experience, except for one crucial part. You have a room filled with people who can make things. Even though these ideas are just bubbling up, and flowing into the world, that doesn’t mean they aren’t getting captured. It just means we are all marinating. Sometimes idea come fourth, are talked about and executed in a matter of hours, or days. Sometimes it can take weeks to get back around because of other pressing matters. It works though.

Here though is the sticking point. I am not sure how well this will scale, I don’t think it can. I could be wrong, it would be wrong to bet the farm on this assumption. I think the point here is that there should be like and idea log. The log metaphorical, but can be instituted as software, you can use a bug tracker, you could use a piece of paper, or a white board. The biggest idea I think is to expose everyone to all the ideas, eventually you will need a curator, but not to limit what can be seen but to group.

Another caveat is communication, in the first two situations it’s not really important. In big companies it’s important to document what you have done, but communication is structural thing, not a p2p thing. In the latter, communication is incredibly important, not just documentation but making sure everyone is informed, that way everyone can give feedback too.

Why instapaper for video as an iOS app is a long ways away

From the first day I created Wacchen people have wanted me to create an app for the iPhone. They wanted something that functions like the instapaper app. I knew from day one that while the idea was the same, there was no way a video for instapaper app could survive in the app store. Now, Wacchen is surrounded by competitors, and even an attempt at an App. I think my time would be better spent else where, so I am seriously changing how Wacchen will work, but I also wanted explain why instapaper for video won’t be in the app store any time soon.

Update:

I shut Wacchen down for a while, but it’s back. I have in a state now that it can run in a mostly unattended fashion

Instapaper as a tool to extend my ability to consume content has been invaluable. Together with things like Google Reader, I feel that I have the beginnings of the tools that I will need to staunch the flow of information. I have never been a worrywart about the amount of information flowing around, but I do know that we need to figure out how to organize a torrent of information into neat little boxes.

For years now, there has been no equivalent for video. There has been a quite, but persistent cry for something like this for years now. You can find it on twitter, blogs, and Instapaper creator, Marco Arments, own forums.

After a fateful, ultimately unsuccessful, interview I built it, quickly. Wacchen was born from a long standing idea stirring in my brain, and the commitment to demonstrate that Rails was not that different from things like Django, infantile, for sure.

Wacchen is like instapaper for video. Arment, the creator of instapaper, has put together an app that’s quite wonderful, and has no doubt spent hours on Instapaper. The amount would make my effort look like a pittance. Even though I know there are probably a million and 1 things that make Instapaper better then Wacchen, and the idea that there is only one major difference is crazy. There is a small difference in how the content is treated. Instapaper has the ability to scrape all the content on the page that you have access too. If you can see it in a browser you can send it to instapaper. If you take apart the bookmarklet you can see how this works, also you can see what kind of data is getting sent back to Instapaper by investigating the HTTP request it makes.

This works for a large range of content, and has made it easier for many people to keep a full copy of all the things they want to read in one place. Then the iPhone app came along, now you could take your full copy and put it on your phone. Now anywhere you had your phone you could read your stuff.

Wacchen works on the same basic principal, you see it in a browser you bookmark it, but the difference is that instead of downloading the video, Wacchen merely figures out how to embed it for you in a central place. Wacchen does not download the videos, but stores references to them.

After I built the site though, I realized what people really wanted was an iPhone app. I knew that would be the best next thing to do, but there was a problem, my app would never stay in the app store for long.

The problem is that video providers are very twitchy about you taking videos offline. It doesn’t matter that you can watch videos in your browser, you are not allowed to take them to go, and are strongly thwarted from trying. It’s clearly not okay, as blah blah, found out after they got revoked from the the app store.I say okay, and not illegal, because being barred from the app store isn’t always a legal question. Even if I wanted to fight the legal battle I would at bare minimum have to fight it without my app being in the app store.

That is where we are, an app that downloads full-text of articles is okay, but an app that downloads full videos is not okay. Mayby someday someone will be able to build full partnership deals, or something but that doesn’t seem likely to happen any time soon.

The limits of localStorage

While building a Google Reader client using jQuery Mobile, and Lawnchair using just localStorage became problematic. No matter the browse they all had a hard 5MB cap, and it wouldn’t grow. 5MB is roughly equivalent to 1600 google reader articles stored as JSON. At any given time a google reader client should be able to store 20,000 to 50,000 stories.

Using localStorage for what some have called a local cookie, is probably a good idea, but it is not meant as a general purpose database.

Right now the only other options for client side storage is WebSQL. WebSQL is on it’s way out, it’s going to be replaced by IndexedDB, but who knows when. This has lead to limited support for WebSQL. This narrowed the browsers on which the Google Reader client could run. Ultimately this was going to be a test for PhoneGap, so mostly WebKit Browsers. The iPhone, and Android, and Palm versions of webkit have WebSQL. Then there is Blackberry, which has some form of Blackberry storage API for which Lawnchair has an adapter for.

Using the WebSQL adaptor for lawnChair was an easy cut-over. There are less storage limits when using WebSQL then when using localStorage. Storing a months worth of unread articles was not a problem. On a side note, dates in JS have always been lacking a great API. Though, there is Datejs. It’s old, and marketed as a fuzzy date parser, but it has a wonderful API for picking dates. For example:

var pastDate = Date.now().add(-7).days();

That is just subtracting 7 days worth of time from right now. You end up with a javascript Date object.

For the uninitated IRC is awesome

I haven’t touched my IM client in a month, and I am loving it. At work we use IRC to keep a running discussion. I was skeptical at first. I wouldn’t be able to get growl notification, and I was going to use irssi, an in terminal IRC client.

On all counts IRC is better. Growl notifications just aren’t needed. As I program there are natural pausing moments. I just pull up the terminal, and see if there is everything new.

irssi is like a an ikea URBAN chair, if you haven’t had the pleasure of these chairs go to an ikea near you. The chair is a champ. It’s injection molded plastic which makes it both easy to clean, and durable. There are no screws to get loose, and there is no paint to scratch off. They just sit around my dinning room table, and work, they do there job. irssi, in combination with screen just work. I ssh’ed into a server the day I started, created a screen running irssi, and it has run since then. I can disconnect, and reconnect whenever. This allows me the capture the entire conversation even when I am not there. This is something that desktop client like colloquy can’t do.

When does it make sense to use objects? answer /via @rmurphey

I didn’t get to go to txjs, but I heard it was awesome. It was organized by Rebecca Murphey. I just got around to seeing her presentation on object literals to organise code. I found this great little slide. If you have ever wrote large amount’s of JS, you might enjoy it as well. When does it make sense to use objects.

If you see this Ms. Murphey, with all do respect, don’t use slide share. Just hook us up with a PDF or something, so we can reference your work without having to take screenshots.

Working for a startup has more layers then I ever imagined

Today, an interesting app was released. I recently started working for another company with it’s own interesting camera app. Some would say the two apps are in the same space.

It would be easy to launch into a diatribe about why the company I work for is awesome, and how our product is better, but that’s all it would be: a diatribe. Details will be dealt with over time, but what suprized me was how visceral a reaction I had personally.

Working at Yahoo, lot’s of things made me happy the people, and learning to work at a large scale, but never once did I enjoy the product. Never once did I feel like I needed to protect anything. I don’t know what it is about my situation that make me feel so visceral, but I like it.

Right now I want to make great stuff. It would be a mistake to say, oh were going to do better then blah blah, because we build blah blah. Instead I am going to focus on doing a great job at what I build, and in working with our team there isn’t any reason we can’t be successful.

Hacker from both Flickr, and Stamen Design writes about curation.

If you are interested at all in what is happing to curation you should read this well-researched article by Aaron Cope. The article stems from Cope’s time at Stamen Design Group, and Flickr, but it doesn’t stop there. This is a deep article with a lot of links, and references.

What the Internet has done to the field of journalism, it will do to the practice of curation. This is not a claim based on empirical study, but rather, a prediction based on the evolution of “social software”, communities of interest that form and exist principally in on-line networks and a need to find stable ground in a digital world where the means and cost of production and distribution has approached zero.

When I look at the future I see curation tools, and I love finding other smart people who seems to be exploring curation.

Targeting high-res displays, and using css sprites.

Targeting high-res displays, and using css sprites.

On the next version of the picplz mobile site we wanted to start targeting iPhone 4 with high-res images, while at the same time keeping the lower resolution images for iPhone < 4. Of course this was going to be a problem for media queries. Using @media screen and (-webkit-device-pixel-ratio: 2) {} you can target any device that is pixel-doubled. As phones a varying pixel density enter the market we can figure out how to target each pixel density specifically.

When you target a higher density screen with background images, you are just slamming in twice the pixels into the same area. This way the raster image doesn’t look grainy. The rest of UI is built at higher resolutions already, and things like fonts, and other graphical elements are just rendered in higher resolution. This is why using something like SVG for background images will be super useful in the future.

Everything worked well until we tried to put in a CSS sprite. I new I would have to tell the browser that while we are giving you this big image, we only want you to treat it like an image of a smaller size. So we use the background-size property to try something like this. .button { background-image: url(/low-res.png) 0 -15px; width: 15px; height: 15px; }

@media screen and (-webkit-device-pixel-ratio: 2) {
    .button {
        background-image: url(/high-res.png) 0 -30px;
        background-size: 15px 15px;
        -webkit-background-size: 15px 15px;
        -moz-background-size: 15px 15px;
    }
}

It seemed logical. The css was telling the browser to use the same size background-image as above, but our image was completely disappearing. Quick side note: where the hell is a firebug, or webkit inspector for the iPhone. Finally, I realized something, that made me feel a little sheepish; background-size isn’t how big your slice of the background should be, you are essentially saying how big the entire background-image should be, and then your width, and height are the window. In the CSS above the background-size property was saying that the entire css sprite was 15px x 15px size, and then we were shifting it 15px up, so it was disappearing.

Once that all fell into place I also figured out that you can statically set one axis and scale the other axis. The other benefit, is the the position math stays the same. All you need to do is replace your image, and redefine the background-image size. Here is the new pseudo-css.

.button {
    background-image: url(/low-res.png) 0 -15px
    width: 15px;
    height: 15px;
}

@media screen and (-webkit-device-pixel-ratio: 2) {
    .button {
        background-image: url(//high-res.png);
        background-size: 15px 15px;
        -webkit-background-size: 15px 15px;
        -moz-background-size: 15px 15px;
    }
}

Now, the next step I think is figuring out how to use SVGs and falling back to PNGs where not supported.

You can checkout the picplz app on iPhone, or Android.

CSS preprocessors like LESS, and SASS can be inefficient, but have some hidden gems

I never regret learning new tools, but after exploring a new tool sometimes I ask what’s so special. I recently had the opportunity to start learning about LESS, SASS is kind of the same deal. Whenever a group of smart people say check this out, I put it on my to checkout list. I dove in. At the end of a couple of weeks checking it out I think that SASS, and LESS are of questionable value to any person working with CSS. At first it just made me feel uneasy like other weird abstraction’s such as Objective-J, and CoffeeScript, but now I have some concrete reasons.Those reasons aren’t the end of LESS, and there are some reasonable ways to use them, but there are some easy ways to be incredibly inefficient. Past what they represent now, I think SASS, and LESS represent the future of CSS, and once we get some of these ideas into an actual CSS spec they will be incredibly useful.

In all cases I think they seem to a person who does not know the base tech very well to be a boon to productivity, and they probably are, but it comes at a cost. I realize that I am about the give the get off my lawn speech, so if you want someone to pat you on the back you should probably go some place else. Besides being crotchety, I never thought I would use the leaky abstraction argument, but now I can see how powerful, and misunderstood it is. Yes, it’s true that all abstractions are leaky, but it’s also true that some abstractions are more leaky then other.

I have some words for SASS, and LESS, but not all CSS helpers are bad. CSS Frameworks like Blueprint, OOCSS, 960gs are great building blocks. Sure, they have their problems, but they don’t leak nearly as much as SASS, and LESS. Even if you don’t use a “CSS Framework” per-se almost all FE developers have some set of CSS they start with. The key to why they are helpful is due to fact that they embrace the underlying tech, instead of trying to squeeze the underlying to tech into some other shape. For a long time I stayed away from both SASS, and LESS. I have not worked with SASS, but after I compared it with LESS, I realized they were making some of the same basic mistakes. At the company where I work now they have been using LESS, and so I embraced it. As I started to tackle new projects I tried to figure out how to make the most out of LESS , but I was consistently confronted with “uhg” moments. Every new feature just failed in some other way.

There were a number of small gotchas, and one big one. If you use something like less.js to help ease the process of development, you end up with a bunch of inline-style code. The code it generates is fine, but there was no way to connect a style rule back to a file. This can be frustrating when you have conflicting styles, and you want to find out where offending class lives. The second small problems is that it creates a way for you to have functions, but then it doesn’t give you an efficient manner to split those functions into a library.

The biggest problem with LESS, and SASS is that the mixin tool allows you to easily write incredibly inefficient code. They allow you to include css will-nilly, and in almost all cases you could have just encapsulated that code in another class. Then using the Cascade in CSS, applied multiple styles to the same markup thus “mixing” the code together.

There is a way to use LESS in an efficient way. For example, If you want to use some of these new fangled CSS techniques like box-shadow, or border-radius you will need to use many browser specific prefixes. Often it will be a -moz, or a -webkit, but other browsers have there own special tags. You end up writing basically the same style rule over, and over again. Now, if you were using a mixin to accomplish this, you would be including 3 to 4 extra CSS properties wherever you included the mixin.

But, there is a way to harness the flexibility of LESS, and combine it with the cascading part of CSS to produce the smallest possible set of properties and still achieve a mixin like effect.

Let’s say I have have some markup like this.

<p class="blue">Blue rounded corner</p>
<p class="orange">Orange Rounded Corners</p>
<p class="green">Green Rounded Corners</p>

It’s easy to end up writing CSS like this, using LESS, that feel’s concise.

.rounded_corners(@radius: 5px){
  border-radius:@radius;
  -moz-border-radius:@radius;
  -webkit-border-radius:@radius;
}

.blue {
   background-color: blue;
   .rounded_corner(5px);
}

.green {
   background-color: green;
   .rounded_corner(5px);
}

.orange {
   background-color: orange;
   .rounded_corner(5px);
}

That feel’s nice right, seems compact, but if you see the rendered CSS it’s a different story.

.blue {
   background-color: blue;
   border-radius:5px;
  -moz-border-radius:5px;
  -webkit-border-radius:5px;
}

.green {
   background-color: green;
   border-radius:5px;
  -moz-border-radius:5px;
  -webkit-border-radius:5px;
}

.orange {
   background-color: orange;
   border-radius:5px;
  -moz-border-radius:5px;
  -webkit-border-radius:5px;
}

Now it doesn’t looks so good. But you can do something, that uses mixins, which is nice because as the browser specific css prefixes fall out of mainstream you only have to edit one section of code. We can change our css to be something like this.

.rounded_corners(@radius: 5px){
  border-radius:@radius;
  -moz-border-radius:@radius;
  -webkit-border-radius:@radius;
}

.rounded-cornders-5 {
    .rounded_corner(5px);
}

.rounded-cornders-10 {
    .rounded_corner(10px);
}

.blue {
   background-color: blue;
}

.green {
   background-color: green;
}

.orange {
   background-color: orange;
}

And you could change your markup to be like this.

<p class="blue rounded-cornders-5">Blue rounded corner</p>
<p class="orange rounded-cornders-5">Orange Rounded Corners</p>
<p class="green rounded-cornders-10">Green Rounded Corners</p>

In using LESS like this there is some benefit to using a CSS pre-processor. The mixin is an interesting idea, even outside of this css-preprocessor world. If only it could be apart of the css spec. That is exactly what Nicole Sullivan is proposing. By getting SASS, and LESS to implement the idea of a mixin, people can see how a useful mental abstraction it can be, but in its current form it’s inefficient, but think about mixin if it was processed by the client. That way you could define the mixin once, and then on the client the mixin will be mixed. That way you aren’t sending excess code over the wire. Hopefully someone at the CSS working group gets started on this.

Getting to know LESS was interesting, and I am glad that I found something useful out of it. Makes me think that in all these abstractions that make me slightly uneasy I might unearth a gem or two, but I wish these ideas could percolate a little faster into standards.