Anders Conbere

I don't usually write much about things that aren't technology related these days. It's a much more comfortable space for me. But something about some recent conversations with my friend Eina Ooka and her struggles recently, a talk by Jill Bolte Taylor at TED and a fantastic photo journal Life Before Death by the guardian, has filled up my head with strange thoughts on the nature of human interaction, and existence.

I've always had a somewhat disconnected relation to death. Only a couple of people in my life have ever died, and most of those I wasn't close to.

First there was my great grandmother, who I remember visiting at a young age. I don't recall much about here besides an old photograph we have of that visit. I remember a sickly green light came in through her window and feeling scared and a bit nauseated, not long after we left she died and I don't remember it making much of an impression on me.

There was the old man who taught CPR at the camp I worked at as a kid. Maybe it was the years of doing CPR or some fluke of his german ancestry. His wife and him and been married for ages and teaching CPR to the camp for the last decade at least. She had a interesting name. And they both had german accents. He had this really big hands and when having you demonstrate the technique he'd show you would sometimes reach out and take hold of an arm to reposition it. And I remember how strong those hands were. The the long thin fingers were incredibly powerful.

He died early one summer before camp started, I was up there for some reason or another and Ken McEdwards took me and a few of the other kids down to his funeral. I believe it was a closed casket. There were a number of folks from the surrounding area there and the feeling was somber and sad. And I just felt a little out of place. I didn't really know the man, and I wasn't as sad as many of the other people there. I remember feeling a bit... awkward.

Then my cousin Matt killed himself when I was a sophomore in college. This was probably the death that had the greatest effect on me. Perhaps due to some of the rough times I was having in college, trying to figure out my own life. He had put a shotgun in his mouth, and left a note that said "wake me up when ..." I don't remember the rest of it "we're truly free", it was something like that. It was one of those deaths that you could really dig into. He had spent the last couple years helping lead a group Teamsters for a Democratic Union and from the sound of it had been creating quite a stir. You couldn't help but think back to things like the Jimmy Hoffa disappearance, but dismiss those concerns as "a little bit crazy".

I have a few memories of Matt, most dating back to the time we used to spend with the Conbere side of the family in Virginia in the summers. I remember as a kid maybe 5 or 6 looking up to my older cousins as they would burry me in the sand, and dig big holes in the ocean, or bring back treasures for me. I recall all Matt and Emily and Nick playing with me, and while I have some pictures of Matt and I think I remember him in particular, it's hard to say as time has blurred all that together. For whatever reason Matt's age and the grizzly nature of his death really kind of shook me, I think it was the first time I felt a distinct pain about death.

And the last person in my life who has passed away is my grandmother "Nana" Nancy Conbere. I had just happened to be visiting the east coast with my girlfriend Zuzka, when my dad called and told me that Nana was sick and was probably going to die. I wasn't too far from there so Zuzka and I got in her car and drove out to visit. I missed seeing her one last time by only a day. I was really happy that I made it out there though. I got to see a number of members of my family, console my grandfather, and my father. But I felt like I had barely known my grandmother. She had visited a few times, and we had visited a couple more. But she was a quiet woman, and could often be found doing needle point, which to a young boy was never particularly exciting. I remember he being kind, and giving me fudgsicles in the hot virginia summers. I remember how old she seemed to me, and frail.

And as I write these things I do feel sad, for each one of these people in my life who I'll never get know, never see again. But it all for the most part feels very unreal.

I think that we've seen some fairly convincing examples of how labeling or tagging can reduce the complexity of grouping sets of data, in particular when it might be difficult to assign the data items into only on individual group. Some big uses of tagging as the primary form of grouping includes gmail, delicious, and flickr. However in XMPP our roster grouping are still relegated to binning or boxing (an item in a group exists in one and only one group). But roster items aren't simple data types, they represent our relationships with people! and people often don't belong to just one group. Rather the people in our lives often belong to many different intersecting groups (my good friend caleb, is both part of the programmers in my life, and my close friends, and my child hood friends, and the people I play soccer with, and climbing). There doesn't seem to be an technical limitation in creating tags or labels for XMPP roster items, there are some questions to be answered about how to store the relations, and what semantics to use when querying them, but these aren't insurmountable.

as an initial reference for XEP's that look / act similar there is

• MetaContacts
• Annotations

My only worry is that both of these use the Storage protocol, and I question how easy it would be to form queries like 'retrieve me all the users with tags "X" and "Y"' Which might be out of band for this XEP but I suspect that queries like that might be powerful.

We are beyond the point where social networks should be valued on the number of locked in customers they posses. This was the wrong way to place value on social networks to begin with, but it's almost silly now. When looking at social network valuation we need to know a couple of things

1) what are the quality of the users who are participating? * that is, how much money do they spend on the products being sold (ads or otherwise)

A social network that engages people with high disposable incomes, and are likely to purchase products online is much more likely to sell products on that network via advertising. If your network is a particular niche, the strength of that relationship might also come into play, as the quality of the products being sold (advertising) might as a result also increase.

2) how active are those users on the network? * that is, how much time are they spending engaging with the network, and how is that resulting in higher sales.

A social network that engages people to spend more time engaging with people, the product and advertising is worth more than a social networks that fails to do those things. Further the correlation between that activity and sales of products or ads interacted with are extremely important metrics when attempting to understand how valuable the time spent there is.

3) What is the quality of the service your application is providing? * How likely is it that your service will continue to engage users in participation.

If you have a crappy photo editing social network, it's less likely to continue to draw engagement from your users over time than a high quality product in the same niche.

So what does this do to the way we perceive the value of social networks today? If you don't care where your users come from, only that they are active on your site and engaged, what benefits do you have to lockin? I would argue none! I think this idea is terrifying to the folks at Myspace and Facebook, but this kind of valuation has huge ramifications to little sites like dopplr or tripit. These sites have no perceived value in building a social network. They are selling a service, and if they could build their product off of a pre-existing network that was large enough to continue to flow traffic through them at an acceptable rate, I would argue that economically that makes way more sense than attempting to entice people into creating new networks there.

This is what Facebook is selling with their app platform, what google wants to see happen with open social, and where a lot of the conversation about Social Network Portability should be happening. This is a fundamental change in the way that the majority of people view the valuation of social applications, and a difficult one to describe. But this is the power of tools like XMPP, FOAF and XFN. They begin to bring about a decentralized social network that the creators of social networks can use to dramatically increase the valuation of their application (by bringing more people through their app, and allowing them to focus on the quality of the services they provide), which for most social network creators, is how they make their money.

Facebook and MySpace might not be rushing out to join this game, but all the small services should be looking around and saying. "Why are we busy building social networks when we could be using a network that already exists to drive more traffic through our site and thus generate more revenue?" And that's the question really, "how do we generate more revenue?"

I feel like this will probably get lost in the huge torrent of comments here. But I think the sentiment of this is important. I've been talking a lot about "how to make creating social networks easier" for the last year. Part of this discussion has involved the people associated with Social Network Portability and some of it has Involved the people associated with Data Portability.

For the last couple of years a number of people have been fighting a battle waged in the world of publishing. "How do we get closed networks to open more of their data". During that time I've been working on problems like "How can we use xmpp's roster as a distributed social network". I feel like the problem of publishing data has largely been solved, this was a marketing problem and the space is now familiar with many of the contemporary companies and the idea has reached a tipping point. What remains is a problem of digestion and one that has gone relatively unheralded. That is, how do we make this data that is now being exposed useful and relevant.

One of the interesting things that I find happens when you reframe the problem like this, is that "importing" users data becomes minimally interesting. As a social network built around providing a set of features being able to import a starting set of social data is only minimally helpful. My big problems still exist. I need to build tools to help me manage those relationships and I have to market my product such that the user base is large enough to make my application useful. If we extend that into other types of data the same kinds of problems emerge.

Basically, we've tackled a problem that exists for consumers, "I want my data back", and done a pretty good job. But forgotten that these tools we're providing need users themselves, and that the users of these tools are not consumers of social applications, but rather the creators of new social networks.

New social networks need a user base, but the creators also want to reduce the complexity of creating their applications. This means that tools that we provide these creators need to offer something. They need to not only help the creator build a user base quickly, but also provide tools for managing complexity. In the case of importing relationship data, the tools should include abstractions for adding and removing friends, and storing the state of that data off site. The tools might want to bring with them features that are painful to create in and of themselves, like messaging, and presence. The should make the life of the creators of social networks easier, and the fact of the matter is, is that right now getting social data doesn't appear to be the problem that social networks creators are facing.

So how do we approach this? I think the first thing to do is to start using the tools available and imaging ways to bring things to the table. If we look at OpenID and oAuth, and open social, those technologies have seen fairly serious attention from large sites. And if you look all of those technologies help reduce the complexity for the people making social networks. They either help reduce the overhead of exposing api's, logging in users, or creating applications on top of your tools. When we look at the other technologies (XFN, FOAF, Microformats, etc.) that Data portability is championing I think we see a very different pattern. I know I've suggested that xfn offers almost nothing back to creators, but there are solutions I've found for FOAF (pinback + oauth + sparql), and certainly XMPP brings a lot to the table. That being said, these tools don't exist yet, and they'll take time to build. But until our tools are tailored for the creators, I don't think we'll see significant adoption.

When I first started playing in the Social Network Portability space I was working on some very light weight social network applications and wanted a way to interact with a global set of users quickly and easily. Right away I discovered foaf and xfn. I was immediately disenchanted with both technologies for what at the time I termed "statelessness", and in some late night discussion I thought about how my IM buddy list captured all the details I wanted from my friends networks, but allowed me to dynamically update it. From that came xmpp-psn, and numerous other projects that have all since failed horribly.

The state of SNP is not the rosy one that is given by the web community. Yes we have XFN, yes we have FOAF and XMPP, and DISO and SIOC. But most all of these technologies have massive flaws. That being said I think there's real hope for both XMPP and FOAF (more on why I'm leaving XFN off this list later).

So getting back to "Statelessness", XFN and FOAF both suffer from being static representations of a social network, both have the ability to describe complex graphs, be pulled real time, and as such aggregated. But the problems arise when a social network decides to make use of this data (say through google's Social Graph API), at this point the data leaves the control of the user and since it's stateless it becomes a "copy" of that data on the social networks network store, and in FOAF and XFN these technologies do not provide the means to communicate new changes back to the original source.

The current method of thinking seems to suggest that each time I join a new social network I'll point them to some canonical representation of my identity (a foaf file, an openID etc.) and the social network will then process the FOAF or XFN data, store the nodes and arcs in their database and connect me via that data to other users in their network. Then they in turn will publish that data. So the current model starts to fall apart at this point, as networks are then forced to aggregate that social data from each other in oder to stay relevant. This results in either having to be smart about caching and networking, or exponentially increasing number of polling connections (presuming we're using http). XFN further exasperates the difference between what's available for the Social Network provider and the canonical definition of the graph (by way of hrefs) as publishers really have almost no way of changing or removing content from that representation once it's been indexed, spread across multiple pages and distributed across the net.

Contrast this now with a technology like XMPP where the roster is stored on a particular server, queryable from any client in the network of federated XMPP clients, and can be adjusted realtime. This means that in order for social networks to stay up to date with XMPP they make a simple call to the users XMPP server and parse the results, and if they have changes they want to make, they can them communicate those changes back to the XMPP server. Looking at the traffic we now are back at a linearly increasing traffic graph, and the complexity of the operations has been reduced significantly. This being said, XMPP has it's own host of issues, not the least of which is it's ability to interact with the web over http, and the complexity of programing api's to communicate with it as it stands today.

It should be clear that the current state of FOAF and XFN aren't really going to cut it for todays social networks. They are great static publish side technologies, but that's not the world we live in with social networks. Social networks are wildly dynamic and they shift with context and need real time awareness. But but FOAF and XFN are doing something great, they are actually getting that data published and out there. Now we need to turn around and look at how those technologies can be made better for the creators of social networks.

So what do we do about it? Surprisingly completely on it's own and unrelated the rdf community has created some technology that I think might be extremely relevant here. It's called SPARQL and you can imagine it as being a simple SQL like query language for RDF stores. This it seems would make an incredible tool for defining the communication methods between hosts providing FOAF data. Now this problem isn't even close to being solved either, but the steps to solve it aren't particularly hard. There have been some interesting extensions to SPARQL that allow for inserts and updates. If we standardize on that as a means of communication then the supplier of FOAF data now needs to provide a means of digesting those SPARQL queries and announcing that they support updates to their FOAF file. Couple this with the work that Dan Brickley's been doing on oAuth bindings for FOAF and you have a powerful tool for not only representing your relationship data, but maintaining the state of those relationships close to realtime.

As for XFN? I don't think XFN is up for the job I don't see any way to repair the problems I've mentioned above with the state of XFN. (If you have one please tell me!)

I'm a Conference Newb, so I can't speak with great detail about how this event related to all the others in the world, but I can say I had a great time, met a ton of great people and solidified my relationship with others. I came away with a much better gut feeling on how the semantic web folks see the end game playing out, and the problems that I see with the current approach (It doesn't provide tools to capture the dynamic nature of social data, social networks need to be able to update a foaf file on a remote machine, not just aggregate it). I also arrived at an interesting to me realization of what I've found so disturbing about CrowdSourcing and how I aim to fix it.

Highlights were:

• Talking with Dan Brickley about FOAF and XMPP
• Meeting Stephanie Booth. Freelancers check out her GoingSolo Conference happening in May
• Meeting Johann Romefort of Seesmic
• Metting Uldis Bojars and the rest of the DERI Folk
• An interesting talk on the future of advertising
• An interesting talk by the BT about Tidly Wiki
• much more I'm sure

I Gave a talk on XMPP two days ago as part of the XNP webcamp in Cork, I attempted to make my Presentation in the style of Lessig. The result is that these notes aren't particularly interesting, but I should have a link to the video up soon. The talk went very well and was quite exciting to give. I thought it had gone well after finishing it, but hadn't realized the impact it had until we had adjourned for lunch and a small group slowly grew around me to discuss the implications of real time messaging on their various ventures.

Overall a simply fantastic experience. Now to see if I can reduce my 20 minute talk to 5 and give it at ignite seattle :-D

I arrived in Cork Ireland today. I'm safe, Jet lagged, exhausted, hungry, lost and all those wonderful feelings that come with traveling. I got to me room at a bed and breakfast around 1:00 (aka 5:00 AM) and promptly took a nice little nap and went exploring.

The city is nice, the weather is almost exactly like Seattle's, I still think irish beer is a bit bland, the people are friendly to a fault and it looks to just be a great town. That being said... if anyone attending the SNP/Blog Talk conference (or anyone in cork in general) is reading this, I would love to go get a beer and get some pointers on things to do /see.

Hurah!

So, talk about storm in a teapot.

As I hinted at in my update there was a pretty significant amount of miscommunication that went on here. Starting with a poorly named service, a terribly worded change log when that poorly named service changed and then a series of not so helpful posts to the newsgroup. The real story is that XMPP isn't meeting all of google's mobile needs (it wasn't designed to), they had broken compatibility to make use of only a few of XMPP's features, and changed the name to match that. These changes have no relevance to the rest of Gtalk, and the team encourages people to use some of the freely available java implementations on android.

But that's somewhat less exciting than where we're at now. After some frantic IM'ing and emailing we've got some really great communication started between the Google android team and the XMPP community. The community has been explosively discussing ways to improve mobile efficiency and a flurry of tests have popped up (and talks of an XEP).

My only regret is that the Android team didn't feel like it could start that way. I'm not sure if this is a result of the pressure of working on android and having to ship, or if it's a failing on the part of the XMPP community to reach out to developers. But this is exactly the sort of thing that Open Source projects try to prevent, the fact is that the xmpp community cares what the implementers think, and the problems they run into. And we want to make xmpp better for them.

Anyway I want to give a big thanks to Dan Morrill of Google's Android team for working with me in making this happen, and his fantastic post to the XMPP mailing list.

A post floated by the XMPP standards mailing list today about a change to the android spec involving their use of XMPP. The post is a scary statement from a company that has shown such willingness to work with the XMPP community to help build their Gtalk service.

The com.google.android.xmppService package has been replaced by the com.google.android.gtalkservice package. This is driven by the fact that the GTalk API is not XMPP compliant, and will be less so going forward. The reason is that XMPP is too verbose and inefficient for mobile network connection, and the GTalk API will be moving to a binary encoding for the protocol between the client and the server.

This distresses me. Not because Google wants to do their own thing with their chat service, but because they feel like these are issues that either can't be remedied, or can't be brought to the xmpp community. The result of course is that the standards list has started a discussion about ways that the problem of network efficiency (on the mobile scale. For many standard uses XMPP is fantastically efficient) can be tackled.

Peter Saint-Andre compiled a list of discussion points that could help with this.

1. Recommendations regarding when to use the TCP binding and when to use the HTTP binding (BOSH).

2. Compression via TLS or XEP-0138 (use it!). Also binary XML as a compression mechanism.

3. Fast reconnect to avoid TLS+SASL+resource-binding packets.

4. ETags for roster-get (see XEP-0150, let's resurrect that).

5. Advisability of presence-only connections (no roster-get, just send presence and whatever you receive is nice).

Anyway I hope that Google recognizes the XMPP communities willingness to work with them to provide standard ways for them to accomplish these goals. Or that they should at least be having this discussion in the XMPP space so that other implementations of the protocol can interface or learn from them.

Update:

There's been an update on the developer google group for android clarifying part of what's going on here. It appears that Google had a particular subset of the functionality of a full messaging stack in mind when adding xmpp into android, and that to optimize that subset of functions they're employing a proprietary protocol.

So this might just be a case of over bad wording on their changes page, which has since percolated into the rest of the XMPP community. That being said there's something silly about implementing a non-standard solution for messaging on the device without consulting with the xmpp community and foundation on ways that you could make that standards compliant. And I think that if you look at what the "right thing" to do is (I'll quote Mark Atwood here)

fix gtalk, not break android, and XMPP compression will beat their proprietary binary protocol

That being said, I think this is good wake up call to the XMPP community that this is a growing problem space, one that we need to think about and possibly one we should employee resources to tackle. And I think it would be very noble of google to work with the community in understanding the kinds of limitations their facing, the tools they need, and the ways that xmpp can help meet those needs.