Anders Conbere

We are beyond the point where social networks should be valued on the number of locked in customers they posses. This was the wrong way to place value on social networks to begin with, but it's almost silly now. When looking at social network valuation we need to know a couple of things

1) what are the quality of the users who are participating? * that is, how much money do they spend on the products being sold (ads or otherwise)

A social network that engages people with high disposable incomes, and are likely to purchase products online is much more likely to sell products on that network via advertising. If your network is a particular niche, the strength of that relationship might also come into play, as the quality of the products being sold (advertising) might as a result also increase.

2) how active are those users on the network? * that is, how much time are they spending engaging with the network, and how is that resulting in higher sales.

A social network that engages people to spend more time engaging with people, the product and advertising is worth more than a social networks that fails to do those things. Further the correlation between that activity and sales of products or ads interacted with are extremely important metrics when attempting to understand how valuable the time spent there is.

3) What is the quality of the service your application is providing? * How likely is it that your service will continue to engage users in participation.

If you have a crappy photo editing social network, it's less likely to continue to draw engagement from your users over time than a high quality product in the same niche.

So what does this do to the way we perceive the value of social networks today? If you don't care where your users come from, only that they are active on your site and engaged, what benefits do you have to lockin? I would argue none! I think this idea is terrifying to the folks at Myspace and Facebook, but this kind of valuation has huge ramifications to little sites like dopplr or tripit. These sites have no perceived value in building a social network. They are selling a service, and if they could build their product off of a pre-existing network that was large enough to continue to flow traffic through them at an acceptable rate, I would argue that economically that makes way more sense than attempting to entice people into creating new networks there.

This is what Facebook is selling with their app platform, what google wants to see happen with open social, and where a lot of the conversation about Social Network Portability should be happening. This is a fundamental change in the way that the majority of people view the valuation of social applications, and a difficult one to describe. But this is the power of tools like XMPP, FOAF and XFN. They begin to bring about a decentralized social network that the creators of social networks can use to dramatically increase the valuation of their application (by bringing more people through their app, and allowing them to focus on the quality of the services they provide), which for most social network creators, is how they make their money.

Facebook and MySpace might not be rushing out to join this game, but all the small services should be looking around and saying. "Why are we busy building social networks when we could be using a network that already exists to drive more traffic through our site and thus generate more revenue?" And that's the question really, "how do we generate more revenue?"

I feel like this will probably get lost in the huge torrent of comments here. But I think the sentiment of this is important. I've been talking a lot about "how to make creating social networks easier" for the last year. Part of this discussion has involved the people associated with Social Network Portability and some of it has Involved the people associated with Data Portability.

For the last couple of years a number of people have been fighting a battle waged in the world of publishing. "How do we get closed networks to open more of their data". During that time I've been working on problems like "How can we use xmpp's roster as a distributed social network". I feel like the problem of publishing data has largely been solved, this was a marketing problem and the space is now familiar with many of the contemporary companies and the idea has reached a tipping point. What remains is a problem of digestion and one that has gone relatively unheralded. That is, how do we make this data that is now being exposed useful and relevant.

One of the interesting things that I find happens when you reframe the problem like this, is that "importing" users data becomes minimally interesting. As a social network built around providing a set of features being able to import a starting set of social data is only minimally helpful. My big problems still exist. I need to build tools to help me manage those relationships and I have to market my product such that the user base is large enough to make my application useful. If we extend that into other types of data the same kinds of problems emerge.

Basically, we've tackled a problem that exists for consumers, "I want my data back", and done a pretty good job. But forgotten that these tools we're providing need users themselves, and that the users of these tools are not consumers of social applications, but rather the creators of new social networks.

New social networks need a user base, but the creators also want to reduce the complexity of creating their applications. This means that tools that we provide these creators need to offer something. They need to not only help the creator build a user base quickly, but also provide tools for managing complexity. In the case of importing relationship data, the tools should include abstractions for adding and removing friends, and storing the state of that data off site. The tools might want to bring with them features that are painful to create in and of themselves, like messaging, and presence. The should make the life of the creators of social networks easier, and the fact of the matter is, is that right now getting social data doesn't appear to be the problem that social networks creators are facing.

So how do we approach this? I think the first thing to do is to start using the tools available and imaging ways to bring things to the table. If we look at OpenID and oAuth, and open social, those technologies have seen fairly serious attention from large sites. And if you look all of those technologies help reduce the complexity for the people making social networks. They either help reduce the overhead of exposing api's, logging in users, or creating applications on top of your tools. When we look at the other technologies (XFN, FOAF, Microformats, etc.) that Data portability is championing I think we see a very different pattern. I know I've suggested that xfn offers almost nothing back to creators, but there are solutions I've found for FOAF (pinback + oauth + sparql), and certainly XMPP brings a lot to the table. That being said, these tools don't exist yet, and they'll take time to build. But until our tools are tailored for the creators, I don't think we'll see significant adoption.

When I first started playing in the Social Network Portability space I was working on some very light weight social network applications and wanted a way to interact with a global set of users quickly and easily. Right away I discovered foaf and xfn. I was immediately disenchanted with both technologies for what at the time I termed "statelessness", and in some late night discussion I thought about how my IM buddy list captured all the details I wanted from my friends networks, but allowed me to dynamically update it. From that came xmpp-psn, and numerous other projects that have all since failed horribly.

The state of SNP is not the rosy one that is given by the web community. Yes we have XFN, yes we have FOAF and XMPP, and DISO and SIOC. But most all of these technologies have massive flaws. That being said I think there's real hope for both XMPP and FOAF (more on why I'm leaving XFN off this list later).

So getting back to "Statelessness", XFN and FOAF both suffer from being static representations of a social network, both have the ability to describe complex graphs, be pulled real time, and as such aggregated. But the problems arise when a social network decides to make use of this data (say through google's Social Graph API), at this point the data leaves the control of the user and since it's stateless it becomes a "copy" of that data on the social networks network store, and in FOAF and XFN these technologies do not provide the means to communicate new changes back to the original source.

The current method of thinking seems to suggest that each time I join a new social network I'll point them to some canonical representation of my identity (a foaf file, an openID etc.) and the social network will then process the FOAF or XFN data, store the nodes and arcs in their database and connect me via that data to other users in their network. Then they in turn will publish that data. So the current model starts to fall apart at this point, as networks are then forced to aggregate that social data from each other in oder to stay relevant. This results in either having to be smart about caching and networking, or exponentially increasing number of polling connections (presuming we're using http). XFN further exasperates the difference between what's available for the Social Network provider and the canonical definition of the graph (by way of hrefs) as publishers really have almost no way of changing or removing content from that representation once it's been indexed, spread across multiple pages and distributed across the net.

Contrast this now with a technology like XMPP where the roster is stored on a particular server, queryable from any client in the network of federated XMPP clients, and can be adjusted realtime. This means that in order for social networks to stay up to date with XMPP they make a simple call to the users XMPP server and parse the results, and if they have changes they want to make, they can them communicate those changes back to the XMPP server. Looking at the traffic we now are back at a linearly increasing traffic graph, and the complexity of the operations has been reduced significantly. This being said, XMPP has it's own host of issues, not the least of which is it's ability to interact with the web over http, and the complexity of programing api's to communicate with it as it stands today.

It should be clear that the current state of FOAF and XFN aren't really going to cut it for todays social networks. They are great static publish side technologies, but that's not the world we live in with social networks. Social networks are wildly dynamic and they shift with context and need real time awareness. But but FOAF and XFN are doing something great, they are actually getting that data published and out there. Now we need to turn around and look at how those technologies can be made better for the creators of social networks.

So what do we do about it? Surprisingly completely on it's own and unrelated the rdf community has created some technology that I think might be extremely relevant here. It's called SPARQL and you can imagine it as being a simple SQL like query language for RDF stores. This it seems would make an incredible tool for defining the communication methods between hosts providing FOAF data. Now this problem isn't even close to being solved either, but the steps to solve it aren't particularly hard. There have been some interesting extensions to SPARQL that allow for inserts and updates. If we standardize on that as a means of communication then the supplier of FOAF data now needs to provide a means of digesting those SPARQL queries and announcing that they support updates to their FOAF file. Couple this with the work that Dan Brickley's been doing on oAuth bindings for FOAF and you have a powerful tool for not only representing your relationship data, but maintaining the state of those relationships close to realtime.

As for XFN? I don't think XFN is up for the job I don't see any way to repair the problems I've mentioned above with the state of XFN. (If you have one please tell me!)

Those of you who don't know I've been working for Fidgt for the last couple of months. Mostly consulting on how best to integrate with the rest of the open social networking world, as well as how we can be the best participants possible. As part of this I've had the joy of being able to work with a number of the available python libraries for things like Microformats, FaceBook, and Flickr.

Sadly, very few of the available tools met our needs going forward, as such much of my time lately has been spent figuring out how to make a set of generic tools that will provide a basis for building up access to each of these services and their web service api's.

One of the first problems we had was that many of them either presumed a single user type solution (a class instantiated with a username and password), this is sub obtimal for our high load use case, and a bit difficult to work with when wanting to be able to generically query various social networks. The result is that the first thing I did was write an Auth layer. The auth layer is very simply a database access layer in sqlalchemy and some other associated functions to ease use. It stores the relation between a username that we use, a username that the remote service uses and a token or password for that user at that network. Since most of the authorization patterns use a very similar workflow for authorizing access from users this has let me abstract much of the background storage and retrieval of tokens.

So where are we on this? We currently have working libraries for FlickrAuth, and FacebookAuth and we're moving towards an OAuth solution.

Writing these libraries is when I began to really understand the beauty of oAuth. Ahhh... imagine only having to ever write one of these libraries and have it "just work" across multiple networks. And not only does oAuth solidify how we go about authorizing a user, but also defines the way we make signed requests to the service, so that i no longer have to write 5 different parameter serialization functions one for each service. It's a fantastic achievement, and my only beef is that it's flexibility lends itself to a much more complex set of tools to use it.

Today I finished the first working xmpp-psn client, it's a small ruby module, and will let you talk http to a running xmpp-psn server anywhere to do xmpp/jabber. I don't have it in subversion yet, but it should be there soon as I clean up a few things and finish some testing. In all the performance looks to be pretty good, and it works :-D

A python client should be coming soon!

As most people around me know, I've been working pretty hard on disseminating the idea of using jabber as a backbone for a truly open social network, as well as attempting to build a set of libraries exposing a simple web api to provide for this.

The two web frameworks I'm most comfortable with are Ruby on Rails and Django (and PHP but I'm not ready to back down that road just yet). So I've been moving forward building some simple web applications in these frameworks, and in general I'm calling the project xmpp-psn short for Extensible Messaging and Presence Protocol - Portable Social Network (I know that's a terrible marketing name, and a mouthful, but I'm focussed on building the tools more than thinking up good names, if you have suggestions... email me!).

Well, Friday talking to Blaine Cook, he brought up a point I had yet to consider and lends itself to a rather tricky problem. I've been writing these tools as if they were there only entry point to the app, of course this means that in order to scale the app you have to put it on better and better hardware, in particular it means given the way that most web servers work (spawning a new instance of the ruby/python interpreter through fastcgi or mod_*) that it can't run on any multiprocess or threaded web server.

So what's the solution?

I think for now I'm going to ignore it. There are solutions that exist drb or dynamic ruby offers a messaging type solution (much like how erlang deals with concurrency) a solution for messaging through apache sessions that I've forgotten the link to and more. The problem with all of these is that the would require a) a lot of restructuring of code I already have written, and b) a lot of time spent thinking about how to do it. So, considering how small my target audience is at the moment, and that I'm more concerned with showing people what can be done, I'm just going to let it slide. Hopefully someone will come along and help me work out that kink when activity picks up.

Today I finished the first version of a Django app that lets you build a social networking site on top of Jabber (XMPP). You can find out more about the basics of the project here, but basically it's an http layer on top of a common python XMPP library that lets you do most of the roster management available through XMPP by way of a simple restful api.

The workflow for using it is simple, just install the app into an existing django project, and that will expose the api. From there you can post your jid and password to the /login/ url which returns a uuid and a roster in JSON format for your web app to digest. (note: this is horrible security practice, don't do this in the wild) Now that you have a uuid you have access to the rest of the api, which links to a thread on the sever that keep track of your jabber session.

the following is a list of the supported roster functions:

• disconnect
• roster
• subscribe
• unsubscribe
• authorize
• unauthorize
• remove
• login

I've been spending sometime thinking about one of my primary concerns with social networking tools today, the non-portability of the data you put into them. That is, you spend a great deal of time on any given social networking service discovering, and managing your social network, yet all the work that's done there is in general trapped on that service.

There are some services that have taken a more socially conscious attitude and allow you to export that data. From the look of Leah Culvers blog pownce (and by association probably digg) will be providing a service to manage your social graphs, Geni has a feature that allows you to export the family tree you've created and so on.

The solutions I've seen thus far are varied and there appears to be a lot of work going on in this space to come up with a solutions that works for service providers (facebook, myspace, pownce) as well as users of these services (my friends). Most of what I'm seeing is a kind of mashup of XFN, microformats, openID and scraping of sites. (The pownce solutions is actually a lot better than this in that it relies on a two way communication between the given services providing an actual api for the service to tie into). Other solutions have involved XML exports, or FOAF, but in the end all of these services rely on a static technology(XML) and a web service to manage the source of that XML document that services such as myspace and facebook could tie into.

I feel like these types of solutions are only stop gap measures for a type of problem that begs for a decentralized dynamic approach.

The approach I've been taking is too take a look at what some of the other protocols available on the web already provide for us. In particular I think that most of what people are looking for in terms of a tool to collect and provide social graph data is cleverly handled by XMPP (jabber).

To better understand where I'm coming from here I want to lay out some of the basic problems we're trying to tackle with portable social networking. As I see them they are:

1. a unique identification system: you have to have some way of uniquely identifying members of your social graph. This could be via openID or another uri type structure, or how email/xmpp does it with unique names per domain.

2. an standardized format for defining relationships: XFN, FOAF, or just a list of friends will do. But people need to know what kind of data they're getting back. Looking at how most providers deal with friend data, the most important datum to know is who you recognize as being a "friend"

3. a simple safe web api for requesting this data for use on other networks.

After one tackles those three primary problems the secondary problems are things like:

1. The system should be distributed: This will allow for the easy use and scaling of the system.

2. The system should be defined in a clear open standard: We don't want people to be tied to a particular language or implementation.

given that problem definition XMPP provides for nearly all of those and more. So how does it work?

We use XMPP's existing network, Jabber, to provide a decentralized set of servers for us to use. Each user on the jabber network has a unique id provided of the form xxx@domain.tld, authentication is over SASL and takes place in direct communication to the users jabber server from the client.

The xmpp term for a social graph is a "roster" or buddy list. The roster provides a list of people with whom you have a trust relationship. By requesting a roster from an xmpp server you receive an xml file with a list of trusted users.

So, we as a social networking provider, ask for jabber authentication information at signup, those details are used to request a roster from their xmpp server, which we then use to form a starting social network on our service. Add a friend links or remove a friend links make a call to the xmpp server to remove or add the unique id's to and from their roster, this can either make direct calls to the webservice api or that can be done on the backend. In this way services are easily able to deploy their own xmpp gateway and yet any work done on their network is directly translated into changes in the jabber roster.

What excites me about this solution is that it actually gives something back to the network providers that they aren't getting now, for free. It gives them chat and presence! not only that but it allows users to manage their friend networks in the ways that they are comfortable with now, either through their current network providers (facebook, myspace), or through their im client, etc.