Anders Conbere

We are beyond the point where social networks should be valued on the number of locked in customers they posses. This was the wrong way to place value on social networks to begin with, but it's almost silly now. When looking at social network valuation we need to know a couple of things

1) what are the quality of the users who are participating? * that is, how much money do they spend on the products being sold (ads or otherwise)

A social network that engages people with high disposable incomes, and are likely to purchase products online is much more likely to sell products on that network via advertising. If your network is a particular niche, the strength of that relationship might also come into play, as the quality of the products being sold (advertising) might as a result also increase.

2) how active are those users on the network? * that is, how much time are they spending engaging with the network, and how is that resulting in higher sales.

A social network that engages people to spend more time engaging with people, the product and advertising is worth more than a social networks that fails to do those things. Further the correlation between that activity and sales of products or ads interacted with are extremely important metrics when attempting to understand how valuable the time spent there is.

3) What is the quality of the service your application is providing? * How likely is it that your service will continue to engage users in participation.

If you have a crappy photo editing social network, it's less likely to continue to draw engagement from your users over time than a high quality product in the same niche.

So what does this do to the way we perceive the value of social networks today? If you don't care where your users come from, only that they are active on your site and engaged, what benefits do you have to lockin? I would argue none! I think this idea is terrifying to the folks at Myspace and Facebook, but this kind of valuation has huge ramifications to little sites like dopplr or tripit. These sites have no perceived value in building a social network. They are selling a service, and if they could build their product off of a pre-existing network that was large enough to continue to flow traffic through them at an acceptable rate, I would argue that economically that makes way more sense than attempting to entice people into creating new networks there.

This is what Facebook is selling with their app platform, what google wants to see happen with open social, and where a lot of the conversation about Social Network Portability should be happening. This is a fundamental change in the way that the majority of people view the valuation of social applications, and a difficult one to describe. But this is the power of tools like XMPP, FOAF and XFN. They begin to bring about a decentralized social network that the creators of social networks can use to dramatically increase the valuation of their application (by bringing more people through their app, and allowing them to focus on the quality of the services they provide), which for most social network creators, is how they make their money.

Facebook and MySpace might not be rushing out to join this game, but all the small services should be looking around and saying. "Why are we busy building social networks when we could be using a network that already exists to drive more traffic through our site and thus generate more revenue?" And that's the question really, "how do we generate more revenue?"

When I first started playing in the Social Network Portability space I was working on some very light weight social network applications and wanted a way to interact with a global set of users quickly and easily. Right away I discovered foaf and xfn. I was immediately disenchanted with both technologies for what at the time I termed "statelessness", and in some late night discussion I thought about how my IM buddy list captured all the details I wanted from my friends networks, but allowed me to dynamically update it. From that came xmpp-psn, and numerous other projects that have all since failed horribly.

The state of SNP is not the rosy one that is given by the web community. Yes we have XFN, yes we have FOAF and XMPP, and DISO and SIOC. But most all of these technologies have massive flaws. That being said I think there's real hope for both XMPP and FOAF (more on why I'm leaving XFN off this list later).

So getting back to "Statelessness", XFN and FOAF both suffer from being static representations of a social network, both have the ability to describe complex graphs, be pulled real time, and as such aggregated. But the problems arise when a social network decides to make use of this data (say through google's Social Graph API), at this point the data leaves the control of the user and since it's stateless it becomes a "copy" of that data on the social networks network store, and in FOAF and XFN these technologies do not provide the means to communicate new changes back to the original source.

The current method of thinking seems to suggest that each time I join a new social network I'll point them to some canonical representation of my identity (a foaf file, an openID etc.) and the social network will then process the FOAF or XFN data, store the nodes and arcs in their database and connect me via that data to other users in their network. Then they in turn will publish that data. So the current model starts to fall apart at this point, as networks are then forced to aggregate that social data from each other in oder to stay relevant. This results in either having to be smart about caching and networking, or exponentially increasing number of polling connections (presuming we're using http). XFN further exasperates the difference between what's available for the Social Network provider and the canonical definition of the graph (by way of hrefs) as publishers really have almost no way of changing or removing content from that representation once it's been indexed, spread across multiple pages and distributed across the net.

Contrast this now with a technology like XMPP where the roster is stored on a particular server, queryable from any client in the network of federated XMPP clients, and can be adjusted realtime. This means that in order for social networks to stay up to date with XMPP they make a simple call to the users XMPP server and parse the results, and if they have changes they want to make, they can them communicate those changes back to the XMPP server. Looking at the traffic we now are back at a linearly increasing traffic graph, and the complexity of the operations has been reduced significantly. This being said, XMPP has it's own host of issues, not the least of which is it's ability to interact with the web over http, and the complexity of programing api's to communicate with it as it stands today.

It should be clear that the current state of FOAF and XFN aren't really going to cut it for todays social networks. They are great static publish side technologies, but that's not the world we live in with social networks. Social networks are wildly dynamic and they shift with context and need real time awareness. But but FOAF and XFN are doing something great, they are actually getting that data published and out there. Now we need to turn around and look at how those technologies can be made better for the creators of social networks.

So what do we do about it? Surprisingly completely on it's own and unrelated the rdf community has created some technology that I think might be extremely relevant here. It's called SPARQL and you can imagine it as being a simple SQL like query language for RDF stores. This it seems would make an incredible tool for defining the communication methods between hosts providing FOAF data. Now this problem isn't even close to being solved either, but the steps to solve it aren't particularly hard. There have been some interesting extensions to SPARQL that allow for inserts and updates. If we standardize on that as a means of communication then the supplier of FOAF data now needs to provide a means of digesting those SPARQL queries and announcing that they support updates to their FOAF file. Couple this with the work that Dan Brickley's been doing on oAuth bindings for FOAF and you have a powerful tool for not only representing your relationship data, but maintaining the state of those relationships close to realtime.

As for XFN? I don't think XFN is up for the job I don't see any way to repair the problems I've mentioned above with the state of XFN. (If you have one please tell me!)

I'm a Conference Newb, so I can't speak with great detail about how this event related to all the others in the world, but I can say I had a great time, met a ton of great people and solidified my relationship with others. I came away with a much better gut feeling on how the semantic web folks see the end game playing out, and the problems that I see with the current approach (It doesn't provide tools to capture the dynamic nature of social data, social networks need to be able to update a foaf file on a remote machine, not just aggregate it). I also arrived at an interesting to me realization of what I've found so disturbing about CrowdSourcing and how I aim to fix it.

Highlights were:

• Talking with Dan Brickley about FOAF and XMPP
• Meeting Stephanie Booth. Freelancers check out her GoingSolo Conference happening in May
• Meeting Johann Romefort of Seesmic
• Metting Uldis Bojars and the rest of the DERI Folk
• An interesting talk on the future of advertising
• An interesting talk by the BT about Tidly Wiki
• much more I'm sure

I arrived in Cork Ireland today. I'm safe, Jet lagged, exhausted, hungry, lost and all those wonderful feelings that come with traveling. I got to me room at a bed and breakfast around 1:00 (aka 5:00 AM) and promptly took a nice little nap and went exploring.

The city is nice, the weather is almost exactly like Seattle's, I still think irish beer is a bit bland, the people are friendly to a fault and it looks to just be a great town. That being said... if anyone attending the SNP/Blog Talk conference (or anyone in cork in general) is reading this, I would love to go get a beer and get some pointers on things to do /see.

Hurah!

So, talk about storm in a teapot.

As I hinted at in my update there was a pretty significant amount of miscommunication that went on here. Starting with a poorly named service, a terribly worded change log when that poorly named service changed and then a series of not so helpful posts to the newsgroup. The real story is that XMPP isn't meeting all of google's mobile needs (it wasn't designed to), they had broken compatibility to make use of only a few of XMPP's features, and changed the name to match that. These changes have no relevance to the rest of Gtalk, and the team encourages people to use some of the freely available java implementations on android.

But that's somewhat less exciting than where we're at now. After some frantic IM'ing and emailing we've got some really great communication started between the Google android team and the XMPP community. The community has been explosively discussing ways to improve mobile efficiency and a flurry of tests have popped up (and talks of an XEP).

My only regret is that the Android team didn't feel like it could start that way. I'm not sure if this is a result of the pressure of working on android and having to ship, or if it's a failing on the part of the XMPP community to reach out to developers. But this is exactly the sort of thing that Open Source projects try to prevent, the fact is that the xmpp community cares what the implementers think, and the problems they run into. And we want to make xmpp better for them.

Anyway I want to give a big thanks to Dan Morrill of Google's Android team for working with me in making this happen, and his fantastic post to the XMPP mailing list.

Those of you who don't know I've been working for Fidgt for the last couple of months. Mostly consulting on how best to integrate with the rest of the open social networking world, as well as how we can be the best participants possible. As part of this I've had the joy of being able to work with a number of the available python libraries for things like Microformats, FaceBook, and Flickr.

Sadly, very few of the available tools met our needs going forward, as such much of my time lately has been spent figuring out how to make a set of generic tools that will provide a basis for building up access to each of these services and their web service api's.

One of the first problems we had was that many of them either presumed a single user type solution (a class instantiated with a username and password), this is sub obtimal for our high load use case, and a bit difficult to work with when wanting to be able to generically query various social networks. The result is that the first thing I did was write an Auth layer. The auth layer is very simply a database access layer in sqlalchemy and some other associated functions to ease use. It stores the relation between a username that we use, a username that the remote service uses and a token or password for that user at that network. Since most of the authorization patterns use a very similar workflow for authorizing access from users this has let me abstract much of the background storage and retrieval of tokens.

So where are we on this? We currently have working libraries for FlickrAuth, and FacebookAuth and we're moving towards an OAuth solution.

Writing these libraries is when I began to really understand the beauty of oAuth. Ahhh... imagine only having to ever write one of these libraries and have it "just work" across multiple networks. And not only does oAuth solidify how we go about authorizing a user, but also defines the way we make signed requests to the service, so that i no longer have to write 5 different parameter serialization functions one for each service. It's a fantastic achievement, and my only beef is that it's flexibility lends itself to a much more complex set of tools to use it.

Today I finished the first working xmpp-psn client, it's a small ruby module, and will let you talk http to a running xmpp-psn server anywhere to do xmpp/jabber. I don't have it in subversion yet, but it should be there soon as I clean up a few things and finish some testing. In all the performance looks to be pretty good, and it works :-D

A python client should be coming soon!

Today I finished the first version of a Django app that lets you build a social networking site on top of Jabber (XMPP). You can find out more about the basics of the project here, but basically it's an http layer on top of a common python XMPP library that lets you do most of the roster management available through XMPP by way of a simple restful api.

The workflow for using it is simple, just install the app into an existing django project, and that will expose the api. From there you can post your jid and password to the /login/ url which returns a uuid and a roster in JSON format for your web app to digest. (note: this is horrible security practice, don't do this in the wild) Now that you have a uuid you have access to the rest of the api, which links to a thread on the sever that keep track of your jabber session.

the following is a list of the supported roster functions:

• disconnect
• roster
• subscribe
• unsubscribe
• authorize
• unauthorize
• remove
• login

Tuesday I went to an event called "green drinks" here in Seattle. It's a very casual, low key event that seeks to get a group of people interested in environmental topics and issues in one place for drinks / socializing. It reminded me a lot of ignite Seattle in that same kind of buzzing - surrounded by like minded people - kind of experience. I think that those kinds of feelings are incredible empowering, it reminds you that you aren't alone and that if you needed to you could rally support for something.

If nothing else Green Drinks succeeds at being fun, something that the founder of the Seattle chapter considers when planning the event. And I agree I think that without that aspect of fun and excitement eventually participation begins to decline and the mix of new people to broaden the horizons of the group or to push them to act fades away.

However I found that much like many of the geek events I go to, the people who attend tend to clump together in familiar and easy social groups (friends, acquaintances or people they otherwise already knew). While this is the way of most large social gatherings (and green drinks appears to be pulling in quite a crowd, I would guess on the order of 300 participants), I always feel like more could be done to help unite the groups, get them outside of their shell, and to think beyond the event. Because while green drinks is great, it is not environmental activism, activism is what happens when these people find each other, become united on a cause and grow the community such that it exists pervasively, not only at a once a month event.

To better define the problem, I believe that at least two things must exist for communities to form.

1. dialog
2. interaction

So how do we begin to start dialog at green drinks, and how to we begin to foster greater interaction.

The answer is going to be difficult, it might be games, or talks or little talking pieces, or maybe and intersection of technologies like noonhat, and it's going to be unique given the group of people that are participating. However, regardless of what it is, it's primary job should be to break down the predefined bounds that hold the groups already in place together, and help to inject the new people into those smaller communities until the mixture is homogeneous (a dangerous word). It occurs to me that the description above is exactly how soap interacts with oil and water.

And so to bring it all back... I think we need some more soap, and not just at Green Drinks, we need it at work, at geek events like ignite Seattle, and we need it in life.

The other day I saw a talk at Ignite Seattle on a tool a guy had made called PicklePost. Pickle Post is a fairly simple tool and not necessarily the most simple concept. Brought down to it's bare bones it's a method of managing and displaying static digital data from the web in a physical space. The idea for this being that by bringing the web into a physical space (by way of a fancy monitor attached to a small wifi enabled computer), you help pull the social web interaction that goes on in a coffee shop, or cafe, into real community interaction.

The problem I have with this is that I don't actually think the idea is helping to tackle the problem. I agree that it is helping bring parts of the web into the physical realm (good!), but I'm not sure I understand how it's helping build the physical communities. Communities are built on dialog and interaction, yet while Pickle Post offers a physical manifestation of the web, it does little to help turn the social web communities that exist into dialog and interaction.

So, what can we do to take what is a good statement of a problem "bring people out of the web and into physical human interaction by way of shared physical space" and help to find a better solution.

In my mind the simplest place to start would be the shared wireless connection. If you are offering an open wireless connection in your place of business, and said wireless is not so powerful as to significantly exit your premise, any people connected to this wireless network will be within the premise. So how do we shape these people's web experience while inside this physical area in order to help promote dialog and interaction amongst them.

One solution to this problem that comes to mind is by simply offering a platform by way of a DNS redirection, that provides for various social features... chat, forum, profiles etc. The idea being that if you give regular customers a place to put something they'll use it, and that if they meet people in that space then they are more likely to attempt to connect to them in real life and by keeping this network available only to the local domain (or at least sign-up), you give that web space a relatively safe starting community.

Participation would be the big problem with this kind of solution, and I can imagine some solutions to that as well... posting discussion topics for the day, repurposing some of the PicklePost technology to display forum threads, having people sign up and take a photo before using the wireless (this seems a bit intrusive but would greatly improve the ability of patrons to find each other).

Tell me what you think.

I've been spending sometime thinking about one of my primary concerns with social networking tools today, the non-portability of the data you put into them. That is, you spend a great deal of time on any given social networking service discovering, and managing your social network, yet all the work that's done there is in general trapped on that service.

There are some services that have taken a more socially conscious attitude and allow you to export that data. From the look of Leah Culvers blog pownce (and by association probably digg) will be providing a service to manage your social graphs, Geni has a feature that allows you to export the family tree you've created and so on.

The solutions I've seen thus far are varied and there appears to be a lot of work going on in this space to come up with a solutions that works for service providers (facebook, myspace, pownce) as well as users of these services (my friends). Most of what I'm seeing is a kind of mashup of XFN, microformats, openID and scraping of sites. (The pownce solutions is actually a lot better than this in that it relies on a two way communication between the given services providing an actual api for the service to tie into). Other solutions have involved XML exports, or FOAF, but in the end all of these services rely on a static technology(XML) and a web service to manage the source of that XML document that services such as myspace and facebook could tie into.

I feel like these types of solutions are only stop gap measures for a type of problem that begs for a decentralized dynamic approach.

The approach I've been taking is too take a look at what some of the other protocols available on the web already provide for us. In particular I think that most of what people are looking for in terms of a tool to collect and provide social graph data is cleverly handled by XMPP (jabber).

To better understand where I'm coming from here I want to lay out some of the basic problems we're trying to tackle with portable social networking. As I see them they are:

1. a unique identification system: you have to have some way of uniquely identifying members of your social graph. This could be via openID or another uri type structure, or how email/xmpp does it with unique names per domain.

2. an standardized format for defining relationships: XFN, FOAF, or just a list of friends will do. But people need to know what kind of data they're getting back. Looking at how most providers deal with friend data, the most important datum to know is who you recognize as being a "friend"

3. a simple safe web api for requesting this data for use on other networks.

After one tackles those three primary problems the secondary problems are things like:

1. The system should be distributed: This will allow for the easy use and scaling of the system.

2. The system should be defined in a clear open standard: We don't want people to be tied to a particular language or implementation.

given that problem definition XMPP provides for nearly all of those and more. So how does it work?

We use XMPP's existing network, Jabber, to provide a decentralized set of servers for us to use. Each user on the jabber network has a unique id provided of the form xxx@domain.tld, authentication is over SASL and takes place in direct communication to the users jabber server from the client.

The xmpp term for a social graph is a "roster" or buddy list. The roster provides a list of people with whom you have a trust relationship. By requesting a roster from an xmpp server you receive an xml file with a list of trusted users.

So, we as a social networking provider, ask for jabber authentication information at signup, those details are used to request a roster from their xmpp server, which we then use to form a starting social network on our service. Add a friend links or remove a friend links make a call to the xmpp server to remove or add the unique id's to and from their roster, this can either make direct calls to the webservice api or that can be done on the backend. In this way services are easily able to deploy their own xmpp gateway and yet any work done on their network is directly translated into changes in the jabber roster.

What excites me about this solution is that it actually gives something back to the network providers that they aren't getting now, for free. It gives them chat and presence! not only that but it allows users to manage their friend networks in the ways that they are comfortable with now, either through their current network providers (facebook, myspace), or through their im client, etc.

Continued from my post on Trust and the OS

So what do we do about trust, the web, and the modern computing platform. As I mentioned the Linux/BSD solution has some excellent aspects to it. The single line of trust, the filtering via "experts", the ability to manually check source code, but it breaks down when users expect the flexibility of the current popular model. Certainly we won't be dropping back on "make && make install" any time soon in major Operating systems.

What I propose is a compromise. I think that windows and OS X should have built in package managers for commonly used applications. Windows' "Add and Remove Programs" is a poor attempt at package control and fails miserably with regards to administrative use. It lacks scripting support, isn't self aware, can't update itself, and has no control over 3rd party applications. So what are the requirements?

1. The operating system needs to restrict installation to packages that meet certain standards. They have to define what files they are placing, where they are placing them, define the author the title, etc. All packages that fail to present this data will fail to install. OS X already provides a platform similar to this and won't install packages that don't meet it's requirements. This will enable the package manager to effectively keep track of applications installed, remove them or update them. Compare this to windows where third party applications are given free reign of what they want to do, just recently were third party applications prevented from attaching to the kernel in order to prevent root-kits (Much to the dismay of anti-virus providers). 3rd part application should in my opinion be by and large locked down to a few basic essentials, they should be forced to conform to the standard installation of the OS.

2. All packages should contain lists of their dependencies internal to themselves. This should be provided in a standard method for the OS.

3. All package installed should be added to the list of installed applications in the package manager, and should be installed through the standard package manager interface (see point 4)

4. A collection of standard applications should be provided by the OS. My biggest worry with this would be the owners of the OS providing privilege to a select few "partners" and excluding others. Perhaps a standardized method of submitting an application for inclusion and a oversight comity to prevent abuse could be implemented. This repository should include commonly installed packages, free or otherwise and could link to purchasing methods for those applications that aren't free. For example. If a user wanted to install photoshop, they might go to the windows package manager, search for art, see a list of packages with art tags, click the "buy this now" button for photoshop, and start their download. Or they might just see "Paint.net" and click install. The in both cases the package manager would handle dependencies (in paint.net that might include the .net runtime environment) and download install the necessary applications.

What does this provide for Operating systems like OS X and Windows?

1. It gives users a centralized, and trusted place to start looking for applications
2. It provides a standard installation interface which has the capability of being much more clear and concise.
3. It allows OS's like OS X or Windows to stop bundling all their own applications in, and offer a more free market place. While this isn't as a much of a worry for OS X without the monopoly litigation hamstringing it, it could be for Windows/Microsoft.

Consider this Microsoft's problems stem mostly from bundling software with their operating system, IE, Windows Media Player, etc. Yet these are critical applications in todays world, people won't accept a computer that doesn't by default have the tools to browse the web, play movies or music. If they were to stop providing any of these by default this would be a disaster, unless they provided a simple way for people to choose to install them on their own.

By having the base windows install be as simple as possible (no games, no office tools, no web browser, no media player) but having a simple package manager with all those tools on the install dvd, people could then choose on boot the applications they wanted, or choose "MS default" or various other options. This would encourage the idea of windows as a platform and allow more free competition.