Share
(1 to 100 of 116 replies)
Gwênlyn Posted 20 years ago
I just posted an update to the Flickr social network analysis.

The data were collected last night - March 14th, 2005 between 1:30 and 3:30 Flickr time. This time I selected 1000 individuals randomly, and crawled their contact networks, finally reaching 90403 users. This means that there could be more than one big network in the final result, and this is indeed the case. Most of us are nevertheless in one main network.
In just over two months, the main network grew by over 75%. You can see the graphs starting from the center of the network and then using lower (more inclusive) cutoffs, which represent the minimal number of mutual contacts to be included.

In the previous analysis I described four clusters - bloggers, GNErs, UAE folks and the main Flickr cluster. The four clusters remain, but there is a significant change: when using more inclusive cutoffs, the bloggers/GNErs/main clusters merge into one, but the UAE cluster (which has grown a lot) remains separate from the rest of the population. There are a few "emissaries" bridging this gap, but it would seem that the language barrier is perpetuating a rather sharp schism within the Flickr population.

For further eye-candy, I posted a (rather long) series of variations on the mc-50 graph (starting here), each one painting the network according to the "contact distance" from one individual - a different person each time, not randomly picked.

In these graphs, dark blue is the individual being highlighted, light blue are his/her immediate contacts, and then lighter colors indicate longer distances away across the network.

These colorings show that there is further substructure within the main cluster, and also that the nodes to the sides of the network may be growing into additional clusters. They also show to what extent the UAE cluster is segregated...

I encourage you to look at the full resolution of the images, to be able to read the names, otherwise it's very hard to make much sense of it. This time I used full names (no more numeric codes!), which caused a new problem - weird characters. I avoided this by replacing weird characters with periods. Sorry about that, v.c.pinta, z.n and every other affected individual...

Enjoy. :)
1
(1 to 100 of 116 replies)
admin
emdot Posted 20 years ago
Gustavo, you rock the flickrverse. Thanks so much for all of your analysis. I find it so interesting. ;)
admin
krazydad / jbum Posted 20 years ago
Hey! I'm the center of my universe! :)
i_am_curiouskiwi Posted 20 years ago
HELP! My contacts page has been taken over by the ghost of Mickey Mouse!
Mouse Attack

Great work, Gustavo, I see that I now make it to the 50 mutual contacts page!
styler* Posted 20 years ago
thanks gustavo
your amazing creations are a wonder
admin
selva Posted 20 years ago
These are so cool!
Vanlal. Posted 20 years ago
Great work Gustavo.
kastner Posted 20 years ago
hehe, perfect topic id.
txkimmers Posted 20 years ago
See, now that is what I want on a t-shirt.
bodhi47 Posted 20 years ago
seems the fotolog.net cluster has been absorbed.
admin
striatic Posted 20 years ago
seems the fotolog.net cluster has been absorbed.

not really .. the fotolog.net group has always been a 'sub-cluster' .. indistinguishable from the main cluster in terms of shape, but discernible as a region inside the main group.

the majority of fotologgers remain grouped together within the main cluster, as they were before.
j david Posted 20 years ago
wow!!!

again!

Thanks for this work, it's wonderful.

Could you post a bit more detail on exactly how you determined "contact distance?" Did you rank mutual relationships (friend-friend, friend-contact, contact-contact, etc.), did you map relationship order (contact of friend, contact of contact, etc.) or some combination?
Gwênlyn Posted 20 years ago
Here's a list of the people for which I created a painted network graph: Aznet, Broutus, caterina, cute_so_cute, Cyprien, dangerpaws, ellipse, emdot, Eric, fubuki, Garnite, George, gretsch, GustavoG, gweedo, hamoOod, iemoto, IniMiTaBLe, j_david, javajive, jbum, jurvetson, kodama, kungfootv, Lianne, lightpainter, macaddict, machete, masazaku-s, Nadar, nthdegx, otherthings, Pandarine, pinhole, quas, ribena, Special, Stewart, striatic, the_nannish_one, väcäpinta, Wattwurm, Ziko.


EDITED important clarification: all these graphs are intentionally low-res. They are re-paintings of the mc-50 graph where you can find all the details. Sorry you can't read the names in the individual graphs, but it didn't make sense to upload so many copies of the same at high resolution...
admin
Special Posted 20 years ago
Dude.. i'm so commenting this time before my evil twin does. heh.

That's a hell of a lot of graphs to go through. Impressive work as always. I love that you did some individual people too. Very interesting.
Neil Posted 20 years ago
What's with the UAE? Has this been explained yet?

The Orkut social network took off in Brazil, for some reason.

Has there been any work done on this? It looks almost like nucleation (literally, with that graph).
j david Posted 20 years ago
yes brevity, there has been a good deal of academic work on this. I think that gustavo's language explanation isn't a bad first cut. It may also run deeper- families, friends and other networks in the UAE are being embedded in flickr.

I can dig up some literature/references if anyone really wants it.
Neil Posted 20 years ago
Oh no, I understand the language barrier problem. I was wondering if there was a reason why the UAE seemed to take to Flickr particularly.

I don't see anything extremely Arabic friendly here... Flickr does okay with unicode, but many interface elements assume a LTR language. And even then, why just the UAE? Why not Saudi Arabia?

I wonder if it comes down to: some person in the UAE, with a lot of friends, got Flickr one day.
jurvetson Posted 20 years ago
maybe eFatima? She was my first contact in the UAE hub.

Early seed users can make a huge difference in a period of exponential growth. We saw this in Hotmail's early adoption in Sweden and India (quickly growing to 4 Hotmail accounts per modem), and Skype's early adoption in Poland, etc.

I tracked Brazil's explosion on Orkut. Brazil is 66% of Orkut today, from almost nowhere a year ago. Orkut membership started with Google’s U.S. employees, so it was U.S. centric at first. Today, Orkut is 10.7% U.S.

Brazil grew from 200 thousand users to 1 million in just 2 months (August 04). Iran was a late entrant, not even on the top 10 list of countries in May 04. In one week it became 3% of Orkut, and grew to 7.7% in the next 6 months. UAE has never been on the top 10 list at Orkut.

On a per capita basis, the penetration is even more interesting. Estonia is number one, with over 3.3% of its entire population on Orkut. Brazil is number two, with 1.5%. Iran is number three, with 0.5%.
Gwênlyn Posted 20 years ago
striatic has recognized another cluster - see ribena's graph for a view of Brooklyn-in-Flickr.
brawny door [deleted] Posted 20 years ago
Thanks Gustavo. I love looking through all of your statistical analyses of flickr.
Gwênlyn Posted 20 years ago
A new view of the main network.

sc-40 GustavoG

In the previous work, I used a "mutual contacts" cutoff for inclusion in the network, otherwise it's too much data to process and the result is too tangled.
This time, I used a "friend of a friend" approach to maintain those (unidirectional!) contact lines that are "most supported" by the rest of the network. The result reveals more clusters and substructure than before.

The UAE cluster is amazingly interconnected, and in this new view it is in fact the most populated cluster.

Not surprisingly, fubuki's position is *very* central: not only connected to many people, but also his friends are well-connected, strengthening these ties.

The "main" cluster, which as striatic showed includes a "fotologgers" subcluster, is now resolved into these two components. A version showing network distances from Buntekuh illustrates the fotologgers subcluster, and the distances from Special highlight the "main" cluster.

The "bloggers" and "GNE" clusters are very well resolved, with Caterina and Stewart between them. (In a version with a slightly higher stringency, C+S are the only links between the bloggers and the rest of the network.)

There are additional subclusters and neighborhoods - feel free to annotate. :)
admin
striatic Posted 20 years ago
looks like i'd better brush up on my arabic ; )

one of the things i love about flickr is that, unlike other sites like fotolog, both language groups seem to be very able to peacefully co-exist .. and have peacefully co-existed for some time now.

i'm not sure if only a vocal minority over at fotolog complains about the brazillian population on that site, or whether they are justified in their complaints or not .. but i haven't heard anybody ever complain even once about the UAE folks .. and i haven't heard any of them complain about the primarily english crowd.

to the contrary, there are many examples of collaboration and friendship between the two clusters, even though the two groups {overall} are clearly distinct and largely autonomous.
Gwênlyn Posted 20 years ago
I'd think the deep separation only "helps" in that sense. I've seen quite a few complaints about not understanding the German in some groups (like ratebild-guess).
admin
striatic Posted 20 years ago
interesting .. i've seen quite a few german speakers on flickr .. but there isn't a distinct "german cluster", is there?

is there one if you lower the cut-off?
Gwênlyn Posted 20 years ago
I'd guess the "fotologgers" cluster has a good proportion of Germans, but they intermix and socialize so nicely, you couldn't tell. :)
In Memoriam: Jakes_World Posted 20 years ago
just to add, thank you G_G for this. very cool, and I even found my name :)
dont get the "may offend" part though. There are MANY ppl posting non-photo images..
quas Posted 20 years ago
This time, I used a "friend of a friend" approach...

Sorry if I'm being dense, but what exactly is this "friend of a friend" approach?

Great graphs, as always.
Gwênlyn Posted 20 years ago
what exactly is this "friend of a friend" approach?

It's an algorithm I made up today and I don't have a better name for it.
The network is too dense, too interconnected. I need to "trim it down" to be able to show something meaningful. So far, I had used a "mutual contacts" approach, i.e. if "A contacted B" and "B contacted A", the A-B relationship is mutual. If A has at least 50 such relations, it's included the mc-50 graph. This works fine, but I thought it might be obscuring some of the higher-order network structure.

In the "friend of a friend" approach, I first test every single contact relationship in the network, and decide whether to keep it or not. Let's say I'm testing the "A contacted B" link. To do this, I identify additional contacts in the form "A contacted C, and C contacted B". If I find at least 40 such "friend of a friend" links, I keep the "A contacted B" for the sc-40 graph. (The "sc" stands for "supported contact".) If not, the contact link is removed. After removing all such "unsupported" links, nodes not connected to anything else are removed from the graph.

Densely connected areas like the UAE cluster tend to remain largely unmodified. Rare links between such clusters tend to be dropped. This unravels the network into its better-connected components.
rvacapinta Posted 20 years ago
Interesting.

That would "penalize" people who make contacts all over the network and promote people who keep their friends within a cluster, right?

So, to find the Flickr geography, you're removing the effects of "airports" and highlighting the local roads.
Gwênlyn Posted 20 years ago
That's quite correct. A more advanced way of depicting the network would be to start with the latest graph I did, and add to it the "long distance networkers" later, once the general layout is already established.
quas Posted 20 years ago
Wow, that is really interesting. Why 40, though? Just that it showed the clusters clearly?

Oh, another question: what software do you use to build these graphs?
Gwênlyn Posted 20 years ago
Forty is an arbitrary cutoff. It's a balance between a skinny graph and too large a graph to process and display (and be able to save to a readable, limited size gif).

I used a Perl script to obtain the data (using the API), another Perl script to do the analysis and produce a file suitable for importing into yEd, in which I did the layout and coloring.
Just_Tom Posted 20 years ago
So just to clarify: in what you have called sc-40, if A and B are mutual contacts, they are only drawn if they have 40 shared contacts?

I have disappeared from this one, because my friends don't all stay and make each other contacts once I invite them :(
Gwênlyn Posted 20 years ago
There's no requirement for anyone to be mutual contacts.

If A contacted B, that link will be drawn only if A contacted forty other people that also contacted B. Perhaps I should relax it to allow more contact directions (e.g. if A and B contacted C).

Again, this is not a comprehensive "who's who" but an attempt to unravel a highly connected network into modules.
Just_Tom Posted 20 years ago
OK, I understand.

Not a criticism - I think this is one of your best (most informative) yet, and I don't consider myself a "who" in any case :)
Gwênlyn Posted 20 years ago
mc-50 with icons

Added the icons to the graph.
Work in progress... feedback is welcome.
hundrednorth Posted 20 years ago
Oh my god!
Norby Posted 20 years ago
Hey Gustavo - I just had an idea (you've probably already had it, but just haven't had time to make all of your dreams into reality yet). Is it possible to overlay multiple graphs onto the same image (with nodes in the same location) such that you can layer mc-25 at the bottom, with mc-50 on top of that, mc-100 above that, then mc-300 last, with each layer in a different color? That way you could get a little perspective (3D-ish) of the various strata as the # of MCs rise ... maybe just take the uber-graph and color each node by their MC count from one end of the spectrum to the other?

-/\/
Gwênlyn Posted 20 years ago
I hadn't thought about that.
Technically possible... that would be quite a challenge, aesthetically! :)
admin
striatic Posted 20 years ago
you could do it by opacity instead of colour, i suppose.

also, i wonder what a 3 dimensional model would look like.

probably not that different, but might help find structure in the areas with more density.
Gwênlyn Posted 20 years ago
The Flickr social pancake

A new look for the Flickr social network!
admin
selva Posted 20 years ago
That's the best thing I've ever seen, Gustavo. :)
brawny door [deleted] Posted 20 years ago
Rotflmao!!!
txkimmers Posted 20 years ago
wow it even has the wolverine-chewed right ear, just like the other one!
zalgon Posted 20 years ago
Gustavo: you're the funniest!

Ears aren't quite big enough, unfortunately.

B-
HyperBob Posted 20 years ago
It looks like a yeast cell budding
admin
selva Posted 20 years ago
Visualizing the social network:

zenera Posted 20 years ago
Really clever GG:)
Gwênlyn Posted 20 years ago
In case it wasn't obvious yet - the network graph with icons (or some variant of it) will become a printed poster.

Are you happy with your icon, as included in that initial run? If not, you can override it by adding the desired image to the FlickrIcons group.

Due to the timing of that previous graph, there are several "bunny ears" icons in it... Your choice. :)
evanescent memory [deleted] Posted 20 years ago
You do amazing work, Gustavo!

Sorry I am late for this disussion, because part of the German Fotolog cluster has visited me in the last weeks, and together we visited another Fotolog icon.

In other words: /buntekuh met /pandarine and they went to see /latenightowl. As Striatic mentioned, we are mingling very nicely - in real life as well! ;-)

Needless to say, we had a GREAT time!
brawny door [deleted] Posted 20 years ago
OOh great! I'll buy a poster and display it with pride over my computer (time to take down the "10 Years of WIRED" poster anyways).
phlezk Posted 20 years ago
i really dont understand it but that's ok
Gwênlyn Posted 20 years ago
Take two of the poster:

mc50 poster take 2

Again: if you don't like your icon in this poster, please add a replacement image to the FlickrIcons group. You can use a note to define which region of the image to use as icon. See the group for more info.

Feedback? Ideas for improvement?
styler* Posted 20 years ago
is there a plan to get this printed?
i'd love to have one
Gwênlyn Posted 20 years ago
Yes, this will be the fourth poster jbum hinted at.

There's a head-count thread for those that want to obtain one.
admin
krazydad / jbum Posted 20 years ago
This is the previously announced "mystery poster" in the set of four posters I am printing in April. I'll post an announcement when the ordering page goes up, in a few weeks.

The complete set:

styler* Posted 20 years ago
YAY!

you guys RAWCK!
admin
emdot Posted 20 years ago
jbum, i'd love to get this set of four, but am already signed up for the SqC poster... does that mean i'll get two of the SqC posters?
i_am_curiouskiwi Posted 20 years ago
emdot, I believe that the threads asking for "head count" on the posters was just for a general guideline. Once jbum has the posters printed (numbers based on those guidelines) then he will be setting up an ordering system for you to order the exact posters you want.
admin
krazydad / jbum Posted 20 years ago
That's right. The headcount is just so I can get a general estimate of how many to print. I'll be taking orders (for realsies) in April, via a website with an order form.
Gwênlyn Posted 20 years ago
Take three of the poster, now based on new data (April 9th):

mc50 poster take 3 - april data

Yet again: if you don't like your icon in this poster, please add a replacement image to the FlickrIcons group.
Gwênlyn Posted 20 years ago
A different view, based on the "supported contacts" algorithm:

sc20 network, April data, reduced

I like this better since it resolves the network structure some more, reveals additional "galaxies", and includes many more people.

Which one would you prefer for the poster?
Withdraw Posted 20 years ago
I like this one much better. Less clutered, easier for the eye to make distinct stops and notice the "zones".
Ranabass Posted 20 years ago
I am staggered. This is SSSOOOO cool!

Thanks so much, Gustavo!

PS -I was curious to notice how readily I saw the other "frog" icons in the image. Are there really alot of them? Or, do I just notice them because I like frogs and have had a frog icon(s)? Hmmm...
admin
striatic Posted 20 years ago
it also shows how amazingly interconnected the UAE cluster is .. and txkimmers yoga chat group is highly visible!
helpful insurance [deleted] Posted 20 years ago
Yoga chat?! Hoo Ha! How does that work?
admin
striatic Posted 20 years ago
you'd have to ask txk.

i think they talk about how creaky they are before stretching.
cobalt123 Posted 20 years ago
I would love to order poster(s) of this graphing! I do have a point re: a question about the connections and inferences. I always add a contact and mark them as a friend. This seems less complicated to me than segregating out folks. Plus, if I have chosen a member as a contact, it is because I have found a very distinct connection within me to their images. Now, according to the various methods of determining interconnections, wouldn't this simple choice to list all as contacts/friends change some of the calculations? I only wish that I had joined months ago, so I could be in the graphs too! I am delighted to see so many of my favorite members and contacts in these graphs.

Another simple question - I am a blogger and have a badge on one blog and post often a post at a time on another. Would this put me into the blogger group? I'm also a longtime member of fotolog.net, so does it also put me there or does one just fit into one main group at a time?
Just_Tom Posted 20 years ago
Can you post a list of people who are on each? I can't spot myself on the sc20 one, so I'd have to say I prefer the mc50 one (even though it's not as well-clustered) for purely selfish reasons!
cobalt123 Posted 20 years ago
Gusatvo,
Just in case, are you at all interested in creating a commissioned graph? I am poor in money but rich in friends. This would mean alot to me.
admin
striatic Posted 20 years ago
Another simple question - I am a blogger and have a badge on one blog and post often a post at a time on another. Would this put me into the blogger group? I'm also a longtime member of fotolog.net, so does it also put me there or does one just fit into one main group at a time?

the graph isn't created based on what you do, it is based on who your contacts are and who their contacts are.
cobalt123 Posted 20 years ago
Then how does GustavoG separate out the bloggers from the other groups? If you list a blog, does that make you in the blogger group?

And, back to my question about those such as I who mark all contacts as friends, which denotes a higher interconnection level, right?
phool 4 XC Posted 20 years ago
I feel silly asking this, but since I couldn't find it elsewhere, this seems as good a place as any to ask. Does anybody know how many active flickr accounts there are?
Gwênlyn Posted 20 years ago
I should post a list somewhere... 2362 names...
I'll implement an override method to keep in people that clearly are interested in being included in the graph. :)

cobalt123, I don't separate out the bloggers. My script obtains the contact lists and after some massage and extra software, I produce a network layout that reveals clusters. The clusters arise from the interpersonal connections.
Then, we look at the clusters and make sense of them. One of the clusters includes mostly bloggers, as recognized by väcäpinta, another includes GNErs, staff and oldtimers.
That's the beauty of this analysis in my eyes: the underlying network structure revealed in it reflects a bit of Flickr's history and population structure, and this wasn't pre-imposed as part of the analysis.
cobalt123 Posted 20 years ago
I am very interested in all this. It's the kind of "systems" my sort of mind likes to play with, although I haven't the mathematics logic to sort effeciently. Get lost in the artistic tangents, and then my ADD brings about lava lamp flickr goops to play with. I still wonder how I would graph out, especially since I tend to collect quite a random assortment of contacts and friends. Hope to be standing near enough to y'all to catch more of this good stuff as you go. Thanks!
admin
striatic Posted 20 years ago
I tend to collect quite a random assortment of contacts and friends.

you're probably somewhere in the main cluster then.
Just_Tom Posted 20 years ago
I should post a list somewhere... 2362 names...


Or perhaps render the output using html (or pdf or postscript), then we can search it. I can help with this at the weekend if you'd like?
cobalt123 Posted 20 years ago
*bump*
Gwênlyn Posted 20 years ago
Okay, a tentative final version of the poster is available.

poster preview

This is a scaled-down preview, and there's also an ugly list of names of people included. (If you have some weird character in your name, try searching also by NSID.) The numbers are coordinates, and will help build a locator application. If you divide your first two numbers by five, those are your X,Y coordinates in the 20% preview image, with (0,0) being the top-left corner.

Please note that the lines appear to be too light/bright, but in print they will be darker.

I'd like to call this the final version. if someone has any major complaint or suggestion, there's still a bit of time to influence the final result. :)
Lynn Morag Posted 20 years ago
So many names!! What a huge amount of work this must have been!
cobalt123 Posted 20 years ago
I'm sooooo excited that I made it in! Is there a way to use the coordinates you list in the table of names to figure out where the heck you are on the graph?
cobalt123 Posted 20 years ago
Ok, I found myself! Good grief! Would it be a bad thing as in rude and disrespectful if I put a little bit of the graph into my photostream? I sure would not do that if anyone thinks it would be improper. I guess the first one to ask is GustavoG.....?
Ron Diorio Posted 20 years ago
That's great work!

A galaxay for sure.
Gwênlyn Posted 20 years ago
I just changed the license to CC's ShareAlike.
aqui-ali Posted 20 years ago
just out of curiosity: have any conclusions been made about the clusters?

I've noticed a lot more mini clusters as well as the original distinct 'mickey mouse' shape, which has become much less obvious. Are more nuclei forming due to the rapid expansion of Flickr, is there a tendency for smaller, more compact and more separate groups to form as more people pour in?
Gwênlyn Posted 20 years ago
Keep in mind that this is not the same algorithm that produced the initial four-cluster shape. This algorithm is designed to "unravel" the network some more. The resulting lower connectivity level also allows me to include many more people, in turn revealing additional, smaller clusters.
aqui-ali Posted 20 years ago
I see. I suppose this one obviously reflects much better the current makeup of flickr. The independent small clusters are very interesting, as they are quite detached in some cases, often having only a few links to the main core. I would guess these are groups of friends who are more interested in sharing images between themselves then generally spreading around flickr?

It's also interesting seeing both Selva and Striatic as major connections between the GNE's and the large core cluster.
cobalt123 Posted 20 years ago
Absolutely fascinating! Because of you, GustavoG, I did some of my own pattern search among my favorites (500 saved from other member photos) and my contacts (100). There was little if any correlation between the two sets. This surprised me. Also narrowed it down by the member/photo and found that there are 7 flickr members who have a profound influence on me when I view their work - enough that I save favorites from these 7 more than anyone else. And I only was conscious of 3 of these 7 that I was saving more of their images than others!

This whole process was really interesting to do. I'm not a math person at all, but I do know how to use NoteTabLight and Word to do all the collection, trim off the html, and finally to sort.
Gwênlyn Posted 20 years ago
Sounds good!

I would encourage you to learn even just a bit of programming - no special math knowledge needed - to leverage what you just did. I can assure you that you'll enjoy it! A little Perl can go a very long way.
cobalt123 Posted 20 years ago
A little Perl- a beautiful gem made over time! I am ok on basic html and trying to get the hang of css and java now. I imagine I will get further along in the next year or so. It's so much fun to be amongst enthusiastic folks, and all the artists - wow! 2 months in flickr and I am in a whole new orbit, let alone the flickrverse, ha ha!
Metahari Posted 20 years ago
neat poster. is there a thread with more information on the poster somewhere?
Gwênlyn Posted 20 years ago
Lachie has built a very nifty People Finder for the poster! Check it out!

Note: if at first you only see "People finder [+]", wait a bit longer, until the poster image loads. Then, click on the [+].
admin
striatic Posted 20 years ago
that .. is just extremely cool.

great interface.

when i combine it with type ahead find, i can find people really quickly.
drp Posted 20 years ago
The geek shall inherit the earth...
sbpoet Posted 20 years ago
Wow.
AdaMacey Posted 20 years ago
krazydad.com never works from work. Can someone resolve the IP for me?
cobalt123 Posted 20 years ago
Omigosh! People Finder! This community never ever ceases to amaze me. Wow. Did I say "wow"? Yikes! Thank you Lachie!
cobalt123 Posted 20 years ago
Having some fun with Flickr Graph, the poster and the People Finder too. Here's one sample:
www.flickr.com/photos/cobalt/10701726/
kirstenpdx Posted 20 years ago
my brain is gonna explode. and i think i wanna have gustavo's babies -- in a theoretical way, of course.
Gwênlyn Posted 20 years ago
Lachie strikes again: he just produced a new version of the People Finder.

The main change? Click on an icon to identify the user!
brawny door [deleted] Posted 20 years ago
ARGH! This is STUPENDOUSLY OUTRAGEOUSLY cool! (*brain explodes*)

I've sent a brief post about this to The Social Software Weblog, you guys deserve wider credit for the fantastic work you've done.
1
(1 to 100 of 116 replies)