Is this the structure of New York City?

    Broadway as the spine is not difficult to believe.

    Data from the Twitter streaming API (10000 points, 30000 vectors). Base map from OpenStreetMap, CC-BY-SA.

    A set of the same technique applied to other cities

    Comments and faves

    1. ★

      straup, robertogreco, danhon, iamdanw, and 145 other people added this photo to their favorites.

    2. ♫ Lion ♫ (16 months ago | reply)

      Wow interesting as usual:O

    3. Matt the Farrell (16 months ago | reply)

      This is amazing. two things:
      You should sell large format prints of this, I would definitely hang this in my apartment.
      And
      I don't totally understand the data, is this the flow of people, or the flow of data? Gothamist linked me here, and I think they might have oversimplified the concept.

    4. Eric Fischer (16 months ago | reply)

      Thanks. I should try to figure out a good way to distribute prints.

      It is an attempt at plotting flow of people, in particular the paths from each place that someone posted a geotagged tweet to the next place that they posted one, and because I don't know anything about what they did in between, routed along the path in between that passes through the most other geotagged points.

    5. Matt the Farrell (16 months ago | reply)

      Brilliant. You could always do it through one of the microstock agencies, like istockphoto, where people could purchase a high res file and handle the printing themselves. Otherwise, there are printing places that people like portrait and wedding photographers use, where you upload the file, they handle all the printing and shipping to your customers, and you split the money.

    6. david (16 months ago | reply)

      Eric, I posted a few questions here: hello.typepad.com/hello/2012/01/is-this-the-s tructure-of-...

      Also, what time period were the tweets over? Were they randomly "spritzed" (do they still have that API?) or was it a solid chunk? Do you still have the tweets? Public & Private both?

      I love this map!

    7. Eric Fischer (16 months ago | reply)

      Public tweets, chronological order (but probably somewhat spritzed, since the API gives no guarantee of completeness) although randomized would probably have been better, beginning May 14, 2011, and I do still have them. Thanks!

    8. MapScience (16 months ago | reply)

      Nice work Eric. The visual display of intra-urban connectivity is fascinating. I think these will provide some grist for discussion among student in one of my courses. Thanks for sharing.

    9. david (16 months ago | reply)

      Thanks Eric - one more question - what's a "vector" in the context of the streaming API? I've never seen that.

    10. bior (16 months ago | reply)

      Although I haven't used it myself, from what I understand DeviantArt has a pretty straightforward mechanism for digital artists to sell physical prints, and DA takes care of all the printing and shipping.

    11. Eric Fischer (16 months ago | reply)

      Oh, sorry to be mysterious! By vector I just meant the motion between two locations implied by two successive geotagged tweets by the same person.

      Thanks for the DeviantArt advice!

    12. adamgdunn1 (16 months ago | reply)

      I like this one a lot. In fact, we use something similar in landscape ecology to look for the pathways most likely to be used by animals (and also seeds dispersed by animals). I've only recently blogged about some old work I did in the area (adamgdunn.com). The basis of a lot of the work is done on either (a) random walks, (b) weighted random walks (c) betweenness centrality, or (d) variants on one of the above. The pictures you've made are remarkably similar (but much more beautiful) than the ones I made for a relatively low-impact journal called Ecological Management & Restoration. Great work!

    13. Steven Vance (16 months ago | reply)

      The vertices and angles of the edges remind me of a Voronoi diagram.

      This was on the first page of search results:

    14. peter_capek (16 months ago | reply)

      Do you have any data on what the disribution is of the times between successive tweets by a single person? If the time since the last tweet is much more than the normal travel time for that distance, the person is likely to have been somewhere else, other than the most-tweeted path between the tweet locations. Would be interesting to see, if not too hard to do.

    15. paulheckbert (16 months ago | reply)

      To answer the question: no, this is not the structure of New York City. This graphic is somewhat bogus. As Eric Fischer says, he doesn't know the route people took between locations at which they tweeted, but this graphic suggests that he does. If someone tweets in Queens, say, and then their next tweet is from Brooklyn, it appears that what Eric is doing is fattening the path that goes across the Queensboro Bridge into Manhattan, then downtown to the Williamsburg Bridge into Brooklyn, even though that is most likely not the path they traveled. In that respect, the graphics are misrepresenting the data. It would be less misleading to simply draw dots at the locations of each tweet. -Paul S. Heckbert

    16. fake is the new real (16 months ago | reply)

      This is lovely. Congrats on discovering the NYC Subway system embedded in tweets. Can you add a subway map overlay?

    17. Eric Fischer (16 months ago | reply)

      Adam, thanks for the link to your work. I'm not surprised that the same technique is also applicable to other data sets, and I'll be interested to learn about your other techniques for predicting routes.

      Steve, I was thinking the same thing about the resemblance to Voronoi diagrams, although here it is routing through the points themselves rather than through lines separating them.

      Peter, I don't have the numbers on the time interval distribution, but it is frequently hours or days between tweets by the same person, so they almost certainly did visit other places in between. I think it is still useful to know what sets of places are visited by the same people, even if they also visit other places that aren't documented.

      Paul, thanks for pointing out the limits of the technique. Perhaps it would be better to say the most interesting route between two points rather than suggesting that it is the most likely.

      Fake, thanks. I'll see if I can find a geographic subway map to overlay.

      A set of the same technique applied to other cities

    18. fake is the new real (16 months ago | reply)

      MTA data: spatialityblog.com/2010/07/08/mta-gis-data-up date/

    19. /\/\ichael Patric|{ (16 months ago | reply)

      I followed the links from Planetizen to Fast Company Design to here. Very nice.

    20. Eric Fischer (16 months ago | reply)

      Oh, thanks—I hadn't seen the Planetizen one.

    21. Kyrion (16 months ago | reply)

      Very, very cool. Thanks!

    22. Emil Stenström (16 months ago | reply)

      No, he doesn't go the shortest route. He uses the shortest route _through as many geotagged points as possible_. That's not the same thing, and a better representation of the real path imho.

    23. jared (16 months ago | reply)

      This is incredible. It is so beautiful. The resemblance to a neuron structure is striking! Great work Eric.

    24. muggle1 (16 months ago | reply)

      WOW!

    25. www.alicialopez.ch (16 months ago | reply)

      Hi Eric,
      Would be a lot of work to apply the same technique to the city of Zurich, Switzerland?
      And yes, me too, I would love the opportunity to buy it in a big format.
      Thank you

    26. Uriel_Li (16 months ago | reply)

      Hi Eric, I'm new here. I was astounded by your maps a lot. Thank you! Because "routed along the path in between that passes through the most other geotagged points", so the wider path will supposed to be through the city center, where is highly populated. Am I right?

    27. Eric Fischer (16 months ago | reply)

      Thanks! I'll see what I've got for Zurich.

      Yes, the widest path will tend to pass through city centers, but the pattern of geotags seems to follow retail density even more closely than it does population density.

    28. Uriel_Li (16 months ago | reply)

      yeah, geotags may be generated around public and commercial points more. Someday maybe you could share the "twitter crawler" tools, it's really cool to do some urban pattern analysis via these data. Thank you.

    29. Eric Fischer (16 months ago | reply)

      There is no real crawler—I just connect to the streaming API at stream.twitter.com/1/statuses/filter.json and watch the tweets as they come in.

    30. j_roberts00 (16 months ago | reply)

      Hey Eric - this is very cool indeed. I was wondering about the algorithm for the assumed paths. Does it have some sort of shortest route finder between two points along the Open Streetmap roads? If so, Broadway's likely to be very heavily weighted. With New York on a grid, and Broadway the only diagonal through much of the city, the shortest path will almost always include some time on Broadway, even though people are likely to avoid it.

      I'm not sure how this plays with your algorithm to pass through other geotagged locations, but the unique nature of Broadway in New York versus other cities might be a source of bias. They are very cool maps - I'm really glad to have stumbled across them!

    31. Eric Fischer (16 months ago | reply)

      Thanks! It is not actually using the street network for finding paths, only the tweets themselves, so it depends both on the diagonal existing and on people going there to tweet from it.

    32. fdqps (15 months ago | reply)

      We would like to make a full wall print of this in our office. Is it possible to get a vector version?

    33. sointroverted (14 months ago | reply)

      Hi I have a completely unrelated question, is that base map you used a free resource? If not would it be possible for you to send it to me to use for a school project? Thanks for your time! I love the digital mapping.

    34. Uriel_Li (13 months ago | reply)

      it's openstreetmap i guess

    35. fdqps (12 months ago | reply)

      Eric we took your photo and made a wall of it at our office - instagr.am/p/Lm-J1xR8oe/

    keyboard shortcuts: ← previous photo → next photo L view in light box F favorite < scroll film strip left > scroll film strip right ? show all shortcuts