|
Search this group's discussions
|
'Second Life as a Platform for Augmented Reality'
|
Wow that looked AMAZING. I bet the hardware you need is expensive.
Very interesting! :)
Posted 53 months ago.
(permalink)
|
|
Yes, it is expensive today probably indeed.
But imagine in about two years orso, this kind of stuff might be as normal and affordable as watching tv is today.
Imagine the impact on society then... For example: If we don't have to risk our lives in physical traffic or pollute the enviroment by taking planes, when the shared experiences of tele-presence are getting good enough for replacing that kind of travel...
"Second Life is REAL" is not a slogan to prove how realistic the product 'Second Life' is today. No, it is about to prove how real revolutionairy impact these kind of developments are having on mankind, now and in the future. (Amongst other things offcourse)
People who think metaverses are 'just a game' and 'not real' are like those people centuries ago, who were convinced the earth was flat. ( i.m.h.o.)
Originally posted 53 months ago.
(permalink)
Ze Moo edited this topic 53 months ago.
|
|
The components components are:
4 sequenced ir inductors (80 each) www.smarthome.com/8120w.html?src=Froogle
A pair vuzix goggles, like the ones here (300 or so dollars): Vuzix goggles
a decent webcam (50 bucks)
Visual studio express edition, C# (to mod the functionality into second life) free
.Net framework (free)
duct tape (50 cents)
Project total: About 700 dollars and a lot of time.
---------------------------------------
Now I'll explain what they do, this process is typically called ir inductor sensing. Its purpose is to relay positional data to simulate perspective.
The IR sensor
The bar on top of the glasses in the video emits two beams, one on either end. You'll note that the avatar is in the same place in the orbiting video? That's where the IR sensor is. The IR sensor controls the perspective of the video relative to the position of the two points. Using these two points, you can judge not only distance and orientation to the sensor, but you can all tell distance, thus producing a shuttle and orbiting effect. As the points move and turn, second life receives positional data based on the origin. There are 4 such sensors, as outlined above, each covering 1 zone so you simulate a 360 degree experience.
The second video is a bit trickier, but here's how they did it. They essentially flipped the setup. They put the transmitters on the ground in a square setup. You'll note the avatar leaves the screen at one point. That is because the center of all four transmitters is the origin in this case. He stepped out of the bounds of his 'world' and therefore was no longer renderable on screen.
The Camera
The camera relays a video stream to second life, and second life uses interpolation data data from the IR sensor to position the avatar at the origin relative the video source (the video doesn't show this, it shows the avatar moving, when controlled by a person at a keyboard, and a recorded view from another avatar orbitting the first).
Essentially, they turn off all rendering save objects and the avatar. This turns the background black. They then patch the video into the background, immediately integrating the 'avatar' into the scene. The glasses are there merely so you can view all this in a first person fashion. I wouldn't recommend wandering around with video goggles that obscure your vision.
This is similar (near identical) to the projects John Chung Lee of Carnegie Mellon does with the Nintendo wii's tracking systems (in fact, they use the same ir inductor the wii does).
--------------------------------------
Reference videos of what it really looks like: www.youtube.com/watch?v=5jtw4GHNnz0&NR=1
What this video shows us is how they simulate the 'interaction with the enviroment.'
They built a copy of their test environment in the real world, in second life. They then remove objects aside from the shelf, so only the avatar and shelf are rendered are rendered in game, and the video is rendered as the world background instead of black. The first shot has a fake shelf, the second has the edit menu open. That is because they're using the hide selected debug option to make it appear as though the object isn't rendering, while still making use of the object's occlusion (masking of the avatar).
---------------------------------------
All that said, none of this is easy. The people who did this put a ton of time into it. It is not extremely cost prohibitive, and can easily be mass produced, there are just a few issues that prevent this from being hugely done, as you can't have two competing IR systems in the same physical area work efficiently.
Edit:
www.youtube.com/watch?v=l7xdIcRYfBE&NR=1
Here's an overview of the technology this stemmed from. One of the things they use in a lot of their demos is the paper registration points, they're a viable option as well, but require a lot of processing power and a better camera than ir sensors, but allows a registration point to be determined without the use of them.
SubEdit:
Another application using a very high horsepower method of the same technology:
www.youtube.com/watch?v=tErauQT6F6A&feature=related
Originally posted 53 months ago.
(permalink)
~Kitty edited this topic 53 months ago.
|
|
|
Wow! What an impressive reply Katarina! Thanks!
Originally posted 53 months ago.
(permalink)
Ze Moo edited this topic 53 months ago.
|
|
Hi Katarina and others! Someone pointed me at this discussion, so I thought I follow up on it.
I'm Blair MacIntyre, the professor at Georgia Tech who supervised Tobias Lang, the student who built the AR interface to SL that you are discussing. Toby was a student at LMU Munich, visiting me at Tech for 6 months to do his thesis.
Since there seems to be some interest in how we did it, I thought I'd just tell you, since there's nothing too secret about it. :)
Katarina, you're guesses are interesting and close, but not quite right. I won't comment on if those ideas would work (they might), but just tell you how things really are done. The big comment is that we tried to avoid using any "hacks" that would get in our way; we really can move around in a 3D space with a see-through head-worn display on and see SL blended with the world around us.
The one thing you are right about, of course, is that it was a vast amount of work; Toby did a fabulous job. What he did is add 3 pieces of technology to the client:
- live video. We used our videowrapper library (http://sourceforge.net/projects/videowrapper), that lets us capture video from a variety of cameras. We tend not to use webcams; while cheap, the video quality and speed tend to stink. In this project, we used Point Grey Firefly MV cameras (www.ptgrey.com), which don't have much better video quality, but do run at 60 frames per second. Using their native API is much faster than DirectShow on Windows. Plus, you can get better lens (as you Flickr-ite's will know, imaging is "all about the glass") that the plastic garbage on most web cams.
- we added support for VRPN (http://www.cs.unc.edu/Research/vrpn/), a hardware abstraction library we use to access a wide range of devices. In the mobile demos (where we are looking around a lab) we are using an Intersense IS-1200 hybrid tracker (http://www.intersense.com/products.aspx?id=45), which tracks its position and orientation precisely over a relatively large area using those circular markers you might be able to see on the ceiling in some of the videos.
- we also added support for feeding the video through the ARToolkitPlus marker tracking software (http://studierstube.icg.tu-graz.ac.at/handheld_ar/artoolkitplus.php) and using that to know where the camera is.
So, that was the first step: putting the technology into the system.
Second was making it usable. Toby implemented a range of modifications to the client that are all controllable by tools in the SL world (basically, object with scripts that send messages to the client). This was intentional and really really cool -- all the setup and configuration is stored in-world, in objects there, so you can save you current client configuration and restore it later. This lets you move between different setups (home and lab, for example) with ease. It also means there are no external config files.
This the in-world tools do include
- the ability to specify which area of the SL world to show overlaid on the camera view, and how it aligns with the physical world (e.g., how the tracking system coordinates relate to the SL coordinates). We use low level OpenGL (the 3D library that SL uses to render the world) operations to control this additional clipping away of the parts of the world beyond the area that aligns with the physical world.
- the ability to control which part of the world (terrain, avatars, sky, objects,....) are shown. Different applications and demos have different needs. You can select the various bits using the tools, and the client renders them, or not.
- the ability to hide some objects visually, but still have them occlude.
Each of these is done by modifying the client to simply "do it", rather than using ad hoc methods like fooling about with debugging modes and comments. Hiding objects was a bit tricky to come up with a solution for, since we couldn't add properties to the objects in world. So, we use a kind of chroma keying (similar to green screen removal): we pick a specific RGB color, and any objects that have that color are not rendered visually, only used for hiding. Since all objects are typically textured, this is not visually noticeable to other SL visitors.
As for costs, it will be cheap eventually, but not yet. :)
-> The cameras we use are $200 ... not bad, and a reasonable price for a 60 fps camera
-> The IS-1200 is the kicker; it's currently about $9000, although it could probably fall to a few hundred if they could sell them in high volumes. They are usually used for research right now.
-> The display you point at has too small a field of view. We currently use an eMagin Z800, which is a bit better (40 degrees diagonal), but not great. $1500.
-> We built the mounts for the camera and mirrors and tracker, to attach to the display, using a stereolithography 3D printer; very little cost once you have the machine (but pretty pricy otherwise).
-> we use a lot of gaffer's tape, not duct tape; a bit more expensive, but no gunky residue. :)
-> All the programming was in C++ (that's what the client is written in), not C# (would be too slow, anyway). Visual C# Express is also free.
HOWEVER, if you only want to the things where graphics are sitting on the paper markers (square patterns) you don't need the display, the IS1200, the 3D printer or the more expensive cameras; a computer and a web cam will do just fine. No other h
The DART (Designer's AR Toolkit) video you point at shows an earlier authoring tool that used similar technology and had similar functionality. It was built on top of Macromedia Director, instead of integrated into something like SL.
Given all that, I have to say that merging SL with the real world is unbelievably cool and inspiring to work with; myself and my students have created lots of AR tools and systems over the last 15 years, and creating things is SL is completely different. Simply put, it's much more like building "things" in the real world than like "programming", even when you are scripting. So many of the benefits of working on real stuff on a desk, like being able to leave something partially done and come back to it, or having friends be able to watch, learn or help, is the same.
In the end, Toby presented his master's project in-world, in my theatre there, and had his committee simultaneous attending from Santa Barbara, Atlanta and Munich. We had a variety of other visitors attend from all over the world; Toby was displayed in the AR view the whole time and did demos during his talk. The presentation setup is still there in our theatre. :)
Anyway, if you want to see or chat more, my SL avatar is Blair Potluck. Our island is "Augmented Reality"; come check it out.
blair
ps. folks who see this as the future of the web are dead-on right. SL has its problems, but it is a pointer to the future. It is to the future 3D web as all those old closed hypertext systems were to the WWW, and when someone creates and open and interoperable "SL", look out world! :) Being able to blend the spatial 3D world over top of the real world is inevitable, and has been in sci-fi or years (see a number of cyberpunk books, like Virtual Light and most recently Rainbow's End). It's fun to be dabbling in it.
pps. Sorry for the length, but I thought you might want to know how it works. :)
Posted 53 months ago.
(permalink)
|
|
|
Awesome awesome, the parts I illustrated of what you need to do this are based on my own toyings, boiling it down to a mechanism that the common user can try if they absolutely wanted to, trying to built interest in the feasibility of the technology. I tend to suggest things that are free so I don't like scare people away by saying: 'all you need is a boxx apexx 8, an nvidia tesla, and Visual Studio teamsuite,' like the setup we use at my office.
I've done a few things at my company with AR for demos, haven't had opportunity to try and integrate it with second life yet. Our project actually ran Direct X 10, Direct 3d, and C# (due to time constraints, had I had the development time I would have gone modular C++), essentially using the sheer horsepower of the system to get around the limitations of the framework. We opted to use RFID for our positioning, as we already had the equipment in house. I'll certainly look into the camera and printing technology if I have a chance to get the project going again.
Originally posted 53 months ago.
(permalink)
~Kitty edited this topic 53 months ago.
|
|
|
Thank you very much Blair MacIntyre for this awesome reply here!
I am very much impressed with all this!
Only I have a fundamental point of critisism:
I quote: "merging SL with the real world". This is not being phrased right i.m.h.o.
Now if we don't start phrasing this right consistantly, (most people make this mistake I think) I feel like it will limit our imaginations and solutions and possibilities of where we are heading in the future.
( it is also not easy for me to phrase this, as english is not my mothertongue)
SL _IS_ part of the real world, as I see it. So it cannot 'merge' with it, when it is already real.
I prefer the word 'physical reality' (some say 'meatspace') as the part of reality that virtual reality has to merge with.
This 'quest' of mine, is one of the main reasons why I created this Flickr group 'Second Life is REAL!'
Opposite to other Flickr groups, where a picture groups have discussions 'attached', I would prefer a textual discours with pictures 'attached' :-)
(By discussing this, I hope I can improve my english on this as well)
Originally posted 53 months ago.
(permalink)
Ze Moo edited this topic 53 months ago.
|
Would you like to comment?
Sign up for a free account, or sign in (if you're already a member).
|
|
Second Life - Is REAL?! /
Discuss
Feed – Subscribe to Second Life - Is REAL?! discussion threads