Oculus Developer Forums

Tutorial: Low cost FPV setup for the rift (130ms latency)

A forum to discuss development for the Oculus Rift.

Tutorial: Low cost FPV setup for the rift (130ms latency)

Postby Grix » Sun Jul 07, 2013 11:54 am

EDIT/UPDATE: Read post 5. Basically, latency has been reduced to only 130ms!


Yesterday night, it finally happened: I got my rift! And one of the first things I tried was flying FPV with my RC quadcopter. I've been wanting to do this since I heard about the rift last year, and I've been researching how to get FPV working since. And now I finally have the confirmation: Although it's not perfect, it definitely works! Basically, what I do is convert the analog signal from the video receiver to an USB webcam video, and feed that to the rift with proper duplication and warping.

What you need:
- An Oculus Rift
- An RC vehicle
- Standard FPV gear (Camera, video transmitter and receiver)
- An RCA to USB video capture device (I use this: http://bit.ly/12daZIR)
- A laptop
- A 12V battery
- 5V voltage regulator (or 5V battery)
- A power cable you don't need that fits the rift
- RCA cable
- Optional, but recommended: A customizable OSD (I uses minimOSD)

First, setup your FPV system if you are new to FPV. I won't cover that in this tutorial, google it. There are however a few things to note: For the rift, a high FOV camera is obviously preferable. I currently use a GoPro which has 110 degrees which matches the rift pretty perfectly. I have however ordered a different camera for reasons I will explain later, and I went with this: http://bit.ly/11sR7Tk . It has roughly the same FOV.
If you are using an OSD, which is really recommend that you do, you'll probably want one that lets you customize the layout, like minimOSD. Otherwise the text may be too hard to read due to placement or size. This is my layout in minimOSD, everything is perfectly readable: http://pastebin.com/qvrGLxya (Save as x.osd and import)

Another note about the capture device: Ebay is riddled with dirt cheap "easycaps". I recommend that you avoid them, because they are most likely fake and will lead only to problems. I first bought one for $5, but it was useless for my purposes, it used some obscure codec that only the program included in the disk would recognize. Fake devices also have worse image quality and latency. As said, I bought this one: http://bit.ly/12daZIR . It uses the same chip as many higher end capture cards, and that site provides proper and updated drivers for download.

After you have installed the capture card drivers and verified that it works, download this program: http://lab.neosolve.de/RemoteEyes/RemoteEyes.zip
(Credit goes to this guy: https://www.spacefish.biz/blog/ ).
It should automatically detect the camera and start outputting in rift format. Press f to go fullscreen and esc to exit. If like me you have a webcam integrated in the laptop, you need to disable it in device manager, because the program won't let you change the camera it uses, it just picks the first one it sees. I've tried to get the guy to put the source on github or something but he hasn't responded. Out of the box the program has wrong warp parameters. Open the file fshader.txt and replace it with this: http://pastebin.com/57YkYYfa . Feel free to experiment with params, I sacrificed a bit of FOV for better overview.

If you turn everything on you should now get the image from the FPV camera to the rift! But the setup is not yet portable. You need to make the rift and video receiver battery powered. For this I soldered together a custom cable: On one end it has a connector for a 12V liPo battery, I had a few of those because I use them on my quadcopter. The cable splits, where one end goes directly to the video receiver with the proper plug, and the other end goes to a 5V regulator with a small heatsink, and then to the rift (I had a spare cable with a plug that fit the rift control box, so I soldered it on).

You might be tempted to run the rift off the laptop's USB power. In fact I tried this first, but after five minutes the computer bluescreened due to too high USB power draw. Luckily the quad was on the ground, if that happens when you are in the air, you're screwed. Also, make sure to disable any power saver settings on the laptop, the program does not stop the screen from dimming after x minutes, and that would also be terrible if it happened while in the air.

So how well does this work? There are a few problems, the biggest is latency. Imeasured my system to around 0.25 seconds. It's flyable, especially in open areas, but it can get a bit annoying. However, my gopro camera is partially at fault, the live feed has a slight delay in itself. That's why I ordered the other camera I mentioned above. Latency is 130ms with the new camera. Much better. Here's a demonstration video: http://youtu.be/6rTkuO_44S8 .
That, and the fact that the image quality is much better, especially in low or high light conditions. The controller radio can also be a factor, I use a very cheap radio (Gotta upgrade that some day), and there might be a delay between me moving the sticks and the motors responding, which would also affect the total latency. Another factor is the software, if someone makes a program that really focuses on low latency, it should shave a few additional milliseconds off. I bet that with proper equipment the latency can definitely become acceptable. Though in fact, I would say it already is acceptable. I tried flying for a few minutes but crashed, but I think this was because it was in a small, bulky area, and this is the first time I've ever flied FPV, so I just need some practice in better conditions.

Another problem is image quality. This would depend on the camera, warp parameter and capture device, but in my case it is not that great, but of course, this is nothing new to FPV. PAL is the highest resolution one can transmit currently, and that's obviously not great. The rift has a limited resolution as well, of course. But again, it's definitely flyable, and I think the image quality will improve a bit when I get the new camera as well.

Two missing features are 3D, and headtracking. Both of these are possible to implement, using a gimbal and two cameras, however, it would be much more expensive. Perhaps someone else has the dough to test it out? :)



Here are some pictures:
http://i.imgur.com/o3Gg366.jpg
http://i.imgur.com/K6KBA9c.jpg
http://i.imgur.com/bsWR9Zq.jpg
I can take a video of a flight once I get to try it properly in an open area.

One a sidenote, I'd like to link to this article, another take on the rift and FPV: http://intuitiveaerial.com/home/2013/6/8/oculus-rift-fpv
These guys are doing it a bit differently. They have two cameras for 3D, and better latency, perhaps better resolution as well. However, I believe there is a few downsides compared to this tutorial:
- It's much more expensive, of course.
- It looks like they are using a digital wireless link. This allows for things like better bandwidth, but it also vastly reduces range, and you'd have to buy a whole new, expensive setup even if you already have traditional FPV gear.
- I'm not sure they would even give the community the details on their hardware and software unless you buy it, because they are a company their solution will most likely be proprietary.
Last edited by Grix on Tue Jul 09, 2013 10:13 am, edited 3 times in total.
Grix
 
Posts: 32
Joined: Sun Apr 28, 2013 2:58 pm

Re: Tutorial: Low cost FPV setup for the rift

Postby Sebbi » Sun Jul 07, 2013 3:30 pm

I'm interested in FPV with the Rift, but I hope I can get it to work with a Raspberry Pi or an Android tablet instead of a laptop. And i want the OSD to be drawn in the ground station ...

Do you plan on developing your own software to improve the state of FPV with the Rift?
User avatar
Sebbi
 
Posts: 85
Joined: Fri Mar 29, 2013 2:12 pm
Location: Germany

Re: Tutorial: Low cost FPV setup for the rift

Postby Grix » Mon Jul 08, 2013 2:29 am

Sebbi wrote:Do you plan on developing your own software to improve the state of FPV with the Rift?


Maybe, but I'm not that good at graphical programs, so I don't know if I have the skills.
Grix
 
Posts: 32
Joined: Sun Apr 28, 2013 2:58 pm

Re: Tutorial: Low cost FPV setup for the rift

Postby Medium » Mon Jul 08, 2013 5:16 am

This is the best idea i have heard since the major games got announced for the rift.

Very cool man! Great that you wrote a tutorial on the whole thing as well.

I am definitely going to give this a shot when my rift arrives. Do you have any tips for quad rotor shopping?

Cheers

Medium
Medium
 
Posts: 10
Joined: Wed Jun 19, 2013 1:38 am

Re: Tutorial: Low cost FPV setup for the rift

Postby OculusChris » Mon Jul 08, 2013 7:21 am

Grix wrote:Two missing features are 3D, and headtracking. Both of these are possible to implement, using a gimbal and two cameras, however, it would be much more expensive. Perhaps someone else has the dough to test it out? :)


Hi there,

i've planned it as part of my diploma thesis ;)!
3-Axis Gimbal (Alexmos http://www.armbgc.com/armbgc-v3-0-alexmos-controller-3-axis/) attached to a cablecam, controlled by the Oculus Rift Headtracker.

Whether or not 3D, I'm not quite sure yet...
Lifeview from a Canon, or this:
http://www.nghobbies.com/cart/index.php?main_page=product_info&products_id=393

For ideas and input i'm also grateful :lol:

Chris
OculusChris
 
Posts: 3
Joined: Mon Jun 03, 2013 11:32 pm

Re: Tutorial: Low cost FPV setup for the rift

Postby Grix » Tue Jul 09, 2013 10:04 am

Very good news, guys! The new camera arrived today, and boy does it make a difference! The latency actually HALVED! Measured now to 4 frames at 30fps, or ~130ms. Moral of the story: Do not use a gopro for live video.

Here's a video demonstrating the latency: http://youtu.be/6rTkuO_44S8

Only problem is, now my OSD is not working for some reason. Maybe I fried it somehow, that would be the second minimosd to fail on me.
Grix
 
Posts: 32
Joined: Sun Apr 28, 2013 2:58 pm

Re: Tutorial: Low cost FPV setup for the rift

Postby OculusChris » Wed Jul 10, 2013 8:00 am

Grix wrote:Here's a video demonstrating the latency: http://youtu.be/6rTkuO_44S8


Awesome! My Rift arrived today, so i can also start experimenting :P .

Cheers
Chris
OculusChris
 
Posts: 3
Joined: Mon Jun 03, 2013 11:32 pm

Re: Tutorial: Low cost FPV setup for the rift (130ms latency

Postby vande361 » Thu Jul 11, 2013 5:46 pm

Has anyone recorded 360 video with multiple GoPros for playback through the rift? Does this work???
vande361
 
Posts: 11
Joined: Wed Jul 03, 2013 1:35 pm

Re: Tutorial: Low cost FPV setup for the rift (130ms latency

Postby geekmaster » Tue Jul 16, 2013 6:05 am

"For the things we have to know before we can do them, we learn by doing them." —Aristotle
"The GREATEST discoveries shall be found in pursuit of diminishing returns." —geekmaster
Всё гениальное просто. Щедрость не имеет пределов.
User avatar
geekmaster
 
Posts: 2547
Joined: Fri Apr 12, 2013 8:07 pm

Re: Tutorial: Low cost FPV setup for the rift (130ms latency

Postby Felix12g » Tue Jul 16, 2013 11:12 am

Wow, that one drags a whole laptop up.
Code Monkey
User avatar
Felix12g
 
Posts: 138
Joined: Sun Apr 07, 2013 8:35 am
Location: Minneapolis, Minnesota

Re: Tutorial: Low cost FPV setup for the rift (130ms latency

Postby captain3d » Tue Jul 16, 2013 11:29 am

I have been flying 3D FPV for 18 months using the '3D Cam FPV' at nghobbies and love it. I think it has to be 3D if you are going to do it but I know many people will be happy with 2D on the large FOV. I see quite a lot of Rift youtube videos of games where the person does not realize it is 2D.

The Rift is a natural target for FPV flight and I have teamed up with Trevor the 3D Cam FPV designer for 'Transporter3D' that will enable the Rift to be used for 2D or 3D FPV. He made the first 3D rift FPV flight a month ago...

http://youtu.be/u74rf4ztJTI
http://youtu.be/M7WZXzhsFgY
http://youtu.be/xiqIPjAh0JQ

Still in progress but no laptop needed :-)

phil
User avatar
captain3d
 
Posts: 19
Joined: Wed Apr 24, 2013 7:39 pm

Re: Tutorial: Low cost FPV setup for the rift (130ms latency

Postby SkyDrone » Sun Aug 18, 2013 8:07 pm

We have just launched our Indiegogo campaign for Sky Drone FPV. A digital low latency HD FPV solution with Oculus Rift support coming: http://skydrone.aero/fpv-indiegogo
User avatar
SkyDrone
 
Posts: 15
Joined: Sun Aug 18, 2013 8:06 pm

Re: Tutorial: Low cost FPV setup for the rift (130ms latency

Postby Grix » Thu Aug 29, 2013 6:55 am

The guy that made the program has now made a new version, and best of all it's now open source!

https://code.google.com/p/remote-eyes/source/browse/
Grix
 
Posts: 32
Joined: Sun Apr 28, 2013 2:58 pm

Re: Tutorial: Low cost FPV setup for the rift (130ms latency

Postby feuerhahn » Sun Oct 13, 2013 1:34 pm

hey man, great stuff. I fly FPV since 2 years and was one of the first that got the oculus. At the beginning there was not much about rift fpv. I'll definetly will give it a try the next days. Any updates from your side? the last status is past a while.

cheers and thanks for your effort!
feuerhahn
 
Posts: 17
Joined: Sun Mar 31, 2013 4:40 pm

Re: Tutorial: Low cost FPV setup for the rift (130ms latency

Postby Grix » Mon Oct 14, 2013 10:00 am

feuerhahn wrote:hey man, great stuff. I fly FPV since 2 years and was one of the first that got the oculus. At the beginning there was not much about rift fpv. I'll definetly will give it a try the next days. Any updates from your side? the last status is past a while.

cheers and thanks for your effort!


I recently moved to a more densely populated area and so I haven't been able to fly the copter much unfortunately. However, many other people have been making progress here, with dual 3D cameras and headtracking setups. You can probably find a lot of info by googling
Grix
 
Posts: 32
Joined: Sun Apr 28, 2013 2:58 pm

Re: Tutorial: Low cost FPV setup for the rift (130ms latency

Postby Regulator » Sat Nov 02, 2013 9:30 am

What's the bandwidth usage on the sky drone? LTE I assume. I'm pulling about 3mb / min. Using my Verizon so that would give a suitable 28 hrs
Regulator
 
Posts: 1
Joined: Sat Nov 02, 2013 9:29 am

Re: Tutorial: Low cost FPV setup for the rift (130ms latency

Postby killerm1 » Thu Dec 12, 2013 12:17 am

Hi,

I have been trying this out and have found that fpv program for converting the video to not be so great. It just comes up all blury for me. I am using one of those fake easy caps u mentioned though. I played around a little and got it to work with vr player in conjunction with vlc. All you have to do is stream your capture device with vlc and then open the vlc process with vr player. Or you can open the stream with vr player aswell using "dshow://" as the url. Having vlc gives a lot more functionality aswell.
killerm1
 
Posts: 1
Joined: Fri Nov 15, 2013 9:35 pm

Re: Tutorial: Low cost FPV setup for the rift (130ms latency

Postby fatslugg » Mon Mar 10, 2014 7:43 am

I don't know if that might be of interest to anyone, but I've been doing some work to use the Oculus Rift as a FPV system. My first attempt at flying FPV with it was pretty successful (it was my first ever FPV flight so I can't compare much to other systems). Here are some links:

The flight
http://bitoniau.blogspot.fr/2013/10/first-oculus-based-fpv.html

And some more recent work on adding pan-tilt:
http://bitoniau.blogspot.fr/2014/02/pan-tilt-fpv-using-oculus-rift.html
fatslugg
 
Posts: 4
Joined: Wed Jul 10, 2013 2:58 am

Re: Tutorial: Low cost FPV setup for the rift (130ms latency

Postby fatslugg » Mon Mar 10, 2014 7:47 am

Oh and I forgot: I've also run some latency tests on various FPV setups including the Oculus Rift. It's about 130 ms as someone mentionned previously, so not great but I guess still acceptable for FPV

http://bitoniau.blogspot.fr/2013/10/video-latency-investigation.html
http://bitoniau.blogspot.fr/2014/02/video-latency-investigation-part-2.html
fatslugg
 
Posts: 4
Joined: Wed Jul 10, 2013 2:58 am

Re: Tutorial: Low cost FPV setup for the rift (130ms latency

Postby vizual54 » Thu Mar 20, 2014 10:41 am

This is really interesting as I have just started a project with the aim of mounting a stereo rig on my multicopter and using a Oculus for viewing.
I've been wanting to do this for a long time but have not had the time. I guess I won't have too much time for this now either with small kids and a toddler : )
Anyway my plan is to use two web cameras, probably Logitech c310, connected to a Intel nuc or a odroid to make the stereoscopic stream barrel correction etc. To start with I will probably use wireless lan or 4g though I have a 1.2 GHz analog video link an a happhauge USB stick to test with as well. The cameras will be mounted on a 3 axis brushless gimbal that will take ptz pwm signals from a APM flight controller.
I just ordered the new oculus today ; )

The system will not be used as FPV primarily since the latency will be to high. Though it will be fun to test the oculus for that purpose as well.
vizual54
 
Posts: 1
Joined: Wed Feb 26, 2014 5:04 am


Return to Oculus General Development

Who is online

Users browsing this forum: No registered users and 5 guests