公开(公告)号：US10397538B2

公开(公告)日：2019-08-27

申请号：US15058128

申请日：2016-03-01

Applicant: NextVR Inc.

Inventor： David Cole ,  Alan McKay Moss

IPC: H04N13/00 ,  H04N13/111 ,  G06F3/01 ,  H04N19/44 ,  H04N13/139 ,  H04N13/161 ,  H04N13/189 ,  H04N13/194 ,  H04N13/204 ,  H04N13/282 ,  G06T17/05 ,  G06T17/20 ,  G06T19/20 ,  G09G5/00 ,  H04N21/434 ,  G09G3/00 ,  H04N13/344 ,  H04N13/232 ,  H04N13/279 ,  H04N13/271 ,  H04N13/239 ,  G06F3/0481 ,  G06K9/00

Abstract: Methods and apparatus for supporting the capture of images of surfaces of an environment visible from a default viewing position and capturing images of surfaces not visible from the default viewing position, e.g., occluded surfaces, are described. Occluded and non-occluded image portions are packed into one or more frames and communicated to a playback device for use as textures which can be applied to a model of the environment where the images were captured. An environmental model includes a model of surfaces which are occluded from view from a default viewing position but which maybe viewed is the user shifts the user's viewing location. Occluded image content can be incorporated directly into a frame that also includes non-occluded image data or sent in frames of a separate, e.g., auxiliary content stream that is multiplexed with the main content stream which communicates image data corresponding to non-occluded environmental portions.

公开(公告)号：US20180288403A1

公开(公告)日：2018-10-04

申请号：US15861670

申请日：2018-01-04

Applicant: NextVR Inc.

Inventor： David Cole ,  Alan McKay Moss

IPC: H04N13/332 ,  G06F1/16

Abstract: Methods and apparatus for using a display in a manner which results in a user perceiving a higher resolution than would be perceived if a user viewed the display from a head on position are described. In some embodiments one or more displays are mounted at an angle, e.g., sometimes in range a range from an angle above 0 degrees to 45 relative to a user's face and thus eyes. The user sees more pixels, e.g., dots corresponding to light emitting elements, per square inch of eye area than the user would see if the user were viewing the display head on due to the angle at which the display or displays are mounted. The methods and display mounting arrangement are well suited for use in head mounted displays, e.g., Virtual Reality (VR) displays for stereoscopic viewing (e.g., 3D) and/or non-stereoscopic viewing of displayed images.

公开(公告)号：US09894350B2

公开(公告)日：2018-02-13

申请号：US15255135

申请日：2016-09-01

Applicant: NextVR Inc.

Inventor： David Cole ,  Alan McKay Moss ,  Hector M. Medina

IPC: H04N17/00 ,  H04N17/02 ,  H04N13/02 ,  H04N5/232 ,  G06T11/00 ,  G06T15/04 ,  G06T7/80 ,  H04N21/414 ,  H04N21/4402 ,  H04N21/485 ,  H04N21/81 ,  G02B27/01

CPC classification number: H04N17/002 ,  G02B27/0172 ,  G02B2027/0138 ,  G02B2027/014 ,  G06T7/85 ,  G06T11/001 ,  G06T15/04 ,  G06T2207/10021 ,  H04N5/23238 ,  H04N13/232 ,  H04N13/246 ,  H04N13/275 ,  H04N21/41407 ,  H04N21/4402 ,  H04N21/4854 ,  H04N21/816

Abstract: Camera and/or lens calibration information is generated as part of a calibration process in video systems including 3-dimensional (3D) immersive content systems. The calibration information can be used to correct for distortions associated with the source camera and/or lens. A calibration profile can include information sufficient to allow the system to correct for camera and/or lens distortion/variation. This can be accomplished by capturing a calibration image of a physical 3D object corresponding to the simulated 3D environment, and creating the calibration profile by processing the calibration image. The calibration profile can then be used to project the source content directly into the 3D viewing space while also accounting for distortion/variation, and without first translating into an intermediate space (e.g., a rectilinear space) to account for lens distortion.

公开(公告)号：US20170359564A1

公开(公告)日：2017-12-14

申请号：US15672236

申请日：2017-08-08

Applicant: NextVR Inc.

Inventor： David Cole ,  Alan McKay Moss ,  Hector M. Medina

IPC: H04N13/00 ,  H04N19/136 ,  H04N21/81 ,  H04N21/44 ,  H04N21/435 ,  H04N19/172 ,  G06T17/20 ,  H04N19/597 ,  H04N19/44 ,  H04N21/472 ,  G06T15/04 ,  H04N13/02 ,  H04N21/2343 ,  H04N21/437 ,  H04N5/232 ,  H04N21/2387 ,  H04N21/426

CPC classification number: H04N13/189 ,  G06T15/04 ,  G06T17/20 ,  G06T2215/12 ,  H04N5/23238 ,  H04N13/106 ,  H04N13/139 ,  H04N13/161 ,  H04N13/194 ,  H04N13/243 ,  H04N13/275 ,  H04N13/279 ,  H04N19/136 ,  H04N19/172 ,  H04N19/44 ,  H04N19/597 ,  H04N21/234363 ,  H04N21/2387 ,  H04N21/42653 ,  H04N21/435 ,  H04N21/437 ,  H04N21/44016 ,  H04N21/472 ,  H04N21/8146 ,  H04N21/816 ,  H04N2013/0088

Abstract: Methods and apparatus for using selective resolution reduction on images to be transmitted and/or used by a playback device are described. Prior to transmission one or more images of an environment are captured. Based on image content, motion detection and/or user input a resolution reduction operation is selected and performed. The reduced resolution image is communicated to a playback device along with information indicating a UV map corresponding to the selected resolution allocation that should be used by the playback device for rendering the communicated image. By changing the resolution allocation used and which UV map is used by the playback device different resolution allocations can be made with respect to different portions of the environment while allowing the number of pixels in transmitted images to remain constant. The playback device renders the individual images with the UV map corresponding to the resolution allocation used to generate the individual images.

公开(公告)号：US09832449B2

公开(公告)日：2017-11-28

申请号：US15009531

申请日：2016-01-28

Applicant: NextVR Inc.

Inventor： David Cole ,  Alan McKay Moss

IPC: H04N13/00 ,  H04N13/04 ,  H04N9/80 ,  H04N13/02 ,  G06F3/01 ,  G06F1/16 ,  H04N5/77 ,  G02B27/01

CPC classification number: H04N13/189 ,  G02B27/017 ,  G02B2027/0138 ,  G02B2027/0187 ,  G06F1/1626 ,  G06F1/163 ,  G06F3/011 ,  G06F3/012 ,  H04N13/117 ,  H04N13/158 ,  H04N13/161 ,  H04N13/194 ,  H04N13/239 ,  H04N13/243 ,  H04N13/368 ,  H04N13/398

Abstract: Methods and apparatus for allowing a user to switch viewing positions and/or perspective while viewing an environment, e.g., as part of a 3D playback/viewing experience, are described. In various embodiments images of the environment are captured using cameras placed at multiple camera positions. During viewing a user can select which camera position he/she would like to experience the environment from. While experiencing the environment from the perspective of a first camera position the user may switch from the first to a second camera position by looking at the second position. A visual indication is provided to the user to indicate that the user can select the other camera position as his/her viewing position. If a user input indicates a desired viewing position change, a switch to the alternate viewing position is made and the user is presented with images captured from the perspective of the user selected alternative viewing position.

公开(公告)号：US20170050743A1

公开(公告)日：2017-02-23

申请号：US14829562

申请日：2015-08-18

Applicant: NextVR Inc.

Inventor： David Cole ,  Alan McKay Moss

IPC: B64D43/00 ,  G06F3/01 ,  G06F1/16 ,  G02B27/01

CPC classification number: B64D43/00 ,  G02B27/017 ,  G02B2027/0138 ,  G02B2027/014 ,  G06F1/1694 ,  G06F3/012 ,  G06F2200/1637

Abstract: A head mounted virtual reality (VR) device including an inertial measurement unit (IMU) is located in a vehicle which may be, and sometimes is, moving. Detected motion attributable to vehicle motion is filtered out based on one or more or all of: vehicle type information, information derived from sensors located in the vehicle external to the head mounted VR device, and/or captured images including a reference point or reference object within the vehicle. An image portion of a simulated VR environment is selected and presented to the user of the head mounted VR device based on the filtered motion information. Thus, the image portion presented to the user of the head mounted VR device is substantially unaffected by vehicle motion and corresponds to user induced head motion.

Abstract translation: 包括惯性测量单元（IMU）的头戴式虚拟现实（VR）设备位于可能并且有时是移动的车辆中。 基于车辆类型信息中的一个或多个或全部滤波出检测到的车辆运动的运动，以及/或包括参考点或参考对象的捕获图像 在车内。 基于经滤波的运动信息，选择模拟VR环境的图像部分并呈现给头戴式VR设备的用户。 因此，呈现给头戴式VR装置的用户的图像部分基本上不受车辆运动的影响，并且对应于用户引起的头部运动。

公开(公告)号：US20150346812A1

公开(公告)日：2015-12-03

申请号：US14726440

申请日：2015-05-29

Applicant: NextVR Inc.

Inventor： David Cole ,  Alan McKay Moss

IPC: G06F3/01

CPC classification number: H04N21/482 ,  G06F3/017 ,  H04L65/4069 ,  H04N13/117 ,  H04N13/161 ,  H04N13/194 ,  H04N13/243 ,  H04N13/344 ,  H04N13/371 ,  H04N19/134 ,  H04N19/146 ,  H04N19/37 ,  H04N19/39 ,  H04N19/587 ,  H04N19/59 ,  H04N19/597

Abstract: Content delivery and playback methods and apparatus are described. The methods and apparatus are well suited for delivery and playback of content corresponding to a 360 degree environment and can be used to support streaming and/or real time delivery of 3D content corresponding to an event, e.g., while the event is ongoing or after the event is over. Portions of the environment are captured by cameras located at different positions. The content captured from different locations is encoded and made available for delivery. A playback device selects the content to be received based in a user's head position.

Abstract translation: 描述内容传送和播放方法和装置。 所述方法和装置非常适合于与360度环境相对应的内容的传送和回放，并且可以用于支持与事件相对应的3D内容的流传输和/或实时传送，例如，当事件正在进行时或者在 事件结束了。 部分环境由位于不同位置的摄像机捕获。 从不同位置捕获的内容被编码并可用于传送。 播放装置根据用户的头位置选择要接收的内容。

公开(公告)号：US10735709B2

公开(公告)日：2020-08-04

申请号：US16013879

申请日：2018-06-20

Applicant: NextVR Inc.

Inventor： Alan McKay Moss ,  Hector M Medina ,  Ryan Michael Sheridan ,  Matthew Yaeger ,  David Ibbitson ,  David Cole

IPC: H04N13/122 ,  H04N13/239

Abstract: An unobstructed image portion of a captured image from a first camera of a camera pair, e.g., a stereoscopic camera pair including fisheye lenses, is combined with a scaled extracted image portion generated from a captured image from a second camera in the camera pair. An unobstructed image portion of a captured image from the second camera of the camera pair is combined with a scaled extracted image portion generated from a captured image from the first camera in the camera pair. As part of the combining obstructed image portions which were obstructed by part of the adjacent camera are replaced in some embodiments. In some embodiments, the obstructions are due to adjacent fisheye lens. In various embodiments fish eye lenses which have been cut to be flat on one side are used for the left and right cameras with the spacing between the optical axis approximating the spacing between the optical axis of a human person's eyes.

公开(公告)号：US10432910B2

公开(公告)日：2019-10-01

申请号：US15806264

申请日：2017-11-07

Applicant: NextVR Inc.

Inventor： David Cole ,  Alan McKay Moss

IPC: H04N13/00 ,  H04N13/189 ,  G06F3/01 ,  H04N13/106 ,  H04N13/243 ,  H04N13/368 ,  H04N13/398 ,  G06F1/16 ,  H04N13/194 ,  H04N13/117 ,  H04N13/161 ,  H04N13/239 ,  H04N9/80 ,  H04N5/77 ,  G02B27/01

Abstract: Methods and apparatus for allowing a user to switch viewing positions and/or perspective while viewing an environment, e.g., as part of a 3D playback/viewing experience, are described. In various embodiments images of the environment are captured using cameras placed at multiple camera positions. During viewing a user can select which camera position he/she would like to experience the environment from. While experiencing the environment from the perspective of a first camera position the user may switch from the first to a second camera position by looking at the second position. A visual indication is provided to the user to indicate that the user can select the other camera position as his/her viewing position. If a user input indicates a desired viewing position change, a switch to the alternate viewing position is made and the user is presented with images captured from the perspective of the user selected alternative viewing position.

公开(公告)号：US10200672B2

公开(公告)日：2019-02-05

申请号：US15239823

申请日：2016-08-17

Applicant: NextVR Inc.

Inventor： David Cole ,  Alan McKay Moss ,  Michael P Straub

IPC: H04N7/01 ,  H04N5/262 ,  G06T7/13 ,  G06T5/50 ,  G06T11/60 ,  G06T3/40 ,  G06K9/00 ,  G06K9/52 ,  H04N5/247 ,  H04N13/232 ,  G02B13/06 ,  G02B27/22 ,  H04N13/239

Abstract: Customer wide angle lenses and methods and apparatus for using such lenses in individual cameras as well as pairs of cameras intended for stereoscopic image capture are described. The lenses are used in combination with sensors to capture different portions of an environment at different resolutions. In some embodiments ground is capture at a lower resolution than sky which is captured at a lower resolution than a horizontal area of interest. Various asymmetries in lenses and/or lens and sensor placement are described which are particularly well suited for stereoscopic camera pairs where the proximity of one camera to the adjacent camera may interfere with the field of view of the cameras.