Predictive Viewport Renderer and Foveated Color Compressor

Content Provider	The Lens
Abstract	An embodiment of a graphics apparatus may include a focus identifier to identify a focus area, and a color compressor to selectively compress color data based on the identified focus area. Another embodiment of a graphics apparatus may include a motion detector to detect motion of a real object, a motion predictor to predict a motion of the real object, and an object placer to place a virtual object relative to the real object based on the predicted motion of the real object. Another embodiment of a graphics apparatus may include a frame divider to divide a frame into viewports, a viewport prioritizer to prioritize the viewports, a renderer to render a viewport of the frame in order in accordance with the viewport priorities, and a viewport transmitter to transmit a completed rendered viewport. Other embodiments are disclosed and claimed.
Related Links	https://www.lens.org/lens/patent/009-871-897-873-085/frontpage
Language	English
Publisher Date	2019-09-24
Access Restriction	Open
Content Type	Text
Resource Type	Patent
Jurisdiction	United States of America
Date Applied	2017-04-01
Agent	Jordan Ip Law, Llc
Applicant	Intel Corp
Application No.	201715476990
Claim	An image compressor apparatus, comprising: a focus identifier, implemented at least partly in one or more of configurable logic or fixed-functionality logic hardware, to identify a focus area, wherein the focus identifier is to identify the focus area based on user gaze information; and a color compressor, implemented at least partly in one or more of configurable logic or fixed-functionality logic hardware, communicatively coupled to the focus identifier to: identify shortest distances between pixels associated with an image and one or more of the focus area or a focal pixel of the image, and selectively compress color data based on the identified focus area and the shortest distances so that a first region of a plurality of regions associated with the image has first color data represented by a first number of bits, a second region of the plurality of regions has second color data represented by a second number of bits, and a third region of the plurality of regions has third color data represented by a third number of bits, wherein the third number of bits, the second number of bits and the first number of bits are different from each other. The apparatus of claim 1 , wherein: the first region is within the focus area; the first, second and third regions are non-intersected regions where the second region borders the first region and the third region borders the second region; and the first number of bits is greater than the second number of bits, and the second number of bits is greater than the third number of bits. The apparatus of claim 2 , wherein the color compressor is further to apply relatively less compression to the second region as compared to the third region. The apparatus of claim 3 , further comprising: a frame buffer communicatively coupled to the color compressor to store image data including the first, second and third color data; and a mask store to store two or more color masks respectively corresponding to the first, second and third regions, wherein the color compressor is further to apply the two or more color masks to the frame buffer to color compress the image data stored in the frame buffer. The apparatus of claim 1 , further comprising: a frame divider, implemented at least partly in one or more of configurable logic or fixed-functionality logic hardware, to divide a frame into two or more viewports; a viewport prioritizer, implemented at least partly in one or more of configurable logic or fixed-functionality logic hardware, communicatively coupled to the frame divider to prioritize the two or more viewports to generate viewport priorities; a renderer, implemented at least partly in one or more of configurable logic or fixed-functionality logic hardware, communicatively coupled to the viewport prioritizer to render a viewport of the frame in accordance with the viewport priorities; and a viewport transmitter, implemented at least partly in one or more of configurable logic or fixed-functionality logic hardware, communicatively coupled to the renderer to transmit a completed rendered viewport, wherein the renderer is further to render another viewport of the frame at a same time as the viewport transmitter transmits the completed rendered viewport. The apparatus of claim 1 , further comprising: a motion detector, implemented at least partly in one or more of configurable logic or fixed-functionality logic hardware, to detect motion of a real object; a motion predictor, implemented at least partly in one or more of configurable logic or fixed-functionality logic hardware, communicatively coupled to the motion detector to predict a motion of the real object; and an object placer, implemented at least partly in one or more of configurable logic or fixed-functionality logic hardware, communicatively coupled to the motion detector and the motion predictor to place a virtual object relative to the real object based on the predicted motion of the real object. The apparatus of claim 1 , wherein: the color compressor is to access calibration data associated with a user to determine the third number of bits, the second number of bits and the first number of bits; the calibration data is to be generated based on a calibration phase in which the user indicates whether calibration colors appear to be the same to the user; and the calibration colors are different from each other. A method of compressing an image, comprising: identifying a focus area based on user gaze information; identifying shortest distances between pixels associated with the image and one or more of the focus area or a focal pixel of the image; and selectively compressing color data based on the identified focus area and the shortest distances so that a first region of a plurality of regions associated with the image has first color data represented by a first number of bits, a second region of the plurality of regions has second color data represented by a second number of bits, and a third region of the plurality of regions has third color data represented by a third number of bits, wherein the third number of bits, the second number of bits and the first number of bits are different from each other. The method of claim 8 , wherein: the first region is within the focus area; the first, second and third regions are non-intersected regions where the second region borders the first region and the third region borders the second region; and the first number of bits is greater than the second number of bits, and the second number of bits is greater than the third number of bits. The method of claim 9 , further comprising: applying relatively less compression to the second region an innermost region as compared to the third region. The method of claim 10 , further comprising: storing image data including the first, second and third color data in a frame buffer; storing two or more color masks respectively corresponding to the first, second and third regions; and applying the two or more color masks to the frame buffer to color compress the image data stored in the frame buffer. The method of claim 8 , further comprising: accessing calibration data associated with a user to determine the third number of bits, the second number of bits and the first number of bits, wherein the calibration data is to be generated based on a calibration phase in which the user indicates whether calibration colors appear to be the same to the user; and further wherein the calibration colors are different from each other. At least one computer readable medium, comprising a set of instructions, which when executed by a computing device, cause the computing device to: identify a focus area based on user gaze information; identify shortest distances between pixels associated with an image and one or more of the focus area or a focal pixel of the image; and selectively compress color data based on the identified focus area and the shortest distances so that a first region of a plurality of regions associated with the image has first color data represented by a first number of bits, a second region of the plurality of regions has second color data represented by a second number of bits, and a third region of the plurality of regions has third color data represented by a third number of bits, wherein the third number of bits, the second number of bits and the first number of bits are different from each other. The at least one computer readable medium of claim 13 , wherein: the first region is within the focus area; the first, second and third regions are non-intersected regions where the second region borders the first region and the third region borders the second region; and the first number of bits is greater than the second number of bits, and the second number of bits is greater than the third number of bits. The at least one computer readable medium of claim 14 , comprising a further set of instructions, which when executed by the computing device, cause the computing device to: apply relatively less compression to the second region as compared to the third region. The at least one computer readable medium of claim 15 , comprising a further set of instructions, which when executed by the computing device, cause the computing device to: store image data including the first, second and third color data in a frame buffer; store two or more color masks respectively corresponding to the first, second and third regions; and apply the two or more color masks to the frame buffer to color compress the image data stored in the frame buffer. The at least one computer readable medium of claim 13 , comprising a further set of instructions, which when executed by the computing device, cause the computing device to: access calibration data associated with a user to determine the third number of bits, the second number of bits and the first number of bits; wherein the calibration data is to be generated based on a calibration phase in which the user indicates whether calibration colors appear to be the same to the user; and further wherein the calibration colors are different from each other. A graphics viewport apparatus, comprising: a frame divider, implemented at least partly in one or more of configurable logic or fixed-functionality logic hardware, to divide a frame into two or more viewports; a viewport prioritizer, implemented at least partly in one or more of configurable logic or fixed-functionality logic hardware, communicatively coupled to the frame divider to prioritize the two or more viewports to generate viewport priorities; a renderer, implemented at least partly in one or more of configurable logic or fixed-functionality logic hardware, communicatively coupled to the viewport prioritizer to render a first viewport of the frame and a second viewport of the frame in accordance with the viewport priorities so that the first viewport is rendered before the second viewport; and a viewport transmitter, implemented at least partly in one or more of configurable logic or fixed-functionality logic hardware, communicatively coupled to the renderer to transmit the first viewport after the first viewport has been rendered but before the second viewport has completed rendering so that the renderer is to render the second viewport at a same time as the viewport transmitter transmits the first viewport, wherein the viewport transmitter is further to transmit the second viewport after the second viewport is rendered by the renderer, wherein after the viewport transmitter transmits the first viewport and the second viewport, the first viewport and the second viewport are to be merged to form an image to be simultaneously displayed. The apparatus of claim 18 , wherein the frame divider is further to dynamically re-divide the frame into a new set of two or more viewports. The apparatus of claim 19 , wherein the frame divider is further to dynamically re-divide the frame into the new set of two or more viewports based on one or more of an area of focus, motion prediction information, or an optical property of a target head mounted display. The apparatus of claim 18 , wherein the viewport prioritizer is further to dynamically prioritize the two or more viewports based on one or more of an area of focus, gaze information, motion information, motion prediction information, and content information, wherein the viewport prioritizer is to determine a first frames per second for the first viewport based on one or more of a head motion or eye tracking information, and a second frames per second for the second viewport based on the one or more of the head motion or the eye tracking information, wherein the first frames per second is different from the second frames per second. The apparatus of claim 21 , further comprising: a multi-view encoder, implemented at least partly in one or more of configurable logic or fixed-functionality logic hardware, communicatively coupled to the renderer to encode the first and second viewports after the first and second viewports are rendered.
CPC Classification	IMAGE DATA PROCESSING OR GENERATION; IN GENERAL PICTORIAL COMMUNICATION; e.g. TELEVISION
Examiner	Gordon G Liu
Extended Family	009-871-897-873-085 141-076-104-034-832 093-639-154-702-613 110-221-950-715-796 071-263-068-859-883 098-766-596-750-480
Patent ID	10424097
Inventor/Author	Vembar Deepak S Kuwahara Atsuo Sakthivel Chandrasekaran Venkataraman Radhakrishnan Insko Brent E Kalra Anupreet S Labbe Hugues Appu Abhishek R Shah Ankur N Ray Joydeep Ould-ahmed-vall Elmoustapha Holland James M
IPC	G09G5/00 G06T9/00 G06T11/60 H04N19/124 H04N19/167 H04N19/17 H04N19/436 H04N19/503
Status	Active
Owner	Intel Corporation
Simple Family	009-871-897-873-085 098-766-596-750-480 093-639-154-702-613 110-221-950-715-796 071-263-068-859-883 141-076-104-034-832
CPC (with Group)	G06T9/00 H04N19/124 H04N19/167 H04N19/17 H04N19/436 H04N19/503 G06T11/60
Issuing Authority	United States Patent and Trademark Office (USPTO)
Kind	Patent/New European patent specification (amended specification after opposition procedure)