Light Field Capture Control Methods and Apparatuses, Light Field Capture Devices

Content Provider	The Lens
Abstract	Embodiments of the subject application disclose various light field capture control methods and apparatuses and various light field capture devices, wherein one light field capture control method comprises: acquiring depth information of a to-be-shot scene; determining target pixel density distribution information of an image sensor of a light field camera according to the depth information; adjusting pixel density distribution of the image sensor according to the target pixel density distribution information; and performing, by the adjusted image sensor, light field capture of the to-be-shot scene. The technical solution provided in the embodiments of the subject application can make full use of overall pixels of the image sensor of the light field camera to unevenly record light field information of different regions of the to-be-shot scene in a depth of field (DOF) direction, thereby improving light field capture efficiency.
Related Links	https://www.lens.org/lens/patent/009-053-097-355-234/frontpage
Language	English
Publisher Date	2019-07-02
Access Restriction	Open
Content Type	Text
Resource Type	Patent
Jurisdiction	United States of America
Date Applied	2015-08-26
Agent	Sheppard Mullin Richter & Hampton Llp
Applicant	Beijing Zhigu Tech Co Ltd
Application No.	201515510803
Claim	A light field capture control method, comprising: acquiring depth information of a to-be-shot scene; determining target pixel density distribution information of an image sensor of a light field camera according to the depth information; adjusting pixel density distribution of the image sensor according to the target pixel density distribution information; and performing, by the adjusted image sensor, light field capture of the to-be-shot scene, wherein determining target pixel density distribution information according to the depth information comprises: performing regional division on the to-be-shot scene in a depth of field (DOF) direction according to the depth information; and determining the target pixel density distribution information according to a result of the regional division, wherein in the target pixel density distribution information, target pixel density distribution corresponding to at least two different regions divided varies. The light field capture control method of claim 1 , wherein in the target pixel density distribution information, average target pixel density corresponding to different regions presents gradual change with a regularity along with an increase of DOF depths of the different regions. The light field capture control method of claim 1 , wherein in the target pixel density distribution information, average target pixel density corresponding to a shallower-DOF region is greater than that corresponding to a deeper-DOF region. The light field capture control method of claim 1 , wherein the determining the target pixel density distribution information according to a result of the regional division comprises: determining distribution information of a sub-lens, which affects light field capture of respective regions in the result of the regional division, in a sub-lens array of the light field camera; determining imaging region distribution information of the image sensor corresponding to the distribution information of the sub-lens; and determining the target pixel density distribution information according to the imaging region distribution information, wherein in the target pixel density distribution information, target pixel density distribution corresponding to at least two different imaging regions varies. The light field capture control method of claim 1 , wherein the determining the target pixel density distribution information according to a result of the regional division comprises: determining target view distribution information of light field capture of the to-be-shot scene according to the result of the regional division; and determining the target pixel density distribution information according to the target view distribution information. The light field capture control method of claim 5 , wherein in the target view distribution information, the number of average target views corresponding to different regions presents gradual change with a regularity along with an increase of DOF depths of the different regions. The light field capture control method of claim 6 , wherein in the target view distribution information, the number of average target views corresponding to a shallower-DOF region is greater than that corresponding to a deeper-DOF region. The light field capture control method of claim 5 , wherein the determining the target pixel density distribution information according to the target view distribution information comprises: determining distribution information of a sub-lens, which affects light field capture of respective regions in the result of the regional division, in a sub-lens array of the light field camera; determining imaging region distribution information of the image sensor corresponding to the distribution information of the sub-lens; determining imaging sub-region distribution information, corresponding to respective imaging regions and the target view distribution information, in the imaging region distribution information; and determining the target pixel density distribution information according to the imaging sub-region distribution information, wherein in the target pixel density distribution information, target pixel density distribution corresponding to at least two different imaging sub-regions varies. The light field capture control method of claim 1 , wherein the adjusting pixel density distribution of the image sensor according to the target pixel density distribution information comprises: determining deformation control information of a controllable deformed material portion according to the target pixel density distribution information; and controlling the controllable deformed material portion to produce deformation according to the deformation control information, so as to correspondingly adjust the pixel density distribution of the image sensor through the deformation of the controllable deformed material portion. The light field capture control method of claim 9 , wherein the controllable deformed material portion is prepared from at least one of the following controllable deformed materials: piezoelectric materials, electroactive polymers, photodeformation materials and magnetostriction materials. A light field capture control apparatus, comprising: a scene depth information capture module, configured to acquire depth information of a to-be-shot scene; a target pixel density distribution information determination module, configured to determine target pixel density distribution information of an image sensor of a light field camera according to the depth information; a pixel density distribution adjustment module, configured to adjust pixel density distribution of the image sensor according to the target pixel density distribution information; and a light field capture module, configured to perform light field capture of the to-be-shot scene via the adjusted image sensor, wherein the target pixel density distribution information determination module comprises: a regional division sub-module, configured to perform regional division on the to-be-shot scene in a depth of field (DOF) direction according to the depth information; and a target pixel density distribution information determination sub-module, configured to determine the target pixel density distribution information according to a result of the regional division, wherein in the target pixel density distribution information, target pixel density distribution corresponding to at least two different regions divided varies. The light field capture control apparatus of claim 11 , wherein the target pixel density distribution information determination sub-module comprises: a sub-lens distribution information determination unit, configured to determine distribution information of a sub-lens, which affects light field capture of respective regions in the result of the regional division, in a sub-lens array of the light field camera; an imaging region distribution information determination unit, configured to determine imaging region distribution information of the image sensor corresponding to the distribution information of the sub-lens; and a first target pixel density distribution information determination unit, configured to determine the target pixel density distribution information according to the imaging region distribution information, wherein in the target pixel density distribution information, target pixel density distribution corresponding to at least two different imaging regions varies. The light field capture control apparatus of claim 11 , wherein the target pixel density distribution information determination sub-module comprises: a target view distribution information determination unit, configured to determine target view distribution information of light field capture of the to-be-shot scene according to the result of the regional division; and a second target pixel density distribution information determination unit, configured to determine the target pixel density distribution information according to the target view distribution information. The light field capture control apparatus of claim 13 , wherein the second target pixel density distribution information determination unit comprises: a sub-lens distribution information determination sub-unit, configured to determine distribution information of a sub-lens, which affects light field capture of respective regions in the result of the regional division, in a sub-lens array of the light field camera; an imaging region distribution information determination sub-unit, configured to determine imaging region distribution information of the image sensor corresponding to the distribution information of the sub-lens; an imaging sub-region distribution information determination sub-unit, configured to determine imaging sub-region distribution information, corresponding to respective imaging regions and the target view distribution information, in the imaging region distribution information; and a target pixel density distribution information determination sub-unit, configured to determine the target pixel density distribution information according to the imaging sub-region distribution information, wherein in the target pixel density distribution information, target pixel density distribution corresponding to at least two different imaging sub-regions varies. The light field capture control apparatus of claim 11 , wherein the pixel density distribution adjustment module comprises: a deformation control information determination sub-module, configured to determine deformation control information of a controllable deformed material portion according to the target pixel density distribution information; and a deformation control sub-module, configured to control the controllable deformed material portion to produce deformation according to the deformation control information, so as to correspondingly adjust the pixel density distribution of the image sensor through the deformation of the controllable deformed material portion. The light field capture control apparatus of claim 15 , wherein the controllable deformed material portion is prepared from at least one of the following controllable deformed materials: piezoelectric materials, electroactive polymers, photodeformation materials and magnetostriction materials. A light field capture device, comprising a light field capture control apparatus, which comprises: a scene depth information capture module, configured to acquire depth information of a to-be-shot scene; a target pixel density distribution information determination module, configured to determine target pixel density distribution information of an image sensor of a light field camera according to the depth information; a pixel density distribution adjustment module, configured to adjust pixel density distribution of the image sensor according to the target pixel density distribution information; and a light field capture module, configured to perform light field capture of the to-be-shot scene via the adjusted image sensor, wherein, the light field capture control apparatus is connected with the light field camera, and wherein the target pixel density distribution information determination module comprises: a regional division sub-module, configured to perform regional division on the to-be-shot scene in a depth of field (DOF) direction according to the depth information; and a target pixel density distribution information determination sub-module, configured to determine the target pixel density distribution information according to a result of the regional division, wherein in the target pixel density distribution information, target pixel density distribution corresponding to at least two different regions divided varies. The light field capture device of claim 17 , wherein the image sensor comprises: multiple image sensor pixels array-distributed; and a controllable deformed material portion, respectively connected with the multiple image sensor pixels; wherein the controllable deformed material portion can produce deformation under the action of an external field, and correspondingly adjusts density distribution of the multiple image sensor pixels through the deformation; and the external field is controlled by the light field capture control apparatus. A non-transitory computer readable storage apparatus, comprising at least one executable instruction, which, in response to execution, causes a light field capture control method, comprising: acquiring depth information of a to-be-shot scene; determining target pixel density distribution information of an image sensor of a light field camera according to the depth information; adjusting pixel density distribution of the image sensor according to the target pixel density distribution information; and performing, by the adjusted image sensor, light field capture of the to-be-shot scene, wherein determining target pixel density distribution information according to the depth information comprises: performing regional division on the to-be-shot scene in a depth of field (DOF) direction according to the depth information; and determining the target pixel density distribution information according to a result of the regional division, wherein in the target pixel density distribution information, target pixel density distribution corresponding to at least two different regions divided varies. A light field capture control apparatus, characterized by comprising a processor and a memory, the memory storing computer executable instructions, the processor being connected to the memory through a communication bus, and when the apparatus for controlling task migration operates, the processor executing the computer executable instructions stored in the memory, so that the apparatus for controlling task migration executes operations, comprising: acquiring depth information of a to-be-shot scene; determining target pixel density distribution information of an image sensor of a light field camera according to the depth information; adjusting pixel density distribution of the image sensor according to the target pixel density distribution information; and performing, by the adjusted image sensor, light field capture of the to-be-shot scene, wherein determining target pixel density distribution information according to the depth information comprises: performing regional division on the to-be-shot scene in a depth of field (DOF) direction according to the depth information; and determining the target pixel density distribution information according to a result of the regional division, wherein in the target pixel density distribution information, target pixel density distribution corresponding to at least two different regions divided varies.
CPC Classification	PICTORIAL COMMUNICATION; e.g. TELEVISION
Examiner	Yogesh K Aggarwal
Extended Family	009-053-097-355-234 018-579-651-448-482 083-897-734-735-816 044-001-440-604-90X 179-073-847-777-042
Patent ID	10341594
Inventor/Author	Zhou Liang Du Lin
IPC	H04N25/00
Status	Active
Owner	Beijing Zhigu Tech Co. Ltd
Simple Family	009-053-097-355-234 018-579-651-448-482 083-897-734-735-816 044-001-440-604-90X 179-073-847-777-042
CPC (with Group)	H04N23/60 H04N25/705 H04N23/815
Issuing Authority	United States Patent and Trademark Office (USPTO)
Kind	Patent/New European patent specification (amended specification after opposition procedure)