1231 av 3
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf
Biological Inspired Deformable Image Sensor
Blekinge Tekniska Högskola, Fakulteten för datavetenskaper, Institutionen för teknik och estetik.ORCID-id: 0000-0003-3887-5972
2019 (Engelska)Doktorsavhandling, sammanläggning (Övrigt vetenskapligt)
Abstract [en]

Nowadays, cameras are everywhere thanks to the tremendous progress on sensor technology. However, their performance is far away from what we experience by our eyes. The study from evolution process shows how the sensor arrangement of retina in human vision has differentiated from other species and is formed into a specific combination of sub-arrangements from hexagonal to elliptical ones. There are three major key differences between our visual cell arrangement and current camera sensors which are: the sub-arrangements, the pixel form and the pixel density.

Despite the advances in sensor technology we face limitations in their further development; i.e. to make the cameras close to the visual system. This is due to the optical diffraction limit which prevents us to increase the sensor resolution, and rigidity of hardware implementation which prevent us to change the image sensor after manufacturing. In the thesis the possibilities to overcome such limitations are investigated where the intention is to find a closer sensory solution to the visual system in comparison to the current ones.

Breaking the diffraction barrier and solving the rigidity problem are simultaneously achieved by introducing and estimating virtual subpixels. A statistical framework consisting of local learning model and Bayesian inference for predicting the incident photons captured on each such a subpixel is used to resample the captured image by any current camera sensor. By investigating the virtual variation of pixel size and fill factor the validity of the proposed idea is proven by which the results show significant changes of dynamic range and tonal levels in relation to the variation. As an example, for both monochrome and color images the results show that by virtual increase of fill factor to 100%, the dynamic range of the images are widened and the tonal levels are enriched significantly over 256 levels for each channel.

The results of virtual variation of the fill factor and pixel size indicates that it is feasible to change the rigidity of the image sensor using the software-based method. Inspired by the mosaic in the fovea, the center of human retina, the hexagonal sub-arrangement and pixel form are proposed to generate images based on the estimated virtual subpixels. Compared to the original square images, not only the dynamic range and tonal levels are improved, but also the hexagonal images are superior in detection of edges, i.e. more edge points on the contour of the objects are detected in hexagonal images.

The evaluation of different sub-arrangements or pixel forms of the image sensor is a challenging task and should be directed to a more specific task. Since the curvature contours contain most of the information related to object perception and human vision is highly evolved to detect curvature object, the task is focused to investigate the impact of the curviness on the different pixel forms and sub-arrangements, by comparing two categories of images; having curved versus linear edges of the objects in a pair of images which have exact similar contents but different contours. The detectability of each of the different sensor structures for curviness is estimated and the results show that the image on hexagonal grid with hexagonal pixel form is the best image type for distinguishing the curvature contours in the images.

According to the pattern of pixels tiling, there are two types of pixel sub-arrangements, i.e. periodic (e.g. square or hexagonal), and aperiodic (e.g. Penrose). Each type of sub-arrangements is investigated where the pixel forms and density are variable. By having at least two generated images of one configuration (i.e. specific sub-arrangement, pixel form and density), the result of histogram of gradient orientation of the certain sensor arrangement shows a stable and specific distribution which we called it the ANgular CHaracteristic of a sensOR structure (ANCHOR). Each ANCHOR has a robust pattern which is changed by the change of the sensor sub-arrangement. This makes it feasible to plan a sensor sub-arrangement in the relation to a specific application and its requirements, and more alike the biological vision sensory. To generate such a flexible sensor, a general framework is proposed for virtual deforming the sensor with a certain configuration of the sensor sub-arrangement, pixel form and pixel density.

Assessing the quality difference between the images generated by different sensor configuration or addressing from on configuration to another one generally needs the conversion of one to another. To overcome this problem, a common space is proposed by implementing a continuous extension of square or hexagonal images based on the orbit function, for quality evaluating the images with different arrangements and addressing from one type of image to another one. The evaluation results show that the creation of such space is feasible which facilitates a usage friendly tool to address an arrangement and assess the changes between different spatial arrangements, for example, it shows richer intensity variation, nonlinear behavior, and larger dynamic range in the hexagonal images compared to the rectangular images.

Ort, förlag, år, upplaga, sidor
Karlskrona: Blekinge Tekniska Högskola, 2019. , s. 207
Serie
Blekinge Institute of Technology Doctoral Dissertation Series, ISSN 1653-2090 ; 4
Nyckelord [en]
image sensor, pixel form, sub-arrangements, fill factor, square image, hexagonal image, deformable sensor, quality assessment.
Nationell ämneskategori
Elektroteknik och elektronik
Identifikatorer
URN: urn:nbn:se:bth-17149ISBN: 978-91-7295-366-6 (tryckt)OAI: oai:DiVA.org:bth-17149DiVA, id: diva2:1272821
Disputation
2019-03-14, J1650, Campus Gräsvik, Karlskrona, 13:15 (Engelska)
Opponent
Handledare
Tillgänglig från: 2018-12-20 Skapad: 2018-12-20 Senast uppdaterad: 2019-02-12Bibliografiskt granskad
Delarbeten
1. Novel Software-based Method to Widen Dynamic Range of CCD Sensor Images
Öppna denna publikation i ny flik eller fönster >>Novel Software-based Method to Widen Dynamic Range of CCD Sensor Images
2015 (Engelska)Ingår i: / [ed] Yu-Jin Zhang, Springer, 2015, Vol. 9218, s. 572-583Konferensbidrag, Publicerat paper (Refereegranskat)
Abstract [en]

In the past twenty years, CCD sensor has made huge progress in improving resolution and low-light performance by hardware. However due to physical limits of the sensor design and fabrication, fill factor has become the bottle neck for improving quantum efficiency of CCD sensor to widen dynamic range of images. In this paper we propose a novel software-based method to widen dynamic range, by virtual increase of fill factor achieved by a resampling process. The CCD images are rearranged to a new grid of virtual pixels com-posed by subpixels. A statistical framework consisting of local learning model and Bayesian inference is used to estimate new subpixel intensity. By knowing the different fill factors, CCD images were obtained. Then new resampled images were computed, and compared to the respective CCD and optical image. The results show that the proposed method is possible to widen significantly the recordable dynamic range of CCD images and increase fill factor to 100 % virtually.

Ort, förlag, år, upplaga, sidor
Springer, 2015
Serie
Lecture Notes in Computer Science, ISSN 0302-9743
Nyckelord
Dynamic range, Fill factor, CCD sensors, Sensitive area, Quantum efficiency
Nationell ämneskategori
Signalbehandling
Identifikatorer
urn:nbn:se:bth-11169 (URN)10.1007/978-3-319-21963-9_53 (DOI)978-3-319-21963-9 (ISBN)
Konferens
International Conference on Image and Graphics 2015, Tianjin, China
Tillgänglig från: 2015-12-11 Skapad: 2015-12-11 Senast uppdaterad: 2018-12-20Bibliografiskt granskad
2. A Software Method to Extend Tonal Levels and Widen Tonal Range of CCD Sensor Images
Öppna denna publikation i ny flik eller fönster >>A Software Method to Extend Tonal Levels and Widen Tonal Range of CCD Sensor Images
2015 (Engelska)Ingår i: 2015 9TH INTERNATIONAL CONFERENCE ON SIGNAL PROCESSING AND COMMUNICATION SYSTEMS (ICSPCS), IEEE Communications Society, 2015Konferensbidrag, Publicerat paper (Refereegranskat)
Abstract [en]

As one of important outcomes of the past decades of researches on sensor arrays for digital cameras, the manufacturers of sensor array technology have responded to the necessity and importance of obtaining an optimal fill factor, which has great impact on collection of incident photons on the sensor, with hardware solution e.g. by introducing microlenses. However it is still impossible to make a fill factor of 100% due to the physical limitations in practical development and manufacturing of digital camera. This has been a bottle neck problem for improving dynamic range and tonal levels for digital cameras e.g. CCD cameras. In this paper we propose a software method to not only widen the recordable dynamic range of a captured image by a CCD camera but also extend its tonal levels. In the method we estimate the fill factor and by a resampling process a virtual fill factor of 100% is achieved where a CCD image is rearranged to a new grid of virtual subpixels. A statistical framework including local learning model and Bayesian inference is used for estimating new sub-pixel intensity values. The highest probability of sub-pixels intensity values in each resampled pixel area is used to estimate the pixel intensity values of the new image. The results show that in comparison to the methods of histogram equalization and image contrast enhancement, which are generally used for improving the displayable dynamic range on only one image, the tonal levels and dynamic range of the image is extended and widen significantly and respectively.

Ort, förlag, år, upplaga, sidor
IEEE Communications Society, 2015
Nyckelord
CCD sensor, tonal range, fill factor, quantum efficiency
Nationell ämneskategori
Kommunikationssystem
Identifikatorer
urn:nbn:se:bth-12967 (URN)000380405700050 ()978-1-4673-8118-5 (ISBN)
Externt samarbete:
Konferens
9th International Conference on Signal Processing and Communication Systems (ICSPCS), DEC 14-16, 2015, Cairns, AUSTRALIA
Tillgänglig från: 2016-08-31 Skapad: 2016-08-30 Senast uppdaterad: 2018-12-20Bibliografiskt granskad
3. Estimation of Image Sensor Fill Factor Using a Single Arbitrary Image
Öppna denna publikation i ny flik eller fönster >>Estimation of Image Sensor Fill Factor Using a Single Arbitrary Image
2017 (Engelska)Ingår i: Sensors, ISSN 1424-8220, E-ISSN 1424-8220, Vol. 17, nr 3, s. 620-Artikel i tidskrift (Refereegranskat) Published
Abstract [en]

Achieving a high fill factor is a bottleneck problem for capturing high-quality images. There are hardware and software solutions to overcome this problem. In the solutions, the fill factor is known. However, this is an industrial secrecy by most image sensor manufacturers due to its direct effect on the assessment of the sensor quality. In this paper, we propose a method to estimate the fill factor of a camera sensor from an arbitrary single image. The virtual response function of the imaging process and sensor irradiance are estimated from the generation of virtual images. Then the global intensity values of the virtual images are obtained, which are the result of fusing the virtual images into a single, high dynamic range radiance map. A non-linear function is inferred from the original and global intensity values of the virtual images. The fill factor is estimated by the conditional minimum of the inferred function. The method is verified using images of two datasets. The results show that our method estimates the fill factor correctly with significant stability and accuracy from one single arbitrary image according to the low standard deviation of the estimated fill factors from each of images and for each camera.

Ort, förlag, år, upplaga, sidor
MDPI, 2017
Nyckelord
fill factor; virtual image; image sensor; pipeline; virtual response function; sensor irradiance
Nationell ämneskategori
Datorseende och robotik (autonoma system)
Identifikatorer
urn:nbn:se:bth-14046 (URN)10.3390/s17030620 (DOI)000398818700193 ()
Anmärkning

open access

Tillgänglig från: 2017-03-24 Skapad: 2017-03-24 Senast uppdaterad: 2018-12-20Bibliografiskt granskad
4. Image Quality Assessment of Enriched Tonal Levels Images
Öppna denna publikation i ny flik eller fönster >>Image Quality Assessment of Enriched Tonal Levels Images
2017 (Engelska)Ingår i: Image and Graphics 9th International Conference, ICIG 2017, Shanghai, China, September 13-15, 2017, Revised Selected Papers, Part II / [ed] Yao Zhao, Xiangwei Kong, David Taubman, Springer, 2017, s. 134-146Konferensbidrag, Publicerat paper (Refereegranskat)
Abstract [en]

The quality assessment of a high dynamic image is a challenging task. The few available no reference image quality methods for high dynamic range images are generally in evaluation stage. The most available image quality assessment methods are designed to assess low dynamic range images. In the paper, we show the assessment of high dynamic range images which are generated by utilizing a virtually flexible fill factor on the sensor images. We present a new method in the assessment process and evaluate the amount of improvement of the generated high dynamic images in comparison to original ones. The results show that the generated images not only have more number of tonal levels in comparison to original ones but also the dynamic range of images have significantly increased due to the measurable improvement values.

Ort, förlag, år, upplaga, sidor
Springer, 2017
Nyckelord
Image quality assessment, High dynamic range image, Tonal levels, Tone mapping, Fill factor
Nationell ämneskategori
Signalbehandling
Identifikatorer
urn:nbn:se:bth-15709 (URN)10.1007/978-3-319-71598-8_13 (DOI)978-3-319-71597-1 (ISBN)
Konferens
9th International Conference on Image and Graphics, Shanghai
Tillgänglig från: 2018-01-02 Skapad: 2018-01-02 Senast uppdaterad: 2018-12-20Bibliografiskt granskad
5. Back to basics: Towards novel computation and arrangement of spatial sensory in images
Öppna denna publikation i ny flik eller fönster >>Back to basics: Towards novel computation and arrangement of spatial sensory in images
2016 (Engelska)Ingår i: Acta Polytechnica, ISSN 1210-2709, Vol. 56, nr 5, s. 409-416Artikel i tidskrift (Refereegranskat) Published
Abstract [en]

The current camera has made a huge progress in the sensor resolution and the low-luminance performance. However, we are still far from having an optimal camera as powerful as our eye is. The study of the evolution process of our visual system indicates attention to two major issues: the form and the density of the sensor. High contrast and optimal sampling properties of our visual spatial arrangement are related directly to the densely hexagonal form. In this paper, we propose a novel software-based method to create images on a compact dense hexagonal grid, derived from a simulated square sensor array by a virtual increase of the fill factor and a half a pixel shifting. After that, the orbit functions are proposed for a hexagonal image processing. The results show it is possible to achieve image processing operations in the orbit domain and the generated hexagonal images are superior, in detection of curvature edges, to the square images. We believe that the orbit domain image processing has a great potential to be the standard processing for hexagonal images.

Ort, förlag, år, upplaga, sidor
CZECH TECHNICAL UNIV PRAGUE, 2016
Nyckelord
Convolution, Fill factor, Hexagonal pixel, Hexagonal processing, Hexagonal sensor array, Orbit functions, Orbit transform, Square pixel
Nationell ämneskategori
Annan data- och informationsvetenskap
Identifikatorer
urn:nbn:se:bth-13798 (URN)10.14311/AP.2016.56.0409 (DOI)000411584300010 ()2-s2.0-85008440583 (Scopus ID)
Anmärkning

open access

Tillgänglig från: 2017-01-20 Skapad: 2017-01-20 Senast uppdaterad: 2018-12-20Bibliografiskt granskad
6. The impact of curviness on four different image sensor forms and structures
Öppna denna publikation i ny flik eller fönster >>The impact of curviness on four different image sensor forms and structures
2018 (Engelska)Ingår i: Sensors, ISSN 1424-8220, E-ISSN 1424-8220, Vol. 18, nr 2, artikel-id 429Artikel i tidskrift (Refereegranskat) Published
Abstract [en]

The arrangement and form of the image sensor have a fundamental effect on any further image processing operation and image visualization. In this paper, we present a software-based method to change the arrangement and form of pixel sensors that generate hexagonal pixel forms on a hexagonal grid. We evaluate four different image sensor forms and structures, including the proposed method. A set of 23 pairs of images; randomly chosen, from a database of 280 pairs of images are used in the evaluation. Each pair of images have the same semantic meaning and general appearance, the major difference between them being the sharp transitions in their contours. The curviness variation is estimated by effect of the first and second order gradient operations, Hessian matrix and critical points detection on the generated images; having different grid structures, different pixel forms and virtual increased of fill factor as three major properties of sensor characteristics. The results show that the grid structure and pixel form are the first and second most important properties. Several dissimilarity parameters are presented for curviness quantification in which using extremum point showed to achieve distinctive results. The results also show that the hexagonal image is the best image type for distinguishing the contours in the images. © 2018 by the authors. Licensee MDPI, Basel, Switzerland.

Ort, förlag, år, upplaga, sidor
MDPI AG, 2018
Nyckelord
Critical points, Curviness quantification, Fill factor, Grid structure, Hessian matrix, Hexagonal image, Pixel form, Software-based, Virtual, Image processing, Image sensors, Semantics, Grid structures, Hessian matrices, Pixels
Nationell ämneskategori
Annan elektroteknik och elektronik
Identifikatorer
urn:nbn:se:bth-15920 (URN)10.3390/s18020429 (DOI)000427544000112 ()2-s2.0-85041511195 (Scopus ID)
Anmärkning

open access

Tillgänglig från: 2018-02-22 Skapad: 2018-02-22 Senast uppdaterad: 2018-12-20Bibliografiskt granskad
7. Virtual deformable image sensors: Towards to a general framework for image sensors with flexible grids and forms
Öppna denna publikation i ny flik eller fönster >>Virtual deformable image sensors: Towards to a general framework for image sensors with flexible grids and forms
2018 (Engelska)Ingår i: Sensors, ISSN 1424-8220, E-ISSN 1424-8220, Vol. 18, nr 6, artikel-id 1856Artikel i tidskrift (Refereegranskat) Published
Abstract [en]

Our vision system has a combination of different sensor arrangements from hexagonal to elliptical ones. Inspired from this variation in type of arrangements we propose a general framework by which it becomes feasible to create virtual deformable sensor arrangements. In the framework for a certain sensor arrangement a configuration of three optional variables are used which includes the structure of arrangement, the pixel form and the gap factor. We show that the histogram of gradient orientations of a certain sensor arrangement has a specific distribution (called ANCHOR) which is obtained by using at least two generated images of the configuration. The results showed that ANCHORs change their patterns by the change of arrangement structure. In this relation pixel size changes have 10-fold more impact on ANCHORs than gap factor changes. A set of 23 images; randomly chosen from a database of 1805 images, are used in the evaluation where each image generates twenty-five different images based on the sensor configuration. The robustness of ANCHORs properties is verified by computing ANCHORs for totally 575 images with different sensor configurations. We believe by using the framework and ANCHOR it becomes feasible to plan a sensor arrangement in the relation to a specific application and its requirements where the sensor arrangement can be planed even as combination of different ANCHORs. © 2018 by the authors. Licensee MDPI, Basel, Switzerland.

Ort, förlag, år, upplaga, sidor
MDPI AG, 2018
Nyckelord
Deformable sensor, Framework, Hexagonal, HoG, Penrose, Pixel form, Sensor grid, Deformation, Image sensors, Pixels, Histogram of gradients, Sensor arrangements, Sensor configurations, Sensor grids, Specific distribution, Anchors
Nationell ämneskategori
Annan elektroteknik och elektronik Annan data- och informationsvetenskap
Identifikatorer
urn:nbn:se:bth-16631 (URN)10.3390/s18061856 (DOI)000436774300190 ()2-s2.0-85048303219 (Scopus ID)
Anmärkning

open access

Tillgänglig från: 2018-06-27 Skapad: 2018-06-27 Senast uppdaterad: 2018-12-20Bibliografiskt granskad

Open Access i DiVA

fulltext(9349 kB)10 nedladdningar
Filinformation
Filnamn FULLTEXT01.pdfFilstorlek 9349 kBChecksumma SHA-512
fa3afdd054845afd48811f28ff95bcb60c8855f67747ee0b8a5bb06aa7465bd1d31c1f1f1a8d0de0c44c1f01667e505428ae7cca06907647700b83618313437d
Typ fulltextMimetyp application/pdf

Sök vidare i DiVA

Av författaren/redaktören
Wen, Wei
Av organisationen
Institutionen för teknik och estetik
Elektroteknik och elektronik

Sök vidare utanför DiVA

GoogleGoogle Scholar
Totalt: 10 nedladdningar
Antalet nedladdningar är summan av nedladdningar för alla fulltexter. Det kan inkludera t.ex tidigare versioner som nu inte längre är tillgängliga.

isbn
urn-nbn

Altmetricpoäng

isbn
urn-nbn
Totalt: 257 träffar
1231 av 3
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf