RefereraExporteraLink to record
Permanent link

Direct link
Referera
Referensformat
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annet format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annet språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf
Biological Inspired Deformable Image Sensor
Blekinge Tekniska Högskola, Fakulteten för datavetenskaper, Institutionen för teknik och estetik.ORCID-id: 0000-0003-3887-5972
2019 (engelsk)Doktoravhandling, med artikler (Annet vitenskapelig)
Abstract [en]

Nowadays, cameras are everywhere thanks to the tremendous progress on sensor technology. However, their performance is far away from what we experience by our eyes. The study from evolution process shows how the sensor arrangement of retina in human vision has differentiated from other species and is formed into a specific combination of sub-arrangements from hexagonal to elliptical ones. There are three major key differences between our visual cell arrangement and current camera sensors which are: the sub-arrangements, the pixel form and the pixel density.

Despite the advances in sensor technology we face limitations in their further development; i.e. to make the cameras close to the visual system. This is due to the optical diffraction limit which prevents us to increase the sensor resolution, and rigidity of hardware implementation which prevent us to change the image sensor after manufacturing. In the thesis the possibilities to overcome such limitations are investigated where the intention is to find a closer sensory solution to the visual system in comparison to the current ones.

Breaking the diffraction barrier and solving the rigidity problem are simultaneously achieved by introducing and estimating virtual subpixels. A statistical framework consisting of local learning model and Bayesian inference for predicting the incident photons captured on each such a subpixel is used to resample the captured image by any current camera sensor. By investigating the virtual variation of pixel size and fill factor the validity of the proposed idea is proven by which the results show significant changes of dynamic range and tonal levels in relation to the variation. As an example, for both monochrome and color images the results show that by virtual increase of fill factor to 100%, the dynamic range of the images are widened and the tonal levels are enriched significantly over 256 levels for each channel.

The results of virtual variation of the fill factor and pixel size indicates that it is feasible to change the rigidity of the image sensor using the software-based method. Inspired by the mosaic in the fovea, the center of human retina, the hexagonal sub-arrangement and pixel form are proposed to generate images based on the estimated virtual subpixels. Compared to the original square images, not only the dynamic range and tonal levels are improved, but also the hexagonal images are superior in detection of edges, i.e. more edge points on the contour of the objects are detected in hexagonal images.

The evaluation of different sub-arrangements or pixel forms of the image sensor is a challenging task and should be directed to a more specific task. Since the curvature contours contain most of the information related to object perception and human vision is highly evolved to detect curvature object, the task is focused to investigate the impact of the curviness on the different pixel forms and sub-arrangements, by comparing two categories of images; having curved versus linear edges of the objects in a pair of images which have exact similar contents but different contours. The detectability of each of the different sensor structures for curviness is estimated and the results show that the image on hexagonal grid with hexagonal pixel form is the best image type for distinguishing the curvature contours in the images.

According to the pattern of pixels tiling, there are two types of pixel sub-arrangements, i.e. periodic (e.g. square or hexagonal), and aperiodic (e.g. Penrose). Each type of sub-arrangements is investigated where the pixel forms and density are variable. By having at least two generated images of one configuration (i.e. specific sub-arrangement, pixel form and density), the result of histogram of gradient orientation of the certain sensor arrangement shows a stable and specific distribution which we called it the ANgular CHaracteristic of a sensOR structure (ANCHOR). Each ANCHOR has a robust pattern which is changed by the change of the sensor sub-arrangement. This makes it feasible to plan a sensor sub-arrangement in the relation to a specific application and its requirements, and more alike the biological vision sensory. To generate such a flexible sensor, a general framework is proposed for virtual deforming the sensor with a certain configuration of the sensor sub-arrangement, pixel form and pixel density.

Assessing the quality difference between the images generated by different sensor configuration or addressing from on configuration to another one generally needs the conversion of one to another. To overcome this problem, a common space is proposed by implementing a continuous extension of square or hexagonal images based on the orbit function, for quality evaluating the images with different arrangements and addressing from one type of image to another one. The evaluation results show that the creation of such space is feasible which facilitates a usage friendly tool to address an arrangement and assess the changes between different spatial arrangements, for example, it shows richer intensity variation, nonlinear behavior, and larger dynamic range in the hexagonal images compared to the rectangular images.

sted, utgiver, år, opplag, sider
Karlskrona: Blekinge Tekniska Högskola, 2019. , s. 207
Serie
Blekinge Institute of Technology Doctoral Dissertation Series, ISSN 1653-2090 ; 4
Emneord [en]
image sensor, pixel form, sub-arrangements, fill factor, square image, hexagonal image, deformable sensor, quality assessment.
HSV kategori
Identifikatorer
URN: urn:nbn:se:bth-17149ISBN: 978-91-7295-366-6 (tryckt)OAI: oai:DiVA.org:bth-17149DiVA, id: diva2:1272821
Disputas
2019-03-14, J1650, Campus Gräsvik, Karlskrona, 13:15 (engelsk)
Opponent
Veileder
Tilgjengelig fra: 2018-12-20 Laget: 2018-12-20 Sist oppdatert: 2019-03-05bibliografisk kontrollert
Delarbeid
1. Novel Software-based Method to Widen Dynamic Range of CCD Sensor Images
Åpne denne publikasjonen i ny fane eller vindu >>Novel Software-based Method to Widen Dynamic Range of CCD Sensor Images
2015 (engelsk)Inngår i: / [ed] Yu-Jin Zhang, Springer, 2015, Vol. 9218, s. 572-583Konferansepaper, Publicerat paper (Fagfellevurdert)
Abstract [en]

In the past twenty years, CCD sensor has made huge progress in improving resolution and low-light performance by hardware. However due to physical limits of the sensor design and fabrication, fill factor has become the bottle neck for improving quantum efficiency of CCD sensor to widen dynamic range of images. In this paper we propose a novel software-based method to widen dynamic range, by virtual increase of fill factor achieved by a resampling process. The CCD images are rearranged to a new grid of virtual pixels com-posed by subpixels. A statistical framework consisting of local learning model and Bayesian inference is used to estimate new subpixel intensity. By knowing the different fill factors, CCD images were obtained. Then new resampled images were computed, and compared to the respective CCD and optical image. The results show that the proposed method is possible to widen significantly the recordable dynamic range of CCD images and increase fill factor to 100 % virtually.

sted, utgiver, år, opplag, sider
Springer, 2015
Serie
Lecture Notes in Computer Science, ISSN 0302-9743
Emneord
Dynamic range, Fill factor, CCD sensors, Sensitive area, Quantum efficiency
HSV kategori
Identifikatorer
urn:nbn:se:bth-11169 (URN)10.1007/978-3-319-21963-9_53 (DOI)978-3-319-21963-9 (ISBN)
Konferanse
International Conference on Image and Graphics 2015, Tianjin, China
Tilgjengelig fra: 2015-12-11 Laget: 2015-12-11 Sist oppdatert: 2018-12-20bibliografisk kontrollert
2. A Software Method to Extend Tonal Levels and Widen Tonal Range of CCD Sensor Images
Åpne denne publikasjonen i ny fane eller vindu >>A Software Method to Extend Tonal Levels and Widen Tonal Range of CCD Sensor Images
2015 (engelsk)Inngår i: 2015 9TH INTERNATIONAL CONFERENCE ON SIGNAL PROCESSING AND COMMUNICATION SYSTEMS (ICSPCS), IEEE Communications Society, 2015Konferansepaper, Publicerat paper (Fagfellevurdert)
Abstract [en]

As one of important outcomes of the past decades of researches on sensor arrays for digital cameras, the manufacturers of sensor array technology have responded to the necessity and importance of obtaining an optimal fill factor, which has great impact on collection of incident photons on the sensor, with hardware solution e.g. by introducing microlenses. However it is still impossible to make a fill factor of 100% due to the physical limitations in practical development and manufacturing of digital camera. This has been a bottle neck problem for improving dynamic range and tonal levels for digital cameras e.g. CCD cameras. In this paper we propose a software method to not only widen the recordable dynamic range of a captured image by a CCD camera but also extend its tonal levels. In the method we estimate the fill factor and by a resampling process a virtual fill factor of 100% is achieved where a CCD image is rearranged to a new grid of virtual subpixels. A statistical framework including local learning model and Bayesian inference is used for estimating new sub-pixel intensity values. The highest probability of sub-pixels intensity values in each resampled pixel area is used to estimate the pixel intensity values of the new image. The results show that in comparison to the methods of histogram equalization and image contrast enhancement, which are generally used for improving the displayable dynamic range on only one image, the tonal levels and dynamic range of the image is extended and widen significantly and respectively.

sted, utgiver, år, opplag, sider
IEEE Communications Society, 2015
Emneord
CCD sensor, tonal range, fill factor, quantum efficiency
HSV kategori
Identifikatorer
urn:nbn:se:bth-12967 (URN)000380405700050 ()978-1-4673-8118-5 (ISBN)
Eksternt samarbeid:
Konferanse
9th International Conference on Signal Processing and Communication Systems (ICSPCS), DEC 14-16, 2015, Cairns, AUSTRALIA
Tilgjengelig fra: 2016-08-31 Laget: 2016-08-30 Sist oppdatert: 2018-12-20bibliografisk kontrollert
3. Estimation of Image Sensor Fill Factor Using a Single Arbitrary Image
Åpne denne publikasjonen i ny fane eller vindu >>Estimation of Image Sensor Fill Factor Using a Single Arbitrary Image
2017 (engelsk)Inngår i: Sensors, ISSN 1424-8220, E-ISSN 1424-8220, Vol. 17, nr 3, s. 620-Artikkel i tidsskrift (Fagfellevurdert) Published
Abstract [en]

Achieving a high fill factor is a bottleneck problem for capturing high-quality images. There are hardware and software solutions to overcome this problem. In the solutions, the fill factor is known. However, this is an industrial secrecy by most image sensor manufacturers due to its direct effect on the assessment of the sensor quality. In this paper, we propose a method to estimate the fill factor of a camera sensor from an arbitrary single image. The virtual response function of the imaging process and sensor irradiance are estimated from the generation of virtual images. Then the global intensity values of the virtual images are obtained, which are the result of fusing the virtual images into a single, high dynamic range radiance map. A non-linear function is inferred from the original and global intensity values of the virtual images. The fill factor is estimated by the conditional minimum of the inferred function. The method is verified using images of two datasets. The results show that our method estimates the fill factor correctly with significant stability and accuracy from one single arbitrary image according to the low standard deviation of the estimated fill factors from each of images and for each camera.

sted, utgiver, år, opplag, sider
MDPI, 2017
Emneord
fill factor; virtual image; image sensor; pipeline; virtual response function; sensor irradiance
HSV kategori
Identifikatorer
urn:nbn:se:bth-14046 (URN)10.3390/s17030620 (DOI)000398818700193 ()
Merknad

open access

Tilgjengelig fra: 2017-03-24 Laget: 2017-03-24 Sist oppdatert: 2018-12-20bibliografisk kontrollert
4. Image Quality Assessment of Enriched Tonal Levels Images
Åpne denne publikasjonen i ny fane eller vindu >>Image Quality Assessment of Enriched Tonal Levels Images
2017 (engelsk)Inngår i: Image and Graphics 9th International Conference, ICIG 2017, Shanghai, China, September 13-15, 2017, Revised Selected Papers, Part II / [ed] Yao Zhao, Xiangwei Kong, David Taubman, Springer, 2017, s. 134-146Konferansepaper, Publicerat paper (Fagfellevurdert)
Abstract [en]

The quality assessment of a high dynamic image is a challenging task. The few available no reference image quality methods for high dynamic range images are generally in evaluation stage. The most available image quality assessment methods are designed to assess low dynamic range images. In the paper, we show the assessment of high dynamic range images which are generated by utilizing a virtually flexible fill factor on the sensor images. We present a new method in the assessment process and evaluate the amount of improvement of the generated high dynamic images in comparison to original ones. The results show that the generated images not only have more number of tonal levels in comparison to original ones but also the dynamic range of images have significantly increased due to the measurable improvement values.

sted, utgiver, år, opplag, sider
Springer, 2017
Emneord
Image quality assessment, High dynamic range image, Tonal levels, Tone mapping, Fill factor
HSV kategori
Identifikatorer
urn:nbn:se:bth-15709 (URN)10.1007/978-3-319-71598-8_13 (DOI)978-3-319-71597-1 (ISBN)
Konferanse
9th International Conference on Image and Graphics, Shanghai
Tilgjengelig fra: 2018-01-02 Laget: 2018-01-02 Sist oppdatert: 2018-12-20bibliografisk kontrollert
5. Back to basics: Towards novel computation and arrangement of spatial sensory in images
Åpne denne publikasjonen i ny fane eller vindu >>Back to basics: Towards novel computation and arrangement of spatial sensory in images
2016 (engelsk)Inngår i: Acta Polytechnica, ISSN 1210-2709, Vol. 56, nr 5, s. 409-416Artikkel i tidsskrift (Fagfellevurdert) Published
Abstract [en]

The current camera has made a huge progress in the sensor resolution and the low-luminance performance. However, we are still far from having an optimal camera as powerful as our eye is. The study of the evolution process of our visual system indicates attention to two major issues: the form and the density of the sensor. High contrast and optimal sampling properties of our visual spatial arrangement are related directly to the densely hexagonal form. In this paper, we propose a novel software-based method to create images on a compact dense hexagonal grid, derived from a simulated square sensor array by a virtual increase of the fill factor and a half a pixel shifting. After that, the orbit functions are proposed for a hexagonal image processing. The results show it is possible to achieve image processing operations in the orbit domain and the generated hexagonal images are superior, in detection of curvature edges, to the square images. We believe that the orbit domain image processing has a great potential to be the standard processing for hexagonal images.

sted, utgiver, år, opplag, sider
CZECH TECHNICAL UNIV PRAGUE, 2016
Emneord
Convolution, Fill factor, Hexagonal pixel, Hexagonal processing, Hexagonal sensor array, Orbit functions, Orbit transform, Square pixel
HSV kategori
Identifikatorer
urn:nbn:se:bth-13798 (URN)10.14311/AP.2016.56.0409 (DOI)000411584300010 ()2-s2.0-85008440583 (Scopus ID)
Merknad

open access

Tilgjengelig fra: 2017-01-20 Laget: 2017-01-20 Sist oppdatert: 2018-12-20bibliografisk kontrollert
6. The impact of curviness on four different image sensor forms and structures
Åpne denne publikasjonen i ny fane eller vindu >>The impact of curviness on four different image sensor forms and structures
2018 (engelsk)Inngår i: Sensors, ISSN 1424-8220, E-ISSN 1424-8220, Vol. 18, nr 2, artikkel-id 429Artikkel i tidsskrift (Fagfellevurdert) Published
Abstract [en]

The arrangement and form of the image sensor have a fundamental effect on any further image processing operation and image visualization. In this paper, we present a software-based method to change the arrangement and form of pixel sensors that generate hexagonal pixel forms on a hexagonal grid. We evaluate four different image sensor forms and structures, including the proposed method. A set of 23 pairs of images; randomly chosen, from a database of 280 pairs of images are used in the evaluation. Each pair of images have the same semantic meaning and general appearance, the major difference between them being the sharp transitions in their contours. The curviness variation is estimated by effect of the first and second order gradient operations, Hessian matrix and critical points detection on the generated images; having different grid structures, different pixel forms and virtual increased of fill factor as three major properties of sensor characteristics. The results show that the grid structure and pixel form are the first and second most important properties. Several dissimilarity parameters are presented for curviness quantification in which using extremum point showed to achieve distinctive results. The results also show that the hexagonal image is the best image type for distinguishing the contours in the images. © 2018 by the authors. Licensee MDPI, Basel, Switzerland.

sted, utgiver, år, opplag, sider
MDPI AG, 2018
Emneord
Critical points, Curviness quantification, Fill factor, Grid structure, Hessian matrix, Hexagonal image, Pixel form, Software-based, Virtual, Image processing, Image sensors, Semantics, Grid structures, Hessian matrices, Pixels
HSV kategori
Identifikatorer
urn:nbn:se:bth-15920 (URN)10.3390/s18020429 (DOI)000427544000112 ()2-s2.0-85041511195 (Scopus ID)
Merknad

open access

Tilgjengelig fra: 2018-02-22 Laget: 2018-02-22 Sist oppdatert: 2018-12-20bibliografisk kontrollert
7. Virtual deformable image sensors: Towards to a general framework for image sensors with flexible grids and forms
Åpne denne publikasjonen i ny fane eller vindu >>Virtual deformable image sensors: Towards to a general framework for image sensors with flexible grids and forms
2018 (engelsk)Inngår i: Sensors, ISSN 1424-8220, E-ISSN 1424-8220, Vol. 18, nr 6, artikkel-id 1856Artikkel i tidsskrift (Fagfellevurdert) Published
Abstract [en]

Our vision system has a combination of different sensor arrangements from hexagonal to elliptical ones. Inspired from this variation in type of arrangements we propose a general framework by which it becomes feasible to create virtual deformable sensor arrangements. In the framework for a certain sensor arrangement a configuration of three optional variables are used which includes the structure of arrangement, the pixel form and the gap factor. We show that the histogram of gradient orientations of a certain sensor arrangement has a specific distribution (called ANCHOR) which is obtained by using at least two generated images of the configuration. The results showed that ANCHORs change their patterns by the change of arrangement structure. In this relation pixel size changes have 10-fold more impact on ANCHORs than gap factor changes. A set of 23 images; randomly chosen from a database of 1805 images, are used in the evaluation where each image generates twenty-five different images based on the sensor configuration. The robustness of ANCHORs properties is verified by computing ANCHORs for totally 575 images with different sensor configurations. We believe by using the framework and ANCHOR it becomes feasible to plan a sensor arrangement in the relation to a specific application and its requirements where the sensor arrangement can be planed even as combination of different ANCHORs. © 2018 by the authors. Licensee MDPI, Basel, Switzerland.

sted, utgiver, år, opplag, sider
MDPI AG, 2018
Emneord
Deformable sensor, Framework, Hexagonal, HoG, Penrose, Pixel form, Sensor grid, Deformation, Image sensors, Pixels, Histogram of gradients, Sensor arrangements, Sensor configurations, Sensor grids, Specific distribution, Anchors
HSV kategori
Identifikatorer
urn:nbn:se:bth-16631 (URN)10.3390/s18061856 (DOI)000436774300190 ()2-s2.0-85048303219 (Scopus ID)
Merknad

open access

Tilgjengelig fra: 2018-06-27 Laget: 2018-06-27 Sist oppdatert: 2018-12-20bibliografisk kontrollert

Open Access i DiVA

fulltext(59692 kB)33 nedlastinger
Filinformasjon
Fil FULLTEXT02.pdfFilstørrelse 59692 kBChecksum SHA-512
afcb0f16458c41c574e8aa521b0f52937d8dfb6656cf89f667a5f7867928e7aa9b156e8538bdc6c1c1524332d704e28226aa16a190fac1a4fadc0f2ddb6a062f
Type fulltextMimetype application/pdf

Søk i DiVA

Av forfatter/redaktør
Wen, Wei
Av organisasjonen

Søk utenfor DiVA

GoogleGoogle Scholar
Totalt: 60 nedlastinger
Antall nedlastinger er summen av alle nedlastinger av alle fulltekster. Det kan for eksempel være tidligere versjoner som er ikke lenger tilgjengelige

isbn
urn-nbn

Altmetric

isbn
urn-nbn
Totalt: 952 treff
RefereraExporteraLink to record
Permanent link

Direct link
Referera
Referensformat
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annet format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annet språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf