EECS209ABME295 - Rendering Techniques for Biomedical Imaging
4/5/2007
Rendering Pipeline – scanning/acquisition to image processing to rendering
So we need to know how the raw material is gathered
X rays penetrate tissue. What is recorded on film is x-ray absorption. Electrodes excited to anode and hits plate, energy loss is released as radiaton. Ionizing radiation. Bones appear white, and tissue, gray. No radiation absorbed is black.
Where to place screw into curved bone with fracture. Denser bone is where more stressed is placed on the body.
Want 3D image. Take xrays from different angles. CT takes images with low does of xrays, 10 times the amount of radiation than conventional xray. Use xrays with 3d database to do the imaging.
Need standards for left, right, top, down.
MRI soft tissue can be distinguished better; bone is black, doesn’t show; 3 standard views – transversal, …
X-ray absorption recorded on film (shadow image)
Ionizing radiation
Negative image – Bone appears white
harmful effects of x-rays include tissue burns or destroy part of DNA (worst case)
bone is like a sponge – bone material expands and contracts based on pressure
old method – risk of destroying nerves, blood vessels
new method – minimal invasive technique
put pin in area of high bone density
3D scan – CAT scan (CT) – high doses of radiation
MRI – no adverse effects
Trade off angles vs. radiation exposure
Correlation with database
e.g. Scaphoid Bone
Joseph Paul Jernigan donated his body to science – visible human project
CT Scan:
• Computed Tomography
• Computed method from many different X-ray angles
• Best for bone
• Cross-sectional rather than shadow image
• Structural scanning
• Records line integral from x-ray source – object – detector
• Uses translation, rotation (leads to round image)
PET Scan:
• Positron Emmision Tomography
• Uses isotopes (contrast agent), looks for protons
• Functional scanning
• Scan 30 minutes after activity
Functional MRI:
• Functional scanning
• Not as good resolution as standard MRI
• Better resolution than PET Scan
• Quality proportional to magnet strength
• Scan immediate after activity (1-5 minutes)
MRI Scan:
• Magnetic Resonance Imaging
• Measures proton density, moist materials show up white
• Bone is black
• Cartilege is visible
• Detail in the brain
• Best for soft tissue
• Structural scanning
Views:
• statutal, sagittal – side slice
• coronal – front of body
• axial (transversal) – orthogonal to long axis of body
Radon Transform – on midterm, do example at home
Inverse Radon Transform
MRI
1) Apply a directional magnetic fields
2) Apply a second orthogonal pulse
3) The relaxation time of the proton spin gives off a characteristic frequency
4) Stronger signal – more hydrogen & vice-versa
PET scanner – inject contrast agent that contains isotope which emits positrons that can be detected. Records brain activity. Sugar levels higher in areas that have higher activity. Functional scan. Whereas ct and mri are for structural scanning.
Optical coherence tomography – laser scanned image. Reference beam superimposed with reflective beam
• High resolution
• Non-invasive
• Scans 3-4mm
Confocal Laser Scan Microscopy
• Optical with laser light
• One focal plane w/ blurred info from other layers
Volume rendering – stacked slices, z is slicing direction
Assign transparency based on pixel value
Low intensity: transparent
High intensity: opaque
Transfer function needed for mapping
2 types: Transparency, Intensity
transfer function(transparency): tt(p(x,y,z)) = 1 – to(p(x,y,z)) = 1 – p(x,y,z);
transfer function(opacity): tt(p(x,y,z)) = p(x,y,z)
Intensity
Pixel – Picture element
Voxel – Volume element
Open GL 3D texture mapping
Application areas:
• Cancer Research
• Neuroscience
• Cell Physiology
• Molecular Diagnostics
X-ray absorption recorded on film (shadow image)
April 10, 2007
Xray image integrates the space and accumulates everything along the way. It’s a through image.
CT scan is a cross section of image, same for all tomographic methods. To get 3D image, take 2D cross sections and reconstruct.
CT scanner detector records line integral, but you can’t tell where the object is. After one scan, you have one 1D array of line integrals. Then you rotate the source and detector and record new integral.
Scanning region is round-shaped
Inverse radon transformation
If reading, shade along entire line, where intersect, that’s a point. Higher signal is the summation or intersection. Stripes are noise, additively, signal will be higher and higher at point of object
Lab 4/13
Digital Subtraction angiography – difference in images before and after injecting contrast agent
Gradient based method for edge detection
Histogram Enhancement Scaling function spreads out the entire value of grey values:
G(i) = [(e(i) – i1)/(i2-i1)]*Imax
Normalized Cumulative Histogram – characterized by a diagonal from 0 to P-1
Use Gaussian distribution for local pixel averaging
Noise Reduction
Assumption: Noise is equally distributed
On average, signal is correct.
Solution: Average the image over a somewhat larger area.
4/17/2007
http://imaging.eng.uci.edu/~jmeyer/COURSES/BME295/xray.jpg
Digital subtraction and geography
Get negative image so blood vessels appear white, then remove the bones so only blood vessels left.
http://imaging.eng.uci.edu/~jmeyer/COURSES/BME295/xray2.jpg
non-uniform exposure
overexposed on the left hand side
standard thresholding to turn into b&w didn’t help much
how can we extract the teeth
used laplacian or differential filter transfer function
made it a grayscale image first because color doesn’t convey any information
try to find quantifiable information in image to help define a good transfer function
my histogram for teeth:
1. Image Enhancement
Histogram:
Intensity Scaling: define an intensity band of interest with min and max
Spreads out the histogram over the entire available range of gray values also known as histogram enhancement
(Pixel-min)*max / (max-min)
Histogram Equalization
Goal: histogram that is equal everywhere
Technique: Normalized cumulative histogram
4/19/2007
Histogram equalization
want the histogram to look like a straight line, all frequencies have the same value
Take original histogram and compute the cumulative histogram, sum up the number of pixels in each bin. Cumulative histogram, looks like a straight incrementing diagonal line. Skip empty areas. Next time you get a meaningful value, that’s when you make the next step in the cumulative histogram.
Noise reduction
Assumption: noise is equally distributed
Solution: average image over a larger area
Convolution Filter Kernel::
3×3 Mean Filter 1/9 [1 1 1] (square)
3×3 Gaussian Filterl: [1 2 1] [2 4 2] [1 2 1] x (1/16)
Image buffers for 3×3: 2
Edge Enhancement: [1 1 1][0 0 0][-1 –1 –1]
Omnidirectional [ -1 –1 –1][-1 8 –1][-1 –1 –1]
Border pixels
1) Design special filters for border pixels
2) Mirror Border pixels
3) Fill pixels
4/24
Temporal Averaging of Images
Assumption: Noise is equally distributed
Can average over time instead of averaging over an area. Take multiple samples of the image, the noise will add to zero and the image will be resulting. Assumes the image does not move. At infinite, zero noise. the spatial resolution is preserved, structures come out more clear.
Noisy image is a composition of the original or noise free image and noise
Summation of noise over time as time -> infinite, equals zero
Assumption, noise has a higher frequency than the image.
Stripes, one frequency
Two frequencies, almost orthogonal
If we know the noise frequency, we can design a filter
Adaptive image filtering (Chapter 2 in text)
Temporal Averaging of Images
Goal: Distinguish image features from the noise
Assumption: Noise is equally distributed
Ai(m,n,) = f(m,n) + di(m,n)
Observation: High frequency noise is more distracting than lower frequency noise. This is the most common case. There are plenty of cases where just the opposite it true.
Solution: (Can just eliminate high frequencies, that’s what a 3×3 averaging filter does.) Use frequency specific filtering, band-pass filtering.
Method: Transformation from spatial to frequency domain
Discrete Fourier Transform (DFT)
f(x) = a0 + summation of a*cos(kx) + b*sin(kx)
Basis function, all possible waves can be derived from them. They are usually orthogonal to each other. a*coskx and b*sinkx are dilated and shifted basis functions.
Thursday 4/26 Adaptive Filtering: Wavelet Decomposition
Wiener filter’s lowpass characteristics give rise to unacceptable blurring of lines and edges.
Backus-Gilbert criterion – maximize stability, rather than smoothness
High-frequency noise can be reduced by eliminating the corresponding coefficiencts.
Problems – Reconstruction is only correct for a single point (and a good approximation for a neighborhood of that point). This method does not extend well to higher dimensions.
Solution: Use a different set of basis functions. – finite support – orthogonal – box shape,
Wavelet Transformation
• Transforms a signal from the spatial to the frequency doman
• Preserves spatial information
• Decomposition of the image into frequency components
Friday 4/27 Lab 2
Code:
http://imaging.eng.uci.edu/~jmeyer/ppm_reader.txt
Image:
http://imaging.eng.uci.edu/~jmeyer/COURSES/BME295/lab2.ppm
PPM – portable pixel map
Magic Numbers for ASCII Variants:
P1 – b&w, 1-bit per pixel (.pbm)
P2 – grayscale, 1 byte per pixel (.pgm)
P3 – Color Images, 16.7 million possible (.ppm) colors per pixel
PPM Syntax:
P3
640 480 (size resolution)
#comment, Ex. (c) Pearl Ho 2007
255 (maximum value each pixel can have)
RGBRGBRGBRGBRGBRGBRGBRGBRGBRGBRGBRGBRGB…pixel values
Magic Numbers for Binary Variants:
P4 – b&w (.pbm)
P5 – grayscale (.pgm)
P6 – color image (.ppm)
For Lab 2 Assignment:
Use ppm_reader code. Write a ppm_writer that just copies the first file.
Adaptive Filtering: Wavelet Decomposition Tuesday 5/01
Non-standard Decomposition
LL LH HL HH
L’ = (l1+l2)/2
H’ = (l1-l2)/2
H’ needs to shift by the maxval/2 to get positive numbers
Midterm Topics:
1. Rendering Pipeline – acquisition modalities
2. Histograms – image enhancement/histogram transformations
3. Image Enhancement – noise reduction, convolution filter kernels (border cases), edge enhancement (horizontal, vertical, omnidirectional), image averaging, digital subtraction angiography
4. Adaptive Image Filtering – Fourier transform, wavelet decomposition/reconstruction (1D, 2D, 3D, ...)
May 8, 2007
Constructing a Wavelet Basis
We associate an image with a vector space V, which consists of a set of piecewise linear functions.
V` is associated with a constant function, which is defined over the domain [0,1] and describesa single pixel.
Vi consists of 2i intervals with a constant function defined on each of the intervals. All vector spaces are subsets of each other:
Vi ( V(i+1), i belongs to N (integers?)
requirements for basis functions
has finite support, only exists between 0-1, it’s zero when greater than one
number of basis functions depends on the size of the image, which should be a power of two. you can bring it up to the next power of two if it is not, and fill with zeroes.
orthogonal – the V functions are not orthogonal, because higher iterations can form lower ones.
We define another vector space Wi, which comprises all functions of V(i+1) and that is orthogonal to all functions in V^i.
The set of basis functions V0, W0…Wi is called a wavelet basis. A discrete signal or the original image is defined the space V(i+1) can be represented as a linear combination of the basis functions from the wavelet basis
a 4 pixel image would have 10 basis functions
V0, 2 for V1, 4 for V2, W0, 2 for W^1; phi00, 10, 11, 20, 21, 22, 23, psi 00, 10, 11
phi(x) = 1 if 0 <= x < 1, 0 otherwise
phi i,j (x) = phi ( 2i * x – j) , j = {0, ..., 2i – 1} , 2^i scales, j shifts…so it shifts, and then dilates
psi(x) = { 1 if 0 <= x < 1/2 , -1 if 1/2 < x < 1, 0 otherwise
psi i,j (x) = psi ( 2i * x – j) , j = {0, ..., 2i – 1}
haar wavelet basis
5/10
Segmentation
Image processing part of rendering pipeline.
Segmentation: Separation of structures of interest from the background
Separation of: anatomical structures, tissue types (grey vs. white matter), function/activity regions, pathology
2D/3D/4D/...nD no limit to dimensions, colors assigned during rendering
Representations: binary image (pixel-based), histogram (grey-level based), contours, area/volume/hypervolume
Criteria – intensity threshold, texture, intensity gradient, texture gradient
laplacian filter to get the edge to help define the the volume that you want to segment
marr-hildreth operator – filter. blurs image to get rid of high frequency noise so you don’t have isolated pixels in image, then laplacian filter to enhance countours, then thresholding filter.
looking for tumors – they’re usually surrounded by fluid
DTI – Diffusion Tensor Imaging. it looks cool. new research. still debating on what the images are showing.
Techniques: contour finding algorithm (ex. marr-hildreth), thresholding (global or adaptive), watershed segmentation, gradient operators, region growing algorithm, multi-image analysis (eg, subtraction)
Methods: supervised(interactive), unsupervised (fully automated)
Textbook:
II. 5.1 – 5.3
read watershed segmentation in particular
5.2 Thresholding, Global & Local (Adaptive), preprocessing (median, gaussian blurs)
5.3 Region Growing – hill climbing
5.4 Watershed
Thursday May 17, 2007
Segmentation Algorithms
1. Global Thresholding
How do we define a threshold? Find a good on by hand, manually. Or, if you want to automate it, you want to chose one that the T and T+deltaT, are not very different. Difference is delta A. delta A is minimal, the slope will be maximal. you want d A(T) / dT to be minimal. Threshold depends on the image; content of image needs to be taken into account.
2. Local Thresholding
Define image in to subareas. Tiles, hexagons, ... but how to you define a meaningful subarea?
The same threshold may not work for the entire image. Make sure your sections have both foreground and background pixels.
Hints: An averaging or Gaussian filter may improve results by reducing noise. Or apply a sequence of filters. Calibration can be done by hand by having an expert mark and image.
3. Region Growing
Find the seed point.
make sure you’re inside the contour.
scan line algorithm.
examine neighboring pixels for similarity, and if they are the same, add them to the region. multiple seed points may be necessary. for complete segmentation.
4. Wavelet Transform
Texture Analysis
-Represent objects on multiple scales
-Select one or more levels of low-pass filter coefficients to pick objects or patterns of a particular size.
-Set other coefficients to zero. Like in the LL parts. Whole image will be black besides elements of that particular size.
OR:
Texture Gradient Analysis
-if you’re interested in texture, look at the high frequency component.
-select one or more levels of high-pass filter coefficients to pick particular gradients. Set other coefficients to zero.
The end of Segmentation, next week, rendering techniques. No lab tomorrow.
Tuesday May 22, 2007
Rendering
Need to make some opacity transfer function. Ex. black values are completely transparent.
History of computer graphics.
3D Visualization
Method: Surface Rendering
-Extract edges (requires thresholding)
-connect edges to form polygons, usually triangles
-isolines, values of the same intensity are connected. for 3D, isosurface
“marching cubes” algorithm
-multiple thresholds will create multiple isosurfaces
Method: Direct Volume Rendering
Final Projects
1. Wavelet-based quantitative measure to characterize the texture of a fabric as vertical or horizontal.
2. Wavelet-based autofocus metric.
3. Laplacian-based autofocus.
4. 3D texture-based volume rendering algoritm.
5. Point-based volume rendering (with illumination).
6. Plane slider to overview 3D dataset. Orthogonal slice browser.
7. “Realistic” color transfer function for black and white (grayscale) video.
8. Color correction for night-vision camera.
9. pseudo-color anaglyph.
10. auto stereo rendering combined with one of the above renderings.
11. TBD by you.
Topic titles emailed to professor by Thursday.
May 24, 2007
Method: Direct Volume Rendering
marching cubes algorithm – lookup table w/256 elements, case table
Thursday, May 31, 2007
Volume Rendering
Lambert
Blinn-Phong Shading Model
Texture-Based Volume Rendering
Z-buffer algorithm, part of projection algorithm
Back-to-Front vs. Front-to-Back Projection
Back-to-front (Painter’s algorithm)
c: color
alpha: opacity
c0 = cbackground
c(i+1) = c(sample i) * alpha(sample i) + ci(1- alpha(sample i))
Advantage: Efficient implementation
Front-to-back:
c0 = 0
alpha0 = 1 (remaining opacity)
c(i+1) = c(sample i) * alpha(i) + ci(1- alpha(i))
alpha(i+1) = alphai * (1-alphasamplei)
Final color : c(i+i) = cbackground * alphai + ci(1- alpha(i))
Tuesday, June 5, 2007
Volume Rendering
Ray casting ignores transparency
Massively parallel volume rendering (in hardware)
Shear-Warp Algorithm
good for parallel projection
Disadvantage – Perspective difficult
2D Texture Mapping
- Map entire picture to new object
- Map segments of picture to segments of new object
Search Wiki:
· Categories
· Title List
· Uncategorized Pages
· Random Page
· File Upload
· Uploaded Files
· Recent Changes
· What Links Here