Feature Article
Texture Synthesis for Mobile Data Communications Hirofumi Otori and Shigeru Kuriyama ■ Toyohashi University of Technology
M
ost mobile phones contain a high-quality digital camera, which can serve as a handy image code detector. Two-dimensional bar codes are frequently used to code URLs for relevant information services on the Web. The 2D bar code’s black-and-white matrix-formed patterns are efficient for robustly coding data, but their meaningless images often diminish the attractiveness of printed matter such as magazines and wrapping paper. We’ve found you can overcome this by introducing recent computer graphics technologies to the data-coding mechanism. (For more on bar codes and alternatives, see the sidebar “Related Work in Data This approach to image Coding and Hiding.”) coding first paints a regularly We propose using texture imarranged dotted pattern, ages of iterative patterns as a using colors picked from type of image code. We directly a texture sample with paint colored dotted patterns to features corresponding to be robustly detected from a phothe embedded data. It then tographed image. Then, we syncamouflages the dotted thesize texture images to conceal pattern using the same texture the patterns while preserving sample while preserving their colors. This method can quality comparable to that of embed more information than existing technologies, and its alexisting synthesis techniques. gorithm for texture image generation ensures the synthesized images’ quality. We’ve demonstrated our method’s feasibility by implementing it on a mobile phone. The results show that our robust, efficient data detection mechanism can read coded data within a reasonable response time and with a reasonable error ratio. Figure 1 shows sample seed images (Figure 1a), textures synthesized with an existing pixel-based method (Figure 1b),1 and 25-byte data-coded textures with our method (Figure 1c). Although the resulting images differ slightly, their quality is nearly the same. 74
November/December 2009
Why Use Texture Images? Using texture images to replace bar codes has two benefits. Iterative textures sometimes serve as background images in printed material, and you can embed information on some blank places unoccupied by foreground images. Another promising application is the use of a camouflage code on real material. By synthesizing a texture with a sample image scanned from a real material such as wood or cloth, you can embed data by inconspicuously affixing a seal of the printed texture on the material. This camouflage code can record product information for the material while avoiding unsightly coded images.
Data Coding Using Local Binary Patterns Our data-coding strategy first converts bit patterns of information into colored dotted patterns. We sparsely arrange colored dots and map binary values to those dots according to the magnitude of a specific color component, which we call a coded component. (In contrast, existing bar codes use densely arranged black and white dots.) Although our texture synthesis allows any type of regularly arranged pattern, we experimentally found that the existing feature vector for texture images, called the local binary pattern (LBP),2 is best suited to synthesize high-quality images with our method for many types of textures. We compute this code by comparing a centered pixel’s value against those of circularly arranged nearby pixels. We divide the coded data by P bits and compute the LBP from the difference of a coded component at a pixel at the center pc of the block and at the P circular positions regularly arranged as p n = p c + R(cos(2pn/P), -sin(2pn/P)), n = 0, ..., P - 1, with the distance R from the center (see Figure 2):
Published by the IEEE Computer Society
0272-1716/09/$26.00 © 2009 IEEE
LBPP .R ( pc ) =
P −1
∑ s ( g ( p ) − g ( p ))2 , n
c
n
n=0
1 : x ≥ 0 s ( x) = , 0 : x < 0
(1)
where g(p) denotes the coded component at p, and s(g(pn) - g(pc)) corresponds to the value of the (n + 1)th bit. We round off the coordinates of the circular positions pn to integers so that we can fit them to discrete pixel positions. We divide a region of synthesized texture image into a predefined number of square blocks. We then divide embedding data and code it by painting the LBP onto each block in a scan-line sequence (that is, from left to right, then top to bottom). To conceal this painted LBP code, we add a texture-image coating. Because each LBP can represent P bits, embedding data of n bits requires regularly arranged n/P image blocks, where denotes a ceil function. So, we recover whole data by concatenating the P bits for each block in the same scan-line sequence. We usually set P = 8 so that each block embeds one byte of data (or character). The original LBP uses a grayscale level or each RGB component as g(p), but we can replace it with an arbitrary color component that can be uniquely converted from RGB. We usually select a color component insensitive to the human vision system as the coded component—for example, the Cb component of the Y-Cb-Cr color space. However, the component must have wide distribution for ensuring robustness against color distortions. So, we must select a visually sensitive component if all insensitive components have narrow distributions.
Painting LBP Code We determine every pixel color of a synthesized texture by adaptively selecting a color from the colors in a sample image called an exemplar. We camouflage the image of LBP code by selecting constituent colors from a subset of the same group that’s classified in the next section.
Classifying Colors
Let c ( p) be the color of the pixel at p in an exemplar, and let g ( p) be the corresponding coded component. We first extract the insensitive color component as g ( p) from all pixels and compute the median g m. We then divide every pixel color c ( p) into three classes: ■■ ■■ ■■
the median class for g ( p) ≡ g m , the upper class for g ( p) ≥ g m + T , and the lower class for g ( p) ≤ g m − T ,
(a)
(b)
(c)
where we neglect the coded components in the middle range, g m − T < g ( p) < g m + T , when painting the LBP. The scalar value T isolates the coded components in the upper and lower classes from the median value gm. An increase of T enhances robustness against the noisy variation caused in printing and photographing. However, too much T greatly narrows the range of usable colors. T = 30 ensures good balance for 8-bit coded components. After dividing the exemplar’s constituent colors into three classes, we paint the synthesized texture’s central pixel with the pixel color whose coded component belongs to the median class as c ( pc ) = c ( pm ), where g ( pm ) ≡ g m. To paint the surrounding pixels, we randomly select the pixel color whose coded component belongs to the upper and lower classes for the embedded binary data of 1 and 0, respectively. We assume that texture generation is based IEEE Computer Graphics and Applications
Figure. 1. Examples of (a) sample images, (b) ordinary synthesis, and (c) data-coded synthesis. These two kinds of synthesized images have comparable quality.
75
Feature Article
Related Work in Data Coding and Hiding
D
enso Wave developed the QR (Quick Response) Code (www.qrcode.com) as an efficient, robust 2D bar code (see Figure A1). It has become a popular way to read URLs for navigating Web pages and services, without the troublesome operations of a small numerical keypad. Researchers have developed several methods to introduce an eye-pleasing aesthetic pattern. For example, ColorCode (www.colorzip.com) adds colors (see Figure A2), and designQR (http://d-qr.net) partially modifies the QR code’s patterns to create small icons (see Figure A3). These approaches, however, employ the matrix form and impose rigid constraints on the codes’ visual design. Recently, researchers have employed natural images as replacements for QR Code. Some methods intentionally modulate the image’s specific color or frequency component according to embedded data. Most methods incorporate one of two numerical techniques: watermarking or steganography.1 Watermarking has been used to protect copyright, which requires robustness against intentional attack through alteration or destruction of images. Steganography increases the amount of hidden data while sacrificing robustness. These traditional approaches have only a limited amount of embedded data (or payload) for assuring robustness against
Figure 2. Pixel-sampling positions for computing the LBP (local binary pattern) code. This schematically shows the embedding of one byte of data for each block (P = 8).
(1)
ColorCode, and (3) designQR. These matrix-formed patterns lack flexibility in designing.
noise caused by lights, lenses, and imaging devices. Also, increasing such robustness requires a large modulation of the original image’s color or frequency component, which often seriously compromises the resulting image’s quality.
Reference 1. N. Provos and P. Honeyman, “Hide and Seek: An Introduction to Steganography,” IEEE Security & Privacy, vol. 1, no. 3, 2003, pp. 32–44.
Screening LBP Colors
g5
g7
gc
g0 R
g3
g1 g2
on a stationary random field. Each block’s color distribution is independent of that block’s location; this property is our ground for introducing a global threshold. Using the centered dots at every block to compute g m seems redundant, but the local comparison of the coded components in the block can enhance data detection’s robustness. The captured image’s color intensity often varies depending on the location owing to the change of illumination, and each locally estimated color at the center serves as the calibrated value of g m for each block. 76
November/December 2009
(3)
Figure. A. Example 2D bar codes: (1) QR (Quick Response) Code, (2)
g6
g4
(2)
To increase robustness against the positional error caused by image distortions, each dot for embedding data requires an area greater than the size of a single pixel. So, we paint the same color on the nearby pixels (in our case, the eight linked neighbors). Figure 3a shows an exemplar; Figure 3b represents examples of the LBP code for 5 × 5 blocks made from that exemplar. An exemplar’s classified colors, however, become conspicuous on a synthesized image if their frequency band is higher than that of the LBP dots whose area contains 3 × 3 pixels. So, we remove the exemplar’s high-frequency color components from the upper or lower class. In practice, we remove the pixel colors if their outputs through a differential filter (we currently use a Sobel filter) exceed a threshold. We’ve carefully determined this threshold so as to ensure a sufficient number of colors for each class. We discuss this screening in more detail later.
Texture Synthesis with LBP To conceal the initially painted LBP code, our method adaptively determines unpainted pixels’ colors by referencing the exemplar’s features. For each unpainted pixel located at p, we compute the sum S of differences D in color between the nearby pixels of offset r and the corresponding pixels at q in the exemplar:
S ( p, q) =
∑ D ( p, q, r) r∈n
0 D ( p, q, r) = 2 c ( p + r) − c (q + r)
if c ( p + r) is null
else ,
(2)
where p denotes the unpainted pixel’s 2D position, n is the set of offset vectors for indicating N = (2w + 1)2 - 1 neighbors centered at p; n := {(s, t)|s, t ∈ {-w, …, -1, 0, 1, …, w}, (s, t) ≠ (0, 0)}, and w denotes the range of neighbors. S(p, q) decreases as the similarity of the synthesized texture and exemplar increases in the color distribution pattern for the nearby pixels of the p and q locations. We then substitute c(p) for the color at q of the exemplar, where S(p, q) takes the minimum value of
qˆ = arg min S ( p, q) , c ( p) = c (qˆ) .
(3)
q
Equations 2 and 3 are basically template matching that regards a square context window of N pixels centering p as a template and searches the best matching region in the exemplar. Larger N can generate images more similar to the exemplar, but the LBP code becomes more conspicuous as a side effect. We experimentally found that w = 7 (N = 224) ensures good balance of these factors. This procedure repeats until we’ve painted all the pixels. Figure 3c shows the texture generated by using this coating algorithm with the exemplar of Figure 3a and the LBP code in Figure 3b.
The Effects of Pixel-Painting Sequence In pixel-based texture synthesis, the sequence of painting pixels plays an important role in the synthesized images’ quality. (For more on pixel-
(a)
(a)
(b)
(c)
based texture synthesis, see the “Related Work in Texture Synthesis” sidebar.) Existing methods usually select unpainted pixels in scan-line sequence for referencing the color coherency of previously painted pixels as much as possible. In our method, however, the scan-line sequence often fails to conceal the LBP code. The colors near the LBP code reveal discontinuity. This is partly because the top pixels used for pattern matching, marked A in Figure 4a, have been coated in former iterations without the effect of the LBP code marked X. In addition, the LBP hardly even affects the nearer pixels marked B because of the smaller area in the context window. (Figure 4a schematically represents a window of w = 2 for simplicity, but we actually use w = 7.) So, the color inconsistency near the LBP’s boundary is unavoidable, which reveals our peculiar problem with using the LBP code as a hard constraint in generating textures. So, we introduce randomness to pixel-painting sequences (see Figure 4b) to blur an LBP dot’s borders. This random sequence can make the number of pattern-matched pixels uneven; the pixels near the LBP code might be heavily influenced by the LBP when they’re occasionally visited in an early stage of the coating process. (Li-Yi Wei and Marc Levoy introduced a similar policy for generating textures over surfaces of arbitrary topology.3) This approach, however, seriously degrades the
27
56
6
11
16
48
22
40
55
19
38
30
33
15
42
51
Figure 3. Datacoded texture synthesis: (a) an exemplar (sample image), (b) LBP code for 25-byte data, and (c) the synthesized texture. The synthesis successfully camouflages the dots of LBP code.
A
A
A
A
A
5
10
54
50
43
24
29
17
B
B
B
B
B
32
41
X
X
X
46
39
20
X
X
X
B
B
26
53
X
X
X
36
35
13
X
X
X
X
X
X
X
X
X
52
45
X
X
X
31
21
23
X
X
X
9
3
44
28
37
47
2
8
X
X
X
49
7
18
1
14
25
4
34
(b)
(c)
Figure 4. A schematic representation of pixel-painting sequences: (a) scan-line, (b) random, and (c) Hilbert curve. Pixels painted for the LBP code are marked X; arrows and numbers represent the sequence of painting pixels.
IEEE Computer Graphics and Applications
77
Feature Article
Related Work in Texture Synthesis
T
exture image synthesis using the example-based approach falls roughly into two types. Pixel-based synthesis determines every pixel’s color through a pattern-matching process using a square sample image as a template. Patchbased synthesis adaptively arranges the sample image, called a patch, while ensuring seamless, continuous connection of patches. (A detailed explanation of these techniques appears elsewhere.1) In the main article, we focus on pixel-based methods,2–5 which usually paint a texture image from scratch. Our method, however, starts painting from an initial dotted pattern generated for coding data. This approach is similar to using texture generation based on initially painted images4 or flow fields5 to transform images. Such techniques try to preserve the initial image’s visual features while allowing a slight change in color with soft constraints. Our method, on the other hand, preserves the initial image’s color with hard constraints while making the visual features inconspicuous. It essentially uses a greedy method to effectively conceal the initial pattern; more-sophisticated optimizations could improve the resulting image’s quality. Existing
optimization,5 however, often decreases color contrast owing to its intrinsic property, especially for our hard local binary pattern (LBP) constraints, and makes the boundaries of LBP dots conspicuous as a side effect of the excessive blur.
References 1. F. Dong and X. Ye, “Multiscaled Texture Synthesis Using Multisized Pixel Neighborhoods,” IEEE Computer Graphics and Applications, vol. 27, no. 3, 2007, pp. 41–47. 2. L.-Y. Wei and M. Levoy, “Fast Texture Synthesis Using TreeStructured Vector Quantization,” Proc. Siggraph, ACM Press, 2000, pp. 479–488. 3. A.A. Efros and T.K. Leung, “Texture Synthesis by Nonparametric Sampling,” IEEE Int’l Conf. Computer Vision, IEEE CS Press, 1999, pp. 1033–1038. 4. M. Ashikhmin, “Synthesizing Natural Textures,” Proc. 2001 Symp. Interactive 3D Graphics, ACM Press, 2001, pp. 217–226. 5. V. Kwatra et al., “Texture Optimization for Example-Based Synthesis,” ACM Trans. Graphics, vol. 24, no. 3, 2005, pp. 795–802.
Figure 5. The effects of pixel-painting sequences: (a) exemplars of a random (top) and regular (bottom) pattern, (b) texture images synthesized using a random sequence, (c) texture images synthesized using a scanline sequence, and (d) texture images synthesized using a Hilbert curve sequence. The images in (d) take over the properties of the images in (b) and (c).
resulting image’s quality when the exemplar has structured patterns. Existing methods using sophisticated techniques such as iterative optimizations or scalable pattern matching4,5 can accurately reproduce structured patterns in an exemplar. This capability, however, conflicts with the hard LBP constraints, so we must find a compromise. Our solution is to introduce a Hilbert curve sequence (see Figure 4c), which allows uniform distribution of the sequence while preserving the
78
November/December 2009
(a)
(b)
(c)
(d)
space coherency in painting order. This solution is a reasonable compromise because the space coherency ensures the capability of reproducing structural patterns in the simplest way, similarly to the scan-line sequence. Also, its randomness of pixeltraversing directions can simultaneously produce the blurring effect somewhat. Figure 5 shows the effects of pixel-painting sequences. Figure 5a shows the exemplars for a random and a regular pattern. A random sequence
(see Figure 5b) can conceal the LBP code better than a scan-line sequence (see Figure 5c) because it can blur the LBP’s discontinuous borders. However, it loses the distinctness of the pattern silhouette inherent in the exemplar because of its lack of reproducing capability. On the other hand, the Hilbert curve sequence (see Figure 5d) can both blur conspicuous spots and preserve silhouettes because it can irregularly paint pixels in series. This property is especially effective for the regular iterative pattern, as the bottom row shows.
Quality Improvement with Recoating The coating process determines the most similar pattern only with the painted pixels, owing to the rule of the dissimilarity computation in Equation 2; D(p, q, r) = 0 if c(p + r) is null. However, this causes inaccurate pattern analysis in the early stage of coating when we introduce randomness in pixel-painting sequences. This property is useful in blurring the LBP’s borders but damages the image quality at regions distant from the LBP. To eliminate this defect, we recoat all pixels except for those on the LBP code. In this second coating phase, all pixel colors have been tentatively determined, so the texture pattern analysis becomes more accurate. As a result, we can improve the synthesized images’ quality. We experimentally confirmed that recoating more than once won’t particularly increase image quality, so we recoat only once. Because the pixel-painting sequence’s effect is negligible in the recoating phase, we use a scan-line sequence. Figure 6 demonstrates improved image quality through recoating. As these examples show, recoating enhances the painting sequences’ advantages. The random sequence makes the spots of LBP code more inconspicuous, and the Hilbert curve sequence recovers the silhouette more clearly and correctly.
(a)
(b)
(c)
(a)
(b)
The Effects of LBP and Randomness We synthesize the texture image so that it has the exemplar’s features, but the frequency components’ features are affected largely by the LBP code’s density. Figure 7 demonstrates the effect of the amount of the coded data. Figure 7a is the exemplar. Figures 7b and 7c denote the LBP for 25- and 100-byte data, respectively, from that exemplar. Figures 7d and 7e show the synthesized textures from the LBP in Figures 7b and 7c, respectively, revealing the change in the frequency pattern. Figure 8 shows the effects of screening LBP colors. By excluding tiny dark colors in the exemplar (see Figure 8a), our color screening removed the conspicuous spots in Figure 8b to create the image in Figure 8c. An exemplar’s randomness plays an important role. Figure 9a shows an exemplar containing a small amount of randomness. Figure 9b shows how the existing method1 based on the scan-line sequence reproduces that exemplar’s structural feature. Our Hilbert curve sequence partially destroys the structure as a side effect of incorporating
(d)
Figure 6. The effects of recoating for images in Figure 5. The magnified images in (a) show intermediate defective patterns before recoating; those in (b) show improved images after recoating. The top images use a random sequence; the bottom ones use a Hilbert curve sequence. These images show the quality improvement with the recoating mechanism.
(e)
Figure 7. Texture synthesis for various densities of LBP code: (a) the exemplar, (b) the LBP for 25 bytes of embedded data, (c) the LBP for 100 bytes, (d) the texture for 25 bytes, and (e) the texture for 100 bytes. The LBP code’s density affects the texture images’ dominant frequencies.
IEEE Computer Graphics and Applications
79
Feature Article
(a) Figure 8. The effect of screening LBP colors: (a) the exemplar, (b) the texture image synthesized by randomly selecting the colors among the corresponding class, and (c) the texture image synthesized using our color-screening mechanism, which removed the conspicuous spots.
(b)
(c)
randomness in the pixel-visiting sequence (see Figure 9c). In addition, its data-embedding synthesis clearly fails to conceal the LBP code (see Figure 9d). All syntheses used a larger context window (w = 15) to enhance structure reproduction. These examples show that our encoding scheme is suitable only for random texture images and suggest the difficulty of camouflaging LBP codes with general images.
Experimental Results We implemented our data-detecting mechanism on a FOMA (Freedom of Mobile Multimedia Access) D904i mobile phone with the DoJa API. Using an Epson PX-G5100 color inkjet printer, we printed 200 × 200 pixel data-coded texture images on a 2 × 2 inch square on superfine A4 paper. We then captured the printed image with the phone’s camera (supporting it by hand), using a macro mode of 480 × 640 pixels at a distance of 5 to 8 cm, and using a fluorescent lamp for lighting. Through 10 trials for each of four examples, we had an average of 0.29 error bits, where the maximum was 6 and the minimum was 0. Figure 10 shows snapshots of the data detection process. To view a movie of this process, visit www. val.ics.tut.ac.jp/otori/forCGAdemo.wmv. As a data detection preprocess, we’ve implemented an image calibration mechanism that automatically detects four corners of a square region and corrects distortions through projective transformations of the four corners.6 We then extract
(a)
(b)
(c)
coded components of all pixels and compute their upper and lower classes for dividing up each coded component of the LBP dots. We divide the calibrated image into a predefined number of blocks and compute the dots’ positions within each block, at which their coded components are determined. Table 1 shows the detailed results of data detection for the images in Figures 3c, 6b (top), 7d, and 7e. The error rate represents the ratio of imperfect code detections; we computed the average and maximum error bits by using the number of wrong bits for the 10 trials. Our method couldn’t completely detect the embedded data, especially for Figure 3c. However, the ratios of error bits are small enough to recover the information with some error-correcting codes such as Reed-Solomon.7 The amount of embedded data (the payload) has little effect on detection accuracy. Table 2 compares our method’s payload to those of three commercial products. The comparison is rough owing to the lack of detailed specification of the products. Also, we can’t accurately estimate these payloads because of the difference in measurement conditions in terms of paper size, camera resolution, lighting, and so on. Although this evaluation lacks accuracy, our method can embed a larger amount of data than existing techniques based on natural-image modulations. So, we believe it’s sufficient for embedding most types of URLs containing anchor information for Web services. On the other hand, our payload is smaller than that of the 2D bar code, and our current capacity isn’t suited for long embedded messages.
W
e’re looking at several ways to improve our method. To reduce the LBP dots’ conspicuousness, we could relax the restriction on color constancy. This relaxation should ensure robust
(d)
Figure 9. Our method’s limitations. (a) This exemplar contains a small amount of randomness. (b) The scanline sequence synthesizes the exemplar’s structural features. (c) However, the Hilbert curve sequence partially destroys the structural features owing to its randomness in pixel-visiting directions. (d) Camouflaging the LBP code for an exemplar with a small amount of randomness is difficult. 80
November/December 2009
code detection and might require integrating optimizations for painting the LBP and textures. Also, extracting the data-embedded square region currently requires drawing some marks or lines on a texture image. However, in some cases, markings should be concealed; we’re working on a method for this. Another challenge will be to extend our target from 2D flat images to deformed images texture-mapped onto a virtual 3D object.
Acknowledgments This work was supported partly by the Global Centers of Excellence Program: Frontiers of Intelligent Sensing, of the Japanese Ministry of Education, Culture, Sports, Science, and Technology.
References 1. L.-Y. Wei and M. Levoy, “Fast Texture Synthesis Using Tree-Structured Vector Quantization,” Proc. Siggraph, ACM Press, 2000, pp. 479–488. 2. T. Mäenpää and M. Pietikäinen, “Texture Analysis with Local Binary Patterns,” Handbook of Pattern Recognition and Computer Vision, 3rd ed., World Scientific, 2005, pp. 197–216. 3. L.-Y. Wei and M. Levoy, “Texture Synthesis over Arbitrary Manifold Surfaces,” Proc. Siggraph, 2001, pp. 355–360. 4. F. Dong and X. Ye, “Multiscaled Texture Synthesis Using Multisized Pixel Neighborhoods,” IEEE Computer Graphics and Applications, vol. 27, no. 3, 2007, pp. 41–47. 5. V. Kwatra et al., “Texture Optimization for ExampleBased Synthesis,” ACM Trans. Graphics, vol. 24, no. 3, 2005, pp. 795–802. 6. A. Katayama et al., “New High-Speed Frame Detection Method: Side Trace Algorithm (STA) for i-appli on Cellular Phones to Detect Watermarks,” Proc. 3rd Int’l Conf. Mobile and Ubiquitous Multimedia, ACM Press, 2004, pp. 109–116. 7. I.S. Reed and G. Solomon, “Polynomial Codes over Certain Finite Fields,” SIAM J., vol. 8, no. 2, 1960, pp. 300–304. 8. T. Nakamura et al., “Improved Digital Watermark Robustness against Translation and/or Cropping of an Image Area,” IEICE Trans. Fundamentals, vol. E83-A, no. 1, 2000, pp. 68–76. 9. T. Noda, J. Moroo, and H. Chiba, “Print-Type Steganography Technology,” Fujitsu, vol. 57, no. 3, 2006, pp. 320–324 (in Japanese).
Hirofumi Otori is a PhD student in the Toyohashi University of Technology’s department of information and computer sciences and a research assistant for the
(a)
(b)
Figure 10. URL detection with a commercial mobile phone: (a) capturing the image and (b) detecting the URL. The detection costs only a few seconds.
Table 1. Error rates for various sizes of embedded data. Payload (bytes)
No. of error bits Error rate
Average
Maximum
Fig. 3c
25
0.5
1.7
6
Fig. 6b (top)
25
0.1
0.1
6
Fig. 7d
25
0.2
0.2
1
Fig. 7e
100
0.1
0.1
1
Example
Table 2. Comparing the payloads of our method and commercial products. Product
Payload (bytes)
Our method
~100
QR Code
~3,000
Pasha-warp8 FP (Fine Picture) Code
9
Encoding scheme LBP (local binary pattern) code 2D bar code
3
Frequency modulation
5
Color modulation
Global Centers of Excellence Program of the Japanese Ministry of Education, Culture, Sports, Science, and Technology. His research interests include data-coding techniques with computer-generated images and applications with mobile graphics in general. Otori has a master’s in engineering from the Toyohashi University of Technology. Contact him at [email protected]. Shigeru Kuriyama is a professor at the Toyohashi University of Technology’s department of information and computer sciences and an invited leader of the visualization team at the Digital Human Research Center at Japan’s Advanced Institute of Science and Technology. His research interests include mobile graphics technology and humanoid animations. Kuriyama has a Doctor of Engineering degree from Osaka University. Contact him at [email protected]. IEEE Computer Graphics and Applications
81
Texture Synthesis for Mobile Data Communications Hirofumi Otori and Shigeru Kuriyama ■ Toyohashi University of Technology
M
ost mobile phones contain a high-quality digital camera, which can serve as a handy image code detector. Two-dimensional bar codes are frequently used to code URLs for relevant information services on the Web. The 2D bar code’s black-and-white matrix-formed patterns are efficient for robustly coding data, but their meaningless images often diminish the attractiveness of printed matter such as magazines and wrapping paper. We’ve found you can overcome this by introducing recent computer graphics technologies to the data-coding mechanism. (For more on bar codes and alternatives, see the sidebar “Related Work in Data This approach to image Coding and Hiding.”) coding first paints a regularly We propose using texture imarranged dotted pattern, ages of iterative patterns as a using colors picked from type of image code. We directly a texture sample with paint colored dotted patterns to features corresponding to be robustly detected from a phothe embedded data. It then tographed image. Then, we syncamouflages the dotted thesize texture images to conceal pattern using the same texture the patterns while preserving sample while preserving their colors. This method can quality comparable to that of embed more information than existing technologies, and its alexisting synthesis techniques. gorithm for texture image generation ensures the synthesized images’ quality. We’ve demonstrated our method’s feasibility by implementing it on a mobile phone. The results show that our robust, efficient data detection mechanism can read coded data within a reasonable response time and with a reasonable error ratio. Figure 1 shows sample seed images (Figure 1a), textures synthesized with an existing pixel-based method (Figure 1b),1 and 25-byte data-coded textures with our method (Figure 1c). Although the resulting images differ slightly, their quality is nearly the same. 74
November/December 2009
Why Use Texture Images? Using texture images to replace bar codes has two benefits. Iterative textures sometimes serve as background images in printed material, and you can embed information on some blank places unoccupied by foreground images. Another promising application is the use of a camouflage code on real material. By synthesizing a texture with a sample image scanned from a real material such as wood or cloth, you can embed data by inconspicuously affixing a seal of the printed texture on the material. This camouflage code can record product information for the material while avoiding unsightly coded images.
Data Coding Using Local Binary Patterns Our data-coding strategy first converts bit patterns of information into colored dotted patterns. We sparsely arrange colored dots and map binary values to those dots according to the magnitude of a specific color component, which we call a coded component. (In contrast, existing bar codes use densely arranged black and white dots.) Although our texture synthesis allows any type of regularly arranged pattern, we experimentally found that the existing feature vector for texture images, called the local binary pattern (LBP),2 is best suited to synthesize high-quality images with our method for many types of textures. We compute this code by comparing a centered pixel’s value against those of circularly arranged nearby pixels. We divide the coded data by P bits and compute the LBP from the difference of a coded component at a pixel at the center pc of the block and at the P circular positions regularly arranged as p n = p c + R(cos(2pn/P), -sin(2pn/P)), n = 0, ..., P - 1, with the distance R from the center (see Figure 2):
Published by the IEEE Computer Society
0272-1716/09/$26.00 © 2009 IEEE
LBPP .R ( pc ) =
P −1
∑ s ( g ( p ) − g ( p ))2 , n
c
n
n=0
1 : x ≥ 0 s ( x) = , 0 : x < 0
(1)
where g(p) denotes the coded component at p, and s(g(pn) - g(pc)) corresponds to the value of the (n + 1)th bit. We round off the coordinates of the circular positions pn to integers so that we can fit them to discrete pixel positions. We divide a region of synthesized texture image into a predefined number of square blocks. We then divide embedding data and code it by painting the LBP onto each block in a scan-line sequence (that is, from left to right, then top to bottom). To conceal this painted LBP code, we add a texture-image coating. Because each LBP can represent P bits, embedding data of n bits requires regularly arranged n/P image blocks, where denotes a ceil function. So, we recover whole data by concatenating the P bits for each block in the same scan-line sequence. We usually set P = 8 so that each block embeds one byte of data (or character). The original LBP uses a grayscale level or each RGB component as g(p), but we can replace it with an arbitrary color component that can be uniquely converted from RGB. We usually select a color component insensitive to the human vision system as the coded component—for example, the Cb component of the Y-Cb-Cr color space. However, the component must have wide distribution for ensuring robustness against color distortions. So, we must select a visually sensitive component if all insensitive components have narrow distributions.
Painting LBP Code We determine every pixel color of a synthesized texture by adaptively selecting a color from the colors in a sample image called an exemplar. We camouflage the image of LBP code by selecting constituent colors from a subset of the same group that’s classified in the next section.
Classifying Colors
Let c ( p) be the color of the pixel at p in an exemplar, and let g ( p) be the corresponding coded component. We first extract the insensitive color component as g ( p) from all pixels and compute the median g m. We then divide every pixel color c ( p) into three classes: ■■ ■■ ■■
the median class for g ( p) ≡ g m , the upper class for g ( p) ≥ g m + T , and the lower class for g ( p) ≤ g m − T ,
(a)
(b)
(c)
where we neglect the coded components in the middle range, g m − T < g ( p) < g m + T , when painting the LBP. The scalar value T isolates the coded components in the upper and lower classes from the median value gm. An increase of T enhances robustness against the noisy variation caused in printing and photographing. However, too much T greatly narrows the range of usable colors. T = 30 ensures good balance for 8-bit coded components. After dividing the exemplar’s constituent colors into three classes, we paint the synthesized texture’s central pixel with the pixel color whose coded component belongs to the median class as c ( pc ) = c ( pm ), where g ( pm ) ≡ g m. To paint the surrounding pixels, we randomly select the pixel color whose coded component belongs to the upper and lower classes for the embedded binary data of 1 and 0, respectively. We assume that texture generation is based IEEE Computer Graphics and Applications
Figure. 1. Examples of (a) sample images, (b) ordinary synthesis, and (c) data-coded synthesis. These two kinds of synthesized images have comparable quality.
75
Feature Article
Related Work in Data Coding and Hiding
D
enso Wave developed the QR (Quick Response) Code (www.qrcode.com) as an efficient, robust 2D bar code (see Figure A1). It has become a popular way to read URLs for navigating Web pages and services, without the troublesome operations of a small numerical keypad. Researchers have developed several methods to introduce an eye-pleasing aesthetic pattern. For example, ColorCode (www.colorzip.com) adds colors (see Figure A2), and designQR (http://d-qr.net) partially modifies the QR code’s patterns to create small icons (see Figure A3). These approaches, however, employ the matrix form and impose rigid constraints on the codes’ visual design. Recently, researchers have employed natural images as replacements for QR Code. Some methods intentionally modulate the image’s specific color or frequency component according to embedded data. Most methods incorporate one of two numerical techniques: watermarking or steganography.1 Watermarking has been used to protect copyright, which requires robustness against intentional attack through alteration or destruction of images. Steganography increases the amount of hidden data while sacrificing robustness. These traditional approaches have only a limited amount of embedded data (or payload) for assuring robustness against
Figure 2. Pixel-sampling positions for computing the LBP (local binary pattern) code. This schematically shows the embedding of one byte of data for each block (P = 8).
(1)
ColorCode, and (3) designQR. These matrix-formed patterns lack flexibility in designing.
noise caused by lights, lenses, and imaging devices. Also, increasing such robustness requires a large modulation of the original image’s color or frequency component, which often seriously compromises the resulting image’s quality.
Reference 1. N. Provos and P. Honeyman, “Hide and Seek: An Introduction to Steganography,” IEEE Security & Privacy, vol. 1, no. 3, 2003, pp. 32–44.
Screening LBP Colors
g5
g7
gc
g0 R
g3
g1 g2
on a stationary random field. Each block’s color distribution is independent of that block’s location; this property is our ground for introducing a global threshold. Using the centered dots at every block to compute g m seems redundant, but the local comparison of the coded components in the block can enhance data detection’s robustness. The captured image’s color intensity often varies depending on the location owing to the change of illumination, and each locally estimated color at the center serves as the calibrated value of g m for each block. 76
November/December 2009
(3)
Figure. A. Example 2D bar codes: (1) QR (Quick Response) Code, (2)
g6
g4
(2)
To increase robustness against the positional error caused by image distortions, each dot for embedding data requires an area greater than the size of a single pixel. So, we paint the same color on the nearby pixels (in our case, the eight linked neighbors). Figure 3a shows an exemplar; Figure 3b represents examples of the LBP code for 5 × 5 blocks made from that exemplar. An exemplar’s classified colors, however, become conspicuous on a synthesized image if their frequency band is higher than that of the LBP dots whose area contains 3 × 3 pixels. So, we remove the exemplar’s high-frequency color components from the upper or lower class. In practice, we remove the pixel colors if their outputs through a differential filter (we currently use a Sobel filter) exceed a threshold. We’ve carefully determined this threshold so as to ensure a sufficient number of colors for each class. We discuss this screening in more detail later.
Texture Synthesis with LBP To conceal the initially painted LBP code, our method adaptively determines unpainted pixels’ colors by referencing the exemplar’s features. For each unpainted pixel located at p, we compute the sum S of differences D in color between the nearby pixels of offset r and the corresponding pixels at q in the exemplar:
S ( p, q) =
∑ D ( p, q, r) r∈n
0 D ( p, q, r) = 2 c ( p + r) − c (q + r)
if c ( p + r) is null
else ,
(2)
where p denotes the unpainted pixel’s 2D position, n is the set of offset vectors for indicating N = (2w + 1)2 - 1 neighbors centered at p; n := {(s, t)|s, t ∈ {-w, …, -1, 0, 1, …, w}, (s, t) ≠ (0, 0)}, and w denotes the range of neighbors. S(p, q) decreases as the similarity of the synthesized texture and exemplar increases in the color distribution pattern for the nearby pixels of the p and q locations. We then substitute c(p) for the color at q of the exemplar, where S(p, q) takes the minimum value of
qˆ = arg min S ( p, q) , c ( p) = c (qˆ) .
(3)
q
Equations 2 and 3 are basically template matching that regards a square context window of N pixels centering p as a template and searches the best matching region in the exemplar. Larger N can generate images more similar to the exemplar, but the LBP code becomes more conspicuous as a side effect. We experimentally found that w = 7 (N = 224) ensures good balance of these factors. This procedure repeats until we’ve painted all the pixels. Figure 3c shows the texture generated by using this coating algorithm with the exemplar of Figure 3a and the LBP code in Figure 3b.
The Effects of Pixel-Painting Sequence In pixel-based texture synthesis, the sequence of painting pixels plays an important role in the synthesized images’ quality. (For more on pixel-
(a)
(a)
(b)
(c)
based texture synthesis, see the “Related Work in Texture Synthesis” sidebar.) Existing methods usually select unpainted pixels in scan-line sequence for referencing the color coherency of previously painted pixels as much as possible. In our method, however, the scan-line sequence often fails to conceal the LBP code. The colors near the LBP code reveal discontinuity. This is partly because the top pixels used for pattern matching, marked A in Figure 4a, have been coated in former iterations without the effect of the LBP code marked X. In addition, the LBP hardly even affects the nearer pixels marked B because of the smaller area in the context window. (Figure 4a schematically represents a window of w = 2 for simplicity, but we actually use w = 7.) So, the color inconsistency near the LBP’s boundary is unavoidable, which reveals our peculiar problem with using the LBP code as a hard constraint in generating textures. So, we introduce randomness to pixel-painting sequences (see Figure 4b) to blur an LBP dot’s borders. This random sequence can make the number of pattern-matched pixels uneven; the pixels near the LBP code might be heavily influenced by the LBP when they’re occasionally visited in an early stage of the coating process. (Li-Yi Wei and Marc Levoy introduced a similar policy for generating textures over surfaces of arbitrary topology.3) This approach, however, seriously degrades the
27
56
6
11
16
48
22
40
55
19
38
30
33
15
42
51
Figure 3. Datacoded texture synthesis: (a) an exemplar (sample image), (b) LBP code for 25-byte data, and (c) the synthesized texture. The synthesis successfully camouflages the dots of LBP code.
A
A
A
A
A
5
10
54
50
43
24
29
17
B
B
B
B
B
32
41
X
X
X
46
39
20
X
X
X
B
B
26
53
X
X
X
36
35
13
X
X
X
X
X
X
X
X
X
52
45
X
X
X
31
21
23
X
X
X
9
3
44
28
37
47
2
8
X
X
X
49
7
18
1
14
25
4
34
(b)
(c)
Figure 4. A schematic representation of pixel-painting sequences: (a) scan-line, (b) random, and (c) Hilbert curve. Pixels painted for the LBP code are marked X; arrows and numbers represent the sequence of painting pixels.
IEEE Computer Graphics and Applications
77
Feature Article
Related Work in Texture Synthesis
T
exture image synthesis using the example-based approach falls roughly into two types. Pixel-based synthesis determines every pixel’s color through a pattern-matching process using a square sample image as a template. Patchbased synthesis adaptively arranges the sample image, called a patch, while ensuring seamless, continuous connection of patches. (A detailed explanation of these techniques appears elsewhere.1) In the main article, we focus on pixel-based methods,2–5 which usually paint a texture image from scratch. Our method, however, starts painting from an initial dotted pattern generated for coding data. This approach is similar to using texture generation based on initially painted images4 or flow fields5 to transform images. Such techniques try to preserve the initial image’s visual features while allowing a slight change in color with soft constraints. Our method, on the other hand, preserves the initial image’s color with hard constraints while making the visual features inconspicuous. It essentially uses a greedy method to effectively conceal the initial pattern; more-sophisticated optimizations could improve the resulting image’s quality. Existing
optimization,5 however, often decreases color contrast owing to its intrinsic property, especially for our hard local binary pattern (LBP) constraints, and makes the boundaries of LBP dots conspicuous as a side effect of the excessive blur.
References 1. F. Dong and X. Ye, “Multiscaled Texture Synthesis Using Multisized Pixel Neighborhoods,” IEEE Computer Graphics and Applications, vol. 27, no. 3, 2007, pp. 41–47. 2. L.-Y. Wei and M. Levoy, “Fast Texture Synthesis Using TreeStructured Vector Quantization,” Proc. Siggraph, ACM Press, 2000, pp. 479–488. 3. A.A. Efros and T.K. Leung, “Texture Synthesis by Nonparametric Sampling,” IEEE Int’l Conf. Computer Vision, IEEE CS Press, 1999, pp. 1033–1038. 4. M. Ashikhmin, “Synthesizing Natural Textures,” Proc. 2001 Symp. Interactive 3D Graphics, ACM Press, 2001, pp. 217–226. 5. V. Kwatra et al., “Texture Optimization for Example-Based Synthesis,” ACM Trans. Graphics, vol. 24, no. 3, 2005, pp. 795–802.
Figure 5. The effects of pixel-painting sequences: (a) exemplars of a random (top) and regular (bottom) pattern, (b) texture images synthesized using a random sequence, (c) texture images synthesized using a scanline sequence, and (d) texture images synthesized using a Hilbert curve sequence. The images in (d) take over the properties of the images in (b) and (c).
resulting image’s quality when the exemplar has structured patterns. Existing methods using sophisticated techniques such as iterative optimizations or scalable pattern matching4,5 can accurately reproduce structured patterns in an exemplar. This capability, however, conflicts with the hard LBP constraints, so we must find a compromise. Our solution is to introduce a Hilbert curve sequence (see Figure 4c), which allows uniform distribution of the sequence while preserving the
78
November/December 2009
(a)
(b)
(c)
(d)
space coherency in painting order. This solution is a reasonable compromise because the space coherency ensures the capability of reproducing structural patterns in the simplest way, similarly to the scan-line sequence. Also, its randomness of pixeltraversing directions can simultaneously produce the blurring effect somewhat. Figure 5 shows the effects of pixel-painting sequences. Figure 5a shows the exemplars for a random and a regular pattern. A random sequence
(see Figure 5b) can conceal the LBP code better than a scan-line sequence (see Figure 5c) because it can blur the LBP’s discontinuous borders. However, it loses the distinctness of the pattern silhouette inherent in the exemplar because of its lack of reproducing capability. On the other hand, the Hilbert curve sequence (see Figure 5d) can both blur conspicuous spots and preserve silhouettes because it can irregularly paint pixels in series. This property is especially effective for the regular iterative pattern, as the bottom row shows.
Quality Improvement with Recoating The coating process determines the most similar pattern only with the painted pixels, owing to the rule of the dissimilarity computation in Equation 2; D(p, q, r) = 0 if c(p + r) is null. However, this causes inaccurate pattern analysis in the early stage of coating when we introduce randomness in pixel-painting sequences. This property is useful in blurring the LBP’s borders but damages the image quality at regions distant from the LBP. To eliminate this defect, we recoat all pixels except for those on the LBP code. In this second coating phase, all pixel colors have been tentatively determined, so the texture pattern analysis becomes more accurate. As a result, we can improve the synthesized images’ quality. We experimentally confirmed that recoating more than once won’t particularly increase image quality, so we recoat only once. Because the pixel-painting sequence’s effect is negligible in the recoating phase, we use a scan-line sequence. Figure 6 demonstrates improved image quality through recoating. As these examples show, recoating enhances the painting sequences’ advantages. The random sequence makes the spots of LBP code more inconspicuous, and the Hilbert curve sequence recovers the silhouette more clearly and correctly.
(a)
(b)
(c)
(a)
(b)
The Effects of LBP and Randomness We synthesize the texture image so that it has the exemplar’s features, but the frequency components’ features are affected largely by the LBP code’s density. Figure 7 demonstrates the effect of the amount of the coded data. Figure 7a is the exemplar. Figures 7b and 7c denote the LBP for 25- and 100-byte data, respectively, from that exemplar. Figures 7d and 7e show the synthesized textures from the LBP in Figures 7b and 7c, respectively, revealing the change in the frequency pattern. Figure 8 shows the effects of screening LBP colors. By excluding tiny dark colors in the exemplar (see Figure 8a), our color screening removed the conspicuous spots in Figure 8b to create the image in Figure 8c. An exemplar’s randomness plays an important role. Figure 9a shows an exemplar containing a small amount of randomness. Figure 9b shows how the existing method1 based on the scan-line sequence reproduces that exemplar’s structural feature. Our Hilbert curve sequence partially destroys the structure as a side effect of incorporating
(d)
Figure 6. The effects of recoating for images in Figure 5. The magnified images in (a) show intermediate defective patterns before recoating; those in (b) show improved images after recoating. The top images use a random sequence; the bottom ones use a Hilbert curve sequence. These images show the quality improvement with the recoating mechanism.
(e)
Figure 7. Texture synthesis for various densities of LBP code: (a) the exemplar, (b) the LBP for 25 bytes of embedded data, (c) the LBP for 100 bytes, (d) the texture for 25 bytes, and (e) the texture for 100 bytes. The LBP code’s density affects the texture images’ dominant frequencies.
IEEE Computer Graphics and Applications
79
Feature Article
(a) Figure 8. The effect of screening LBP colors: (a) the exemplar, (b) the texture image synthesized by randomly selecting the colors among the corresponding class, and (c) the texture image synthesized using our color-screening mechanism, which removed the conspicuous spots.
(b)
(c)
randomness in the pixel-visiting sequence (see Figure 9c). In addition, its data-embedding synthesis clearly fails to conceal the LBP code (see Figure 9d). All syntheses used a larger context window (w = 15) to enhance structure reproduction. These examples show that our encoding scheme is suitable only for random texture images and suggest the difficulty of camouflaging LBP codes with general images.
Experimental Results We implemented our data-detecting mechanism on a FOMA (Freedom of Mobile Multimedia Access) D904i mobile phone with the DoJa API. Using an Epson PX-G5100 color inkjet printer, we printed 200 × 200 pixel data-coded texture images on a 2 × 2 inch square on superfine A4 paper. We then captured the printed image with the phone’s camera (supporting it by hand), using a macro mode of 480 × 640 pixels at a distance of 5 to 8 cm, and using a fluorescent lamp for lighting. Through 10 trials for each of four examples, we had an average of 0.29 error bits, where the maximum was 6 and the minimum was 0. Figure 10 shows snapshots of the data detection process. To view a movie of this process, visit www. val.ics.tut.ac.jp/otori/forCGAdemo.wmv. As a data detection preprocess, we’ve implemented an image calibration mechanism that automatically detects four corners of a square region and corrects distortions through projective transformations of the four corners.6 We then extract
(a)
(b)
(c)
coded components of all pixels and compute their upper and lower classes for dividing up each coded component of the LBP dots. We divide the calibrated image into a predefined number of blocks and compute the dots’ positions within each block, at which their coded components are determined. Table 1 shows the detailed results of data detection for the images in Figures 3c, 6b (top), 7d, and 7e. The error rate represents the ratio of imperfect code detections; we computed the average and maximum error bits by using the number of wrong bits for the 10 trials. Our method couldn’t completely detect the embedded data, especially for Figure 3c. However, the ratios of error bits are small enough to recover the information with some error-correcting codes such as Reed-Solomon.7 The amount of embedded data (the payload) has little effect on detection accuracy. Table 2 compares our method’s payload to those of three commercial products. The comparison is rough owing to the lack of detailed specification of the products. Also, we can’t accurately estimate these payloads because of the difference in measurement conditions in terms of paper size, camera resolution, lighting, and so on. Although this evaluation lacks accuracy, our method can embed a larger amount of data than existing techniques based on natural-image modulations. So, we believe it’s sufficient for embedding most types of URLs containing anchor information for Web services. On the other hand, our payload is smaller than that of the 2D bar code, and our current capacity isn’t suited for long embedded messages.
W
e’re looking at several ways to improve our method. To reduce the LBP dots’ conspicuousness, we could relax the restriction on color constancy. This relaxation should ensure robust
(d)
Figure 9. Our method’s limitations. (a) This exemplar contains a small amount of randomness. (b) The scanline sequence synthesizes the exemplar’s structural features. (c) However, the Hilbert curve sequence partially destroys the structural features owing to its randomness in pixel-visiting directions. (d) Camouflaging the LBP code for an exemplar with a small amount of randomness is difficult. 80
November/December 2009
code detection and might require integrating optimizations for painting the LBP and textures. Also, extracting the data-embedded square region currently requires drawing some marks or lines on a texture image. However, in some cases, markings should be concealed; we’re working on a method for this. Another challenge will be to extend our target from 2D flat images to deformed images texture-mapped onto a virtual 3D object.
Acknowledgments This work was supported partly by the Global Centers of Excellence Program: Frontiers of Intelligent Sensing, of the Japanese Ministry of Education, Culture, Sports, Science, and Technology.
References 1. L.-Y. Wei and M. Levoy, “Fast Texture Synthesis Using Tree-Structured Vector Quantization,” Proc. Siggraph, ACM Press, 2000, pp. 479–488. 2. T. Mäenpää and M. Pietikäinen, “Texture Analysis with Local Binary Patterns,” Handbook of Pattern Recognition and Computer Vision, 3rd ed., World Scientific, 2005, pp. 197–216. 3. L.-Y. Wei and M. Levoy, “Texture Synthesis over Arbitrary Manifold Surfaces,” Proc. Siggraph, 2001, pp. 355–360. 4. F. Dong and X. Ye, “Multiscaled Texture Synthesis Using Multisized Pixel Neighborhoods,” IEEE Computer Graphics and Applications, vol. 27, no. 3, 2007, pp. 41–47. 5. V. Kwatra et al., “Texture Optimization for ExampleBased Synthesis,” ACM Trans. Graphics, vol. 24, no. 3, 2005, pp. 795–802. 6. A. Katayama et al., “New High-Speed Frame Detection Method: Side Trace Algorithm (STA) for i-appli on Cellular Phones to Detect Watermarks,” Proc. 3rd Int’l Conf. Mobile and Ubiquitous Multimedia, ACM Press, 2004, pp. 109–116. 7. I.S. Reed and G. Solomon, “Polynomial Codes over Certain Finite Fields,” SIAM J., vol. 8, no. 2, 1960, pp. 300–304. 8. T. Nakamura et al., “Improved Digital Watermark Robustness against Translation and/or Cropping of an Image Area,” IEICE Trans. Fundamentals, vol. E83-A, no. 1, 2000, pp. 68–76. 9. T. Noda, J. Moroo, and H. Chiba, “Print-Type Steganography Technology,” Fujitsu, vol. 57, no. 3, 2006, pp. 320–324 (in Japanese).
Hirofumi Otori is a PhD student in the Toyohashi University of Technology’s department of information and computer sciences and a research assistant for the
(a)
(b)
Figure 10. URL detection with a commercial mobile phone: (a) capturing the image and (b) detecting the URL. The detection costs only a few seconds.
Table 1. Error rates for various sizes of embedded data. Payload (bytes)
No. of error bits Error rate
Average
Maximum
Fig. 3c
25
0.5
1.7
6
Fig. 6b (top)
25
0.1
0.1
6
Fig. 7d
25
0.2
0.2
1
Fig. 7e
100
0.1
0.1
1
Example
Table 2. Comparing the payloads of our method and commercial products. Product
Payload (bytes)
Our method
~100
QR Code
~3,000
Pasha-warp8 FP (Fine Picture) Code
9
Encoding scheme LBP (local binary pattern) code 2D bar code
3
Frequency modulation
5
Color modulation
Global Centers of Excellence Program of the Japanese Ministry of Education, Culture, Sports, Science, and Technology. His research interests include data-coding techniques with computer-generated images and applications with mobile graphics in general. Otori has a master’s in engineering from the Toyohashi University of Technology. Contact him at [email protected]. Shigeru Kuriyama is a professor at the Toyohashi University of Technology’s department of information and computer sciences and an invited leader of the visualization team at the Digital Human Research Center at Japan’s Advanced Institute of Science and Technology. His research interests include mobile graphics technology and humanoid animations. Kuriyama has a Doctor of Engineering degree from Osaka University. Contact him at [email protected]. IEEE Computer Graphics and Applications
81
