Kang S.B., Quan L. Image-Based Modeling of Plants and Trees

Подождите немного. Документ загружается.

Image-Based Modeling

of Plants and Trees

Synthesis Lectures on

Computer Vision

Editors

Gérard Medioni, University of Southern California

Sven Dickinson, University of Toronto

Image-Based Modeling of Plants and Trees

Sing Bing Kang and Long Quan

2009

any form or by any means—electronic, mechanical, photocopy, recording, or any other except for brief quotations in

printed reviews, without the prior permission of the publisher.

Image-Based Modeling of Plants and Trees

Sing Bing Kang and Long Quan

www.morganclaypool.com

ISBN: 9781608452446 paperback

ISBN: 9781608452453 ebook

DOI 10.2200/S002 ED1V01Y200911COV001

A Publication in the Morgan & Claypool Publishers series

SYNTHESIS LECTURES ON COMPUTER VISION

Lecture #1

Series Editors: Gérard Medioni, Universit y of Southern California

Sven Dickinson, University of Toronto

Series ISSN

Synthesis Lectures on Computer Vision

ISSN pending.

Image-Based Modeling

of Plants and Trees

Sing Bing Kang

Microsoft Research

Long Quan

The Hong Kong University of Science and Technology

SYNTHESIS LECTURES ON COMPUTER VISION #1

cLaypoolMorgan publishers

ABSTRACT

Plants and trees are among the most complex natural objects. Much work has been done attempting

to model them, with varying degrees of success. In this book, we review the various approaches in

computer graphics, which we categorize as rule-based, image-based, and sketch-based methods.

We describe our approaches for modeling plants and trees using images. Image-based ap-

proaches have the distinct advantage that the resulting model inherits the realistic shape and com-

plexity of a real plant or tree. We use different techniques for modeling plants (with relatively large

leaves) and trees (with relatively small leaves). With plants, we model each leaf from images, while

for trees, the leaves are only approximated due to their small size and large number. Both techniques

start with the same initial step of structure from motion on multiple images of the plant or tree that

is to be modeled.

For our plant modeling system, because we need to model the individual leaves, these leaves

need to be segmented out from the images. We designed our plant modeling system to be interactive,

automating the process of shape recovery while relying on the user to provide simple hints on

segmentation. Segmentation is performed in both image and 3D spaces, allowing the user to easily

visualize its effect immediately. Using the segmented image and 3D data, the geometry of each leaf

is then automatically recovered from the multiple views by ﬁtting a deformable leaf model. Our

system also allows the user to easily reconstruct branches in a similar manner.

To model trees, because of the large leaf count, small image footprint, and widespread occlu-

sions, we do not model the leaves exactly as we do for plants. Instead, we populate the tree with leaf

replicas from segmented source images to reconstruct the overall tree shape. In addition, we use the

shape patterns of visible branches to predict those of obscured branches. As a result, we are able to

design our tree modeling system so as to minimize user intervention.

We also handle the special case of modeling a tree from only a single image. Here, the user

is required to draw strokes on the image to indicate the tree crown (so that the leaf region is

approximately known) and to reﬁne the recovery of branches. As before, we concatenate the shape

patterns from a library to generate the 3D shape.

To substantiate the effectiveness of our systems, we show realistic reconstructions of a variety

of plants and trees from images. Finally, we offer our thoughts on improving our systems and on the

remaining challenges associated with plant and tree modeling.

KEYWORDS

tree modeling, plant modeling, image-based modeling

vii

Contents

Figure Credits ...................................................................ix

Introduction ......................................................................1

Review of Plant and Tree Modeling Techniques ......................................5

2.1 Rule-based methods ........................................................5

2.2 Sketch-based methods ......................................................9

2.3 Image-based methods ......................................................11

2.4 Modeling Leaves, Flowers, and Bark ........................................13

2.5 Modeling Environmental Effects............................................15

2.6 Modeling Other Flora .....................................................16

2.7 Appendix: Brief description of L-system .....................................16

Image-Based Technique for Modeling Plants .......................................19

3.1 Overview of Plant Modeling System ........................................19

3.2 Preliminary Processes ......................................................19

3.3 Graph-based Leaf Extraction ...............................................20

3.3.1 Graph partition 21

3.3.2 User interface 23

3.3.3 Graph update 23

3.3.4 Boundary segmentation 24

3.4 Model-based Leaf Reconstruction...........................................24

3.4.1 Extraction of a generic leaf model 24

3.4.2 Leaf reconstruction 24

3.5 Branch Extraction and Reconstruction .......................................26

3.6 Results ...................................................................27

3.7 Discussion ................................................................31

viii CONTENTS

3.8 Summary .................................................................32

Image-Based Technique for Modeling Trees ........................................33

4.1 Overview of the system ....................................................33

4.2 Image capture and 3D point recovery ........................................35

4.3 Branchrecovery...........................................................35

4.3.1 Reconstruction of visible branches 35

4.3.2 Reconstruction of occluded branches 37

4.4 Populating the tree with leaves ..............................................39

4.4.1 Image segmentation and clustering 39

4.4.2 Adding leaves to branches 41

4.5 Results ...................................................................42

4.6 Discussion ................................................................45

4.7 Summary .................................................................45

Single Image Tree Modeling ......................................................49

5.1 Overview of System .......................................................49

5.2 Image Plane Sketching .....................................................49

5.3 Tree Growing .............................................................53

5.3.1 Growth engine 54

5.3.2 Data-driven attractors 56

5.4 Completing the Tree .......................................................57

5.5 Results ...................................................................57

5.6 Summary .................................................................62

Summary and Concluding Remarks ...............................................63

Acknowledgments ...............................................................65

Bibliography ....................................................................67

Authors’ Biographies.............................................................73

Figure Credits

Figure 2.1 is from

P. Prusinkiewicz, M. Hammel, J. Hanan, and R. Mech, “L-systems: from the theory to visual models

of plants,” In M. T. Michalewicz (Ed.), Proceedings of the 2nd CSIRO Symposium on Computa-

tional Challenges in Life Sciences. CSIRO Publishing, 1996. Reprinted by permission.

Figure 2.2 is from

P. de Reffye, C. Edelin, J. Francon, M. Jaeger, and C. Puech, “Plant models faithful to botanical

puting Machinery, Inc. Reprinted by permission.

http://doi.acm.org/10.1145/54852.378505.

Figure 2.3 is from

J. Weber and J. Penn, “Creation and rendering of realistic trees,” ACM SIGGRAPH, pp. 119-127,

http://doi.acm.org/10.1145/218380.218427.

Figure 2.4 is from

B. Lintermann and O. Deussen, “Interactive modeling of plants,” IEEE Computer Graphics and

http://doi.ieeecomputersociety.org/10.1109/38.736469.

Figure 2.5 is from

Makoto Okabe, Shigeru Owada,Takeo Igarashi, “Interactive Design of Botanical Trees Using Free-

hand Sketches and Example-based Editing”,Computer Graphics Forum (Proc.Eurographics 2005),

by permission.

http://dx.doi.org/10.1111/j.1467-8659.2005.00874.x.

Figure 2.6 is from

X. Chen, B. Neubert, Y.-Q. Xu, O. Deussen, and S.B. Kang, “Sketch-based tree modeling using

Markov random ﬁeld,” ACM SIGGRAPH Asia and ACM Transaction on Graphics, 27(5), article 109,

http://doi.acm.org/10.1145/1457515.1409062.

Figure 2.7 is from

Reche-Martinez, I. Martin, and G. Drettakis, “Volumetric reconstruction and interactive render-

Computing Machinery, Inc. Reprinted by permission.

http://doi.acm.org/10.1145/1015706.1015785.

x FIGURE CREDITS

Figure 2.8 is from

B. Neubert, T. Franken, and O. Deussen, “Approximate image-based tree-modelling using particle

Inc. Reprinted by permission.

http://doi.acm.org/10.1145/1275808.1276487.