Feature Article
User-Friendly Graph Editing for Procedural Modeling of Buildings Gustavo Patow ■ Universitat de Girona
G
rammar-based procedural modeling has emerged as a powerful technique for generating architectural geometry.1,2 However, current procedural-modeling systems are mostly text based and impractical for the intended users—that is, artists. Recently, Markus Lipp and his colleagues introduced a visual-editing approach that provides direct, fine-grained local control of all aspects of the grammar for buildings,3 but the underlying paradigm didn’t change. (See the sidebar for more on related A proposed rule-based visualresearch.) That is, users still editing metaphor intuitively must generate rules and apply lets artists create buildings them to sets of shapes to create without changing their product shapes. In fact, Lipp and workflow. It’s based on the his colleagues identified one of realization that the rule base the most important shortcomrepresents a directed acyclic ings that stems directly from the graph and on a shift in the rule-based nature of shape grammars: the dependency between development paradigm from rules isn’t clearly visible, making product-based to rule-based it hard to exchange and underrepresentations. stand rules created by different artists. One reason for this comes from the disassociation between the rules used and the generated product tree. I propose to change the paradigm for editing shape grammars. This change is based on two concepts: ■■
■■
66
the observation that the rule base actually represents a directed acyclic graph (DAG), in which nodes represent operations and wires represent the geometry stream, and a shift in the development paradigm from a product-based representation to a user-friendly visual-operation-based one.
March/April 2012
This change will let users create rule bases from scratch without having to edit text files; it also bridges the disassociation between rules, their interrelationships, and their applications. Users can also employ a visual programming language for interactive graphics, adding and editing rules, interactively connecting them, and controlling the creation process workflow.4,5 This new paradigm integrates seamlessly into current contentcreation pipelines, which expands the artist’s toolbox (for example, by adding Boolean operations) and helps with modeling complex buildings (see Figure 1). This new metaphor opens the door to new possibilities, ranging from model verification to full model editing through graph rewriting. For instance, simple graph traversals let users use the automatic derivation of a rule’s text file version in any traditional application such as CityEngine.2
Shape Grammars Here, I loosely follow the notation Pascal Müller and his colleagues used for their shape grammar,2 although the concepts also work for other grammars. The shape grammar starts with an initial axiom shape (for example, a building outline), and rules are iteratively applied, replacing selected shapes with others. A rule has a labeled shape on the left, called a predecessor, and one or more shapes and commands on the right, called successors. Commands are macros that create shapes or commands. An example rule could be predecessor → CommandA; CommandB. Müller and his colleagues introduced four commands:2
Published by the IEEE Computer Society
0272-1716/12/$31.00 © 2012 IEEE
Related Work in Procedural Modeling and Visual Editing
P
eter Wonka and his colleagues produced promising results in the procedural modeling of buildings by using grammar-based procedural techniques.1 Pascal Müller and his colleagues improved that approach. 2 However, these approaches allowed only indirect control over the final model by changing a text-based representation of the underlying grammar and provided interactive control only by changing some constructive parameters on the final model. Later, Marcus Lipp and his colleagues introduced realtime interactive visual editing for shape grammars, allowing the creation of rule bases from scratch without text-file editing.3 Although they realized that artists had difficulty seeing a rule’s effects, they still relied on separate workspaces for rule editing and rule execution. My approach complements theirs in that it lets artists naturally visualize the flow between rules, making their dependency clearly visible and understandable. Pascal Müller and his colleagues4 and Panagiotis Koutsourakis and his colleagues5 presented methods to automatically recover shape grammars from photographs of buildings by combining the grammars with image-based analysis. Daniel Aliaga and his colleagues presented style grammars for quick visualization of buildings and structures.6 They proposed a system that automatically derives a grammar from existing buildings. My graph-rewriting system can be considered as an important generalization of these derivations, which allows for a much more general kind of operation. Finally, my graph-based method can be naturally integrated with a visual programming language for interactive graphics, such as Conman.7 These languages have become increasingly popular in computer graphics and have been applied in commercial tools such as Autodesk’s Maya Hypergraph shader editor8 and even full procedural modelers such as SideFX’s Houdini (www.sidefx.com). These tools use an intuitive (and familiar) node-based visual streaming system that’s fully oriented to operations. That is, artists control the operations they perform on the data, which is equivalent to controlling the data itself. In this context (vi-
■■
■■ ■■
■■
Split splits the current shape into multiple shapes. Repeat splits one shape multiple times. Comp (component split) creates shapes (for example, faces or edges) from the current shape’s components. Insert employs premade assets to replace a current predecessor.
Traditionally, the application of a rule generates a hierarchy of shapes corresponding to a particular instance the grammar creates by inserting rule successor shapes as children of the rule’s predeces
sual procedural-modeling languages), the visual language that Bernd Lintermann and Oliver Deussen developed for branching structures such as flowers, bushes, and trees is the most similar to mine to date.9 Commercial tools such as Procedural’s CityEngine (www.procedural.com) and Epic’s UDK (www.udk.com) have incorporated modules with a visual-editing graph metaphor, confirming the visual approach’s advantages. For a thorough review of general visual programming languages, see “Advances in Dataflow Programming Languages.”10
References 1. P. Wonka et al., “Instant Architecture,” ACM Trans. Graphics, vol. 22, no. 3, 2003, pp. 669–677. 2. P. Müller et al., “Procedural Modeling of Buildings,” ACM Trans. Graphics, vol. 25, no. 3, 2006, pp. 614–623. 3. M. Lipp, P. Wonka, and M. Wimmer, “Interactive Visual Editing of Grammars for Procedural Architecture,” ACM Trans. Graphics, vol. 27, no. 3, 2008, article 102. 4. P. Müller et al., “Image-Based Procedural Modeling of Facades,” ACM Trans. Graphics, vol. 26, no. 3, 2007, article 85. 5. P. Koutsourakis et al., “Single View Reconstruction Using Shape Grammars for Urban Environments,” Proc. 2009 IEEE 12th Int’l Conf. Computer Vision, IEEE CS, 2009, pp. 1795–1802. 6. D.G. Aliaga, P.A. Rosen, and D.R. Bekins, “Style Grammars for Interactive Visualization of Architecture,” IEEE Trans. Visualization and Computer Graphics, vol. 13, no. 4, 2007, pp. 786–797. 7. P.E. Haeberli, “Conman: A Visual Programming Language for Interactive Graphics,” Siggraph Computer Graphics Q., vol. 22, no. 4, 1988, pp. 103–111. 8. M.-A. Guindon, Learning Maya: Foundation, Alias Learning Tools, 2005. 9. B. Lintermann and O. Deussen, “Interactive Modeling of Plants,” IEEE Computer Graphics and Applications, vol. 19, no. 1, 1999, pp. 56–65. 10. W.M. Johnston, J.R.P. Hanna, and R.J. Millar, “Advances in Dataflow Programming Languages,” ACM Computing Surveys, vol. 36, no. 1, 2004, pp. 1–34.
sor shape.2,3 This process executes until only terminal shapes remain (see Figure 2a). This rule base automatically generates the instance and associated shape hierarchy (see Figure 2b). The figures follow Lipp and his colleagues’ visual notation.3 Shape grammars’ main potential lies in the variations they can produce; each created instance could look different. However, shape grammars’ rule-based nature introduces the shortcomings I mentioned earlier.3
Visual Grammar Editing In my approach, users employ a visual data-flow IEEE Computer Graphics and Applications
67
Feature Article
Figure 1. Users can directly design buildings with a user-friendly graph-based paradigm. From an initial building—the Raccolet House in Moret-sur-Loing (second building from the left)—I easily created the other buildings in this article.
metaphor to construct and modify buildings by creating components (shapes or commands) that are interconnected on the screen. This results in a streaming visual node-editing paradigm for shape grammars, in which each node is an operation applied on its incoming geometry stream. A connection wire routes the output from a node to the input of another downstream node.4,5 This change bridges the gap between the rule base and its application, instantly solving the shortcomings I mentioned. This way, the rules’ interdependence results in a DAG, which can be processed with simple graph-based tools. Figure 2c shows the results of the rules in Figure 2a. This approach lets users see how any node affects the whole model (see Figure 3).
Graph Operations Some algorithms are simpler when used on DAGs instead of general graphs, owing to topological ordering. A topological sort orders vertices in a DAG so that, if a path exists from node u to node v, then v appears after u in the sort. Here, I introduce some operations that apply this characteristic of DAGs to automatically perform complex tasks.
Rule-Set Evaluation By establishing a topological order, the DAG evaluation becomes a typical scheduling problem for a system of tasks (the node executions) with ordering constraints (a node’s dependence on previous 68
March/April 2012
node evaluations). The rule set doesn’t need to execute more than once because the DAG naturally provides the rule order. However, in the prototype, I didn’t need to implement this functionality because the underlying system already provides it. The topological sort frees me from synchronization tasks between nodes because it guarantees that all inputs to a node are evaluated before that node executes. This way, new operators such as Boolean ones don’t need special care.
Traditional Rule-Set File Compatibility Because the graph-based representation incorporates previous text-based approaches, it’s logical to expect that the system can process and generate text-based rule sets from the graph. Reading a text-based rule-set file is simple. For each rule read, I identify the nodes with the predecessor label and instantiate the nodes for each command (and shape) in the successor. I then link their inputs to the corresponding predecessor output. Although, in this case, I don’t need to maintain compatibility with the original shapegrammar proposal,2 only one use of each predecessor label is allowed. So, when reading a file, I also maintain the shape configuration as a list of pairs of active labels and references to the respective nodes. If the system doesn’t need to enforce backward compatibility, a simple graph traversal collecting all nodes with the needed output label will suffice to apply any rule.
To generate a rule-set file, I use topological node ordering to implement a graph traversal that visits each node and generates the respective rule. So, if users want backward compatibility with tools such as CityEngine, they could keep track of all commands that use the same label as a predecessor when they visit each node. Finally, I save all rules for each evaluation-ordered label at once, which guarantees compatibility.
L
L
L
÷
a
·|·
Terminal shape
Shape
a
C
Component split
—
·|·
Split Repeated splits
÷
(a)
Consistency Verification When the user loads rules from a standard representation, or during the subsequent editing, errors might occur that result in backward incompatibilities or even inconsistencies in the model. One of the first things to avoid is cycles in the graph, which are easy to identify algorithmically. To prevent further inconsistencies, users should run both local and global consistency checks. A local check verifies that every node connects correctly to its wired nodes. A connection is incorrect when a node fails to produce the predecessor label used by the node connected to its output. The system also warns users when a node produces a product that isn’t consumed by any other node. However, the system doesn’t indicate an error because this situation can happen when a node is a terminal node used to visualize the final building. Global checking is more demanding. That is, to obey the rules defined by grammar-based approaches, every node that produces a certain product must be connected to every node that has that label as a predecessor, and vice versa. These global verifications often include the results of local ones, but local verifications generally provide much more detailed information.
Initial shape
—
F
Interaction with the DAG New interaction possibilities arise when interacting with a DAG—for example, visually copying and pasting subgraphs and rewiring nodes. These operations result in new buildings that reuse a subset of existing rules, but without the hassle of the complex manual verification that a text- or tree-based rule set would require.
Legend
F
C
L
a
a
F
a
L
a
a
(b)
F
—
C L
L
÷
a
·|·
Graph Layout When users create a building using the DAG-based approach, they usually keep it tidy and well organized for editing purposes, but this doesn’t always happen. Also, if a building’s procedural model is naively or linearly translated to this representation, the resulting graph won’t be that interesting and won’t necessarily be easy to edit. To help users maintain well-ordered graphs, the
(c) Figure 2. Applying rules. (a) A visual representation of a simple rule base that selects two facades (L and F) from the input mass model, subdivides them into floors (“a” in the rule base), and applies a common window pattern and a door. (b) Applying these rules results in a production workflow, (c) whereas the rules themselves form a graph. I found this disassociation awkward and error prone; writing the rules directly on the graph was much simpler. IEEE Computer Graphics and Applications
69
Feature Article
Figure 3. Users can visualize how a node affects the whole model by selecting it at the visual interface. Here, the user has selected the node that inserts the window asset.
system incorporates graph visualization elements. It uses pydot (http://code.google.com/p/pydot instead), a wrapper around Graphviz (www.graphviz. org), to create hierarchical or layered drawings of directed graphs. The layout algorithm aims edges in the same direction (top to bottom) and then tries to avoid edge crossings and reduce edge length. The system uses the Graphviz dot tool to compute the nodes’ locations and moves them to their respective positions after adapting them to the system coordinates. In my experience, the resulting layout closely follows the one a user might do. I’ve considered using extensions that position graph branches relative to each other according to geometrical relationships (counterclockwise for Comp nodes, bottom-up for Split and Repeat nodes, and so on). However, this hasn’t been necessary in practice, even for the more complex buildings I’ve created.
However, completely new commands should be added to the tools that will execute the rules (for example, as plug-ins). Of course, new commands such as Boolean operations could be hard to accommodate in traditional grammar-based models.
Extensibility
To export to systems where extensibility isn’t direct (for example, as in Müller and his colleagues’ initial implementation2), this rule can be gracefully degraded to the original Repeat commands.
Because I propose a visual programming language for interactive graphics, users can easily generate new nodes or combine existing ones to create commands that smoothly integrate into the current pipeline (see the “Implementation” section). However, maintaining backward compatibility with existing tools will require writing extra code. For instance, if a new node was created as a compound of previously defined commands, it must be broken down into the atomic elements so that software can use it without an extensibility mechanism. 70
March/April 2012
Rule-combination extensibility. You can extend the available functionality simply and seamlessly by combining existing primitives into a new primitive. For instance, you can create the Multisplit(dimX, dimY) command from two chained Repeat commands: pred → Repeat(Y){dimY : pred_prod} pred_prod → Repeat(X){dimX : prod}. Once you’ve defined it, you can use it as pred → Multisplit(dimX, dimY){prod}.
Newly defined operator extensibility. Users can also easily add new commands by using any tool in the artist’s toolbox. For instance, they can seamlessly add simple Boolean operations such as subtraction, union, and intersection to the production workflow. This mechanism introduces some interesting possibilities, as I explain later.
Model Verification My graph-based workflow lets the user exploit graph theory’s full power to perform computations with a building graph. For example, you can perform different kinds of correctness verification on the models you build. Integrity. A model can have holes when a rule is defined using only absolute value sizes. You can easily verify that holes exist by traversing the graph, recursively checking each node for integrity. A node satisfies the verification if the sum of the successor’s product sizes is equal to its predecessor’s size. Basically, the Insert and Split commands can cause holes, but their integrity verification in each direction is trivial. Resizability. You can define a building’s resizability as when a model passes the integrity check and, after a redimensioning transform (for example, a change of scale), no new holes appear. All the command nodes that Müller and his colleagues proposed (except the Split command) are, by their definition, consistent. 2 The Split command can be problematic when it’s defined with all constant subdivision measures. That is, if the Split node receives a geometry that the upstream nodes defined as having a constant size, the input geometry’s size should be equal to or smaller than the sum of the sizes defined in the subdivision node. However, if the input geometry is relatively sized (that is, it can change when the size of the building changes) and the Split command has all constant subdivision sizes, the model simply isn’t resizable. Scalability. We’re interested in scalable buildings that we can use to continuously produce plausible buildings. In this case, the scalability depends mainly on the effect that the different commands produce on the final asset inserts. For instance, if a window asset is in a relatively sized Split node, and nothing prevents this node from being infinitely resized, the window will be stretched if the building is nonuniformly scaled. Verifying this is tricky because all paths from the root node (or nodes) to any Insert node should be tracked and verified in a top-down approach. Basically, there must be a Repeat node in every direction (for example, horizontal and vertical) between any relatively sized Split node and the Insert node. For example, the model in Figure 4b isn’t scalable, as you can see in Figure 4c. Fortunately, the system can easily solve this problem, as I explain later.
(a)
(b)
(c)
(d) Figure 4. The scalability of building models. (a) A photograph of a building. (b) A model using the assets in Figure 1. (c) The model is resizable but not scalable. (d) The model is scalable once the user has applied the graph-rewriting rules to correct the unscalability.
Detecting connected components. The graph’s connected components generally represent the graph’s pieces. Two nodes are in the same component of a graph only if there’s a path that connects them (without considering the paths’ directions). By detecting the connected components, you can easily determine whether any rules or sets of rules are unused, and then notify the user of this.
Graph Rewriting One of the new paradigm’s most important applications is the ability to apply graph rewriting to the graph-based buildings. The idea is to automate the creation of new graphs from the original graph. Formally, a graph-rewriting system consists of a set of graph-rewrite rules of the form p : L → R, IEEE Computer Graphics and Applications
71
Feature Article
with L being the pattern graph and R being the replacement graph. A graph-rewrite rule is applied to the host graph by searching for an occurrence of the pattern graph and replacing that occurrence with an instance of the replacement graph. Developing a full graph-rewriting system is beyond this article’s scope, so I now present a few examples that address the scalability issues
One of the most practical criteria is to use a filter based on the labels the geometry has received during its previous upstream processing.
sume that, starting from the bottom, the first floor is the ground floor and the last one is the roof.1 Then, this operation applies a Repeat node to the intermediate floors, if it wasn’t already applied. If the node has only two splits, repeat the topmost one: c : Split(Y){G…I|T} → Split(Y){G…Repeat{I}|T}, A Repeat node should not follow the Split(Y). The rationale behind this is that buildings tend to have unique structures for their lower floors, but the upper ones tend to be uniform. Ground floor repetition. At the ground floor, you apply a Repeat operation to everything that doesn’t have a “door” label: d : Split(X){door|A} → Split(X){door|Repeat{A}}.
I mentioned earlier. These examples have proven useful for automatically creating buildings using captured data.6,7 In particular, the following rules a and b′ are inspired by Paul Merrell’s research,8 and b and c are inspired by Daniel Aliaga and his colleagues’ research.9
Poor man’s horizontal repetition. For any floor but the first, if you can’t apply rules b or c, repeat the full floor pattern (see the rightmost building’s roof in Figure 1):
Repeated-entry generalization. If a Split node has a tag “A” repeated twice or more, you can transform it into two new operations: a Split and a Repeat node. The new Split node would be identical to the original one, but with the repeated entry replaced by a single subdivision with size equal to the sum of the original ones. The output of the Split node will be routed to the Repeat node, which will have the original size from “A” as the size for the repeated split.
The results. By applying only rules a and c, graph rewriting automatically transformed the nonscalable building in Figure 4c into the scalable building in Figure 4d. I created all the models in Figure 1 by applying user-configurable combinations of these rules to initially nonscalable buildings.
a : Split{…A|A…} → Split{…Repeat(A)…}. In the system, the assets already represent a window and part of its surrounding wall, so I can’t ignore this case, as others do.9 Repeated pairs. This case is similar to the previous one, but for repetitions of AB pairs: b : Split{…A|B|A|B…} → Split{…Repeat{A|B}…}. A simple variant is b′ : Split{…A|B|A…} → Split{…Repeat{A|B}|A…}.8 Upper-floor repetition. You can assume that, starting from the root, the first Split(Y) node you find subdivides the facade into floors. You can also as72
March/April 2012
e : Split(X){…} → Repeat{Split(X){…}}.
Local Control I added direct, persistent local control over the generated instances without breaking the graphbased streaming process.3 For that, I used the extensibility mechanism I described earlier to create an Exception node that splits the geometry into a new stream, using user-defined criteria. One of the most practical criteria is to use a filter based on the labels the geometry has received during its previous upstream processing. Besides the labels used to define the rules, the system attaches to each geometric element labels related to the Split and Repeat nodes it goes through. That is, at each of these commands, the geometry receives a label identifying the command plus the ordering number. This way, if you want all windows in a column, you only need to filter the geometry by identifying the Split(X) command in addition to the column number you’re interested in. Figure 3 shows this situation; Figure 5 shows both an example in which one window in a façade changes and the resulting graph. This mechanism
InitialShape Comp
Subdiv_Facade Repeat_Floors Insert_door Repeat_Windows
Insert_Window
InitialShape Comp
Subdiv_Facade Repeat_Floors Insert_door Repeat_Windows Exception plain_Window Insert_Window
Figure 5. You can easily add local control by using the simple Exception node, which classifies geometry on the basis of a given expression.
differs from the one that Bernd Lintermann and Oliver Deussen presented10 because my method doesn’t imply duplication of the node that’s wired to the exception, nor is it limited to a simple numbering mechanism. This allows the full hierarchical control described in other research.3 In this DAG-based context, you face the same persistence problems that Lipp and his colleagues described when the graph changes through user interaction.3 Fortunately, you can easily adapt their algorithm to the graph-based representation, so you can use their solutions. For example, you can easily implement exact in
stance locators and semantic instance locators as pathfollowing algorithms inside the DAG. To compute sequential orderings, you can implement a two-pass algorithm. The first pass recursively computes the number of shapes for each node, pulling the values from the leaves so that each upstream node knows the number of columns (or floors) for each of its outputs. For Split operations, the number of columns (or floors) is the sum of the numbers for each subdivision. Repeat nodes require special care; they have a variable number of output shapes depending on the input’s size. Keeping track of semantic instance IEEE Computer Graphics and Applications
73
Feature Article
(a)
(b)
(c)
Figure 6. A complex model (the Raccolet House) recreated by the system. (a) Different construction stages (from left to right). (b) The graph for the left wing and the whole model. (c) The textured model. Here, you can see the simplicity that the visual metaphor and the shift in the development paradigm represent for the end user compared to traditional text-based methodologies.
74
locators easily solves this. Any other node simply returns the maximum value among its outputs. The second pass is another recursive traversal that assigns each terminal shape the accumulated value of its position with respect to every upstream node. However, I’ve found that the simplicity and robustness of using an Exception node renders it almost unnecessary to define these locating mechanisms. That’s because it’s much simpler to let users manually reconfigure the node. This remains an open area of research.
Other researchers have defined an oriented bounding box in space called a scope, using it so that any nonterminal symbol in a rule will be created with the current scope.2 Because I use a streaming node-based paradigm, users must take care when tracking the affected geometry’s scope. In this implementation, each geometric element that’s streamed through any node carries its own oriented bounding box, so every subsequent operation can be executed as described in “Procedural Modeling of Buildings.”2
Implementation
Discussion
I developed a fully functional prototype for generating graph-based buildings on top of SideFX Houdini (for more on Houdini, see the sidebar). For that, I used a mixture of Houdini’s own nodes, embedded Python scripts, and external Python methods. I put a selector text box (called a filter) on every node so that users can select a predecessor label from the incoming geometry stream. I implemented labels from Houdini’s grouping mechanism. Every node also provides a labeled output stream. For Comp and Split nodes that can provide geometry with different successor labels, I labeled the resulting streamed geometry by the respective operations and merged it to provide a final, unified output. I used external Python methods to implement graph visitors, which are recursive procedures that implement either a preorder graph traversal or a topological sort-based one. Figure 6 shows a complex model I created with this prototype.
My easy-to-understand visual language for rule applications lets users interactively construct and modify buildings on the fly. Compared to editing a tree’s parameters,3 editing at the graph level is more intuitive because it frees users from browsing the tree for a shape. Also, the local control mechanism I described earlier provides the same controlling capabilities at a higher abstraction level. My approach incorporates previous grammarbased schemes because they basically consist of commands applied to geometric shapes. That is, sequential applications only require linearly chaining nodes, whereas an independent application (using a stack-based notation) simply means wiring more than one output node to a given input one. But not all graphs can be mapped to grammars (for example, a node requiring two different inputs). Users could do some of the operations I described using a traditional text-based approach. However, many of these operations would require treating
March/April 2012
the text-based rules as a graph-data structure so that the users can follow executions from one rule to another. So, the need to traverse the rules as a graph confirms the need for a user-friendly graphediting tool. For instance, a text-based representation could perform the upper-floor repetition I described, but it would need to verify whether a Repeat operation is wired to the Split node. This implies following the rule execution, which is a graph-based operation. This approach has some intrinsic limitations, though. Imagine that a user is scaling parts of a building for which no repeat operation is possible. If the shapes of certain assets are baked into the scaled asset itself (for example, the windows in the roof shape in Figure 4), preserving the shapes of the windows under the roof so that they don’t get stretched is almost impossible. The only solution in this case is to use a rule such as rule e in the “Graph Rewriting” section to make it repeatable. Another limitation of the approach, and of shape grammars in general, is that some architectural styles don’t lend themselves to shape grammars— for example, Antonio Gaudi’s and Frank Gehry’s designs.
I
barely scratched the surface of the many possibilities that graph rewriting introduces for creating procedural models of buildings. One of the most exciting lines for future research is automatic rule-set generation, which would unburden artists who must create whole cities. Instead of creating cities building by building, they could create an adequately rich initial building and then automatically create several variations. In this line of reasoning, it’s sensible to consider using graph-transformation grammars to enumerate all possible graphs from a given graph. (Sometimes “graph grammar” is a synonym for graph-rewriting systems, especially in the context of formal languages.) This way, you could describe a higher-level graph-transformation language, instead of transforming a given graph into a new graph. Also, taking ideas from compiler theory, you could perform common subgraph elimination efficiently (replacing all repetitions by a single instance). This would result in much simpler representations that would be faster to evaluate. Following the same line of reasoning, using automatic parallelization would allow parallel rule-set evaluation, making this approach suitable for modern parallel architectures such as GPUs and multiprocessor-based CPUs.
Acknowledgments I thank Ferran Roure, Remei Ridorsa, Francisco Gonzalez, CG&A’s editors, all members of the Geometry and Graphics Group at Universitat de Girona, my reviewers, and everyone who helped at the Houdini forums. The Raccolet House was modeled by DAZ 3D software (www.daz3d.com). Spain’s Minister of Science and Innovation funded this research under grant TIN2010-20590-C02-02.
References 1. P. Wonka et al., “Instant Architecture,” ACM Trans. Graphics, vol. 22, no. 3, 2003, pp. 669–677. 2. P. Müller et al., “Procedural Modeling of Buildings,” ACM Trans. Graphics, vol. 25, no. 3, 2006, pp. 614–623. 3. M. Lipp, P. Wonka, and M. Wimmer, “Interactive Visual Editing of Grammars for Procedural Architecture,” ACM Trans. Graphics, vol. 27, no. 3, 2008, article 102. 4. P.E. Haeberli, “Conman: A Visual Programming Language for Interactive Graphics,” Siggraph Computer Graphics Q., vol. 22, no. 4, 1988, pp. 103–111. 5. C. Upson et al., “The Application Visualization System: A Computational Environment for Scientific Visualization,” IEEE Computer Graphics and Applica tions, vol. 9, no. 4, 1989, pp. 30–42. 6. P. Müller et al., “Image-Based Procedural Modeling of Facades,” ACM Trans. Graphics, vol. 26, no. 3, 2007, article 85. 7. P. Koutsourakis et al., “Single View Reconstruction Using Shape Grammars for Urban Environments,” Proc. 2009 IEEE 12th Int’l Conf. Computer Vision, IEEE CS, 2009, pp. 1795–1802. 8. P. Merrell, “Example-Based Model Synthesis,” Proc. Symp. Interactive 3D Graphics and Games, ACM, 2007, pp. 105–112. 9. D.G. Aliaga, P.A. Rosen, and D.R. Bekins, “Style Grammars for Interactive Visualization of Archi tecture,” IEEE Trans. Visualization and Computer Graphics, vol. 13, no. 4, 2007, pp. 786–797. 10. B. Lintermann and O. Deussen, “Interactive Modeling of Plants,” IEEE Computer Graphics and Applications, vol. 19, no. 1, 1999, pp. 56–65. Gustavo Patow is an associate professor in Universitat de Girona’s computer science and applied math department. His research interests include modeling, texturing, GPUbased rendering, and procedural modeling. Patow has a PhD in computer science from Universitat Politècnica de Catalunya. Contact him at [email protected]. Selected CS articles and columns are also available for free at http://ComputingNow.computer.org. IEEE Computer Graphics and Applications
75
User-Friendly Graph Editing for Procedural Modeling of Buildings Gustavo Patow ■ Universitat de Girona
G
rammar-based procedural modeling has emerged as a powerful technique for generating architectural geometry.1,2 However, current procedural-modeling systems are mostly text based and impractical for the intended users—that is, artists. Recently, Markus Lipp and his colleagues introduced a visual-editing approach that provides direct, fine-grained local control of all aspects of the grammar for buildings,3 but the underlying paradigm didn’t change. (See the sidebar for more on related A proposed rule-based visualresearch.) That is, users still editing metaphor intuitively must generate rules and apply lets artists create buildings them to sets of shapes to create without changing their product shapes. In fact, Lipp and workflow. It’s based on the his colleagues identified one of realization that the rule base the most important shortcomrepresents a directed acyclic ings that stems directly from the graph and on a shift in the rule-based nature of shape grammars: the dependency between development paradigm from rules isn’t clearly visible, making product-based to rule-based it hard to exchange and underrepresentations. stand rules created by different artists. One reason for this comes from the disassociation between the rules used and the generated product tree. I propose to change the paradigm for editing shape grammars. This change is based on two concepts: ■■
■■
66
the observation that the rule base actually represents a directed acyclic graph (DAG), in which nodes represent operations and wires represent the geometry stream, and a shift in the development paradigm from a product-based representation to a user-friendly visual-operation-based one.
March/April 2012
This change will let users create rule bases from scratch without having to edit text files; it also bridges the disassociation between rules, their interrelationships, and their applications. Users can also employ a visual programming language for interactive graphics, adding and editing rules, interactively connecting them, and controlling the creation process workflow.4,5 This new paradigm integrates seamlessly into current contentcreation pipelines, which expands the artist’s toolbox (for example, by adding Boolean operations) and helps with modeling complex buildings (see Figure 1). This new metaphor opens the door to new possibilities, ranging from model verification to full model editing through graph rewriting. For instance, simple graph traversals let users use the automatic derivation of a rule’s text file version in any traditional application such as CityEngine.2
Shape Grammars Here, I loosely follow the notation Pascal Müller and his colleagues used for their shape grammar,2 although the concepts also work for other grammars. The shape grammar starts with an initial axiom shape (for example, a building outline), and rules are iteratively applied, replacing selected shapes with others. A rule has a labeled shape on the left, called a predecessor, and one or more shapes and commands on the right, called successors. Commands are macros that create shapes or commands. An example rule could be predecessor → CommandA; CommandB. Müller and his colleagues introduced four commands:2
Published by the IEEE Computer Society
0272-1716/12/$31.00 © 2012 IEEE
Related Work in Procedural Modeling and Visual Editing
P
eter Wonka and his colleagues produced promising results in the procedural modeling of buildings by using grammar-based procedural techniques.1 Pascal Müller and his colleagues improved that approach. 2 However, these approaches allowed only indirect control over the final model by changing a text-based representation of the underlying grammar and provided interactive control only by changing some constructive parameters on the final model. Later, Marcus Lipp and his colleagues introduced realtime interactive visual editing for shape grammars, allowing the creation of rule bases from scratch without text-file editing.3 Although they realized that artists had difficulty seeing a rule’s effects, they still relied on separate workspaces for rule editing and rule execution. My approach complements theirs in that it lets artists naturally visualize the flow between rules, making their dependency clearly visible and understandable. Pascal Müller and his colleagues4 and Panagiotis Koutsourakis and his colleagues5 presented methods to automatically recover shape grammars from photographs of buildings by combining the grammars with image-based analysis. Daniel Aliaga and his colleagues presented style grammars for quick visualization of buildings and structures.6 They proposed a system that automatically derives a grammar from existing buildings. My graph-rewriting system can be considered as an important generalization of these derivations, which allows for a much more general kind of operation. Finally, my graph-based method can be naturally integrated with a visual programming language for interactive graphics, such as Conman.7 These languages have become increasingly popular in computer graphics and have been applied in commercial tools such as Autodesk’s Maya Hypergraph shader editor8 and even full procedural modelers such as SideFX’s Houdini (www.sidefx.com). These tools use an intuitive (and familiar) node-based visual streaming system that’s fully oriented to operations. That is, artists control the operations they perform on the data, which is equivalent to controlling the data itself. In this context (vi-
■■
■■ ■■
■■
Split splits the current shape into multiple shapes. Repeat splits one shape multiple times. Comp (component split) creates shapes (for example, faces or edges) from the current shape’s components. Insert employs premade assets to replace a current predecessor.
Traditionally, the application of a rule generates a hierarchy of shapes corresponding to a particular instance the grammar creates by inserting rule successor shapes as children of the rule’s predeces
sual procedural-modeling languages), the visual language that Bernd Lintermann and Oliver Deussen developed for branching structures such as flowers, bushes, and trees is the most similar to mine to date.9 Commercial tools such as Procedural’s CityEngine (www.procedural.com) and Epic’s UDK (www.udk.com) have incorporated modules with a visual-editing graph metaphor, confirming the visual approach’s advantages. For a thorough review of general visual programming languages, see “Advances in Dataflow Programming Languages.”10
References 1. P. Wonka et al., “Instant Architecture,” ACM Trans. Graphics, vol. 22, no. 3, 2003, pp. 669–677. 2. P. Müller et al., “Procedural Modeling of Buildings,” ACM Trans. Graphics, vol. 25, no. 3, 2006, pp. 614–623. 3. M. Lipp, P. Wonka, and M. Wimmer, “Interactive Visual Editing of Grammars for Procedural Architecture,” ACM Trans. Graphics, vol. 27, no. 3, 2008, article 102. 4. P. Müller et al., “Image-Based Procedural Modeling of Facades,” ACM Trans. Graphics, vol. 26, no. 3, 2007, article 85. 5. P. Koutsourakis et al., “Single View Reconstruction Using Shape Grammars for Urban Environments,” Proc. 2009 IEEE 12th Int’l Conf. Computer Vision, IEEE CS, 2009, pp. 1795–1802. 6. D.G. Aliaga, P.A. Rosen, and D.R. Bekins, “Style Grammars for Interactive Visualization of Architecture,” IEEE Trans. Visualization and Computer Graphics, vol. 13, no. 4, 2007, pp. 786–797. 7. P.E. Haeberli, “Conman: A Visual Programming Language for Interactive Graphics,” Siggraph Computer Graphics Q., vol. 22, no. 4, 1988, pp. 103–111. 8. M.-A. Guindon, Learning Maya: Foundation, Alias Learning Tools, 2005. 9. B. Lintermann and O. Deussen, “Interactive Modeling of Plants,” IEEE Computer Graphics and Applications, vol. 19, no. 1, 1999, pp. 56–65. 10. W.M. Johnston, J.R.P. Hanna, and R.J. Millar, “Advances in Dataflow Programming Languages,” ACM Computing Surveys, vol. 36, no. 1, 2004, pp. 1–34.
sor shape.2,3 This process executes until only terminal shapes remain (see Figure 2a). This rule base automatically generates the instance and associated shape hierarchy (see Figure 2b). The figures follow Lipp and his colleagues’ visual notation.3 Shape grammars’ main potential lies in the variations they can produce; each created instance could look different. However, shape grammars’ rule-based nature introduces the shortcomings I mentioned earlier.3
Visual Grammar Editing In my approach, users employ a visual data-flow IEEE Computer Graphics and Applications
67
Feature Article
Figure 1. Users can directly design buildings with a user-friendly graph-based paradigm. From an initial building—the Raccolet House in Moret-sur-Loing (second building from the left)—I easily created the other buildings in this article.
metaphor to construct and modify buildings by creating components (shapes or commands) that are interconnected on the screen. This results in a streaming visual node-editing paradigm for shape grammars, in which each node is an operation applied on its incoming geometry stream. A connection wire routes the output from a node to the input of another downstream node.4,5 This change bridges the gap between the rule base and its application, instantly solving the shortcomings I mentioned. This way, the rules’ interdependence results in a DAG, which can be processed with simple graph-based tools. Figure 2c shows the results of the rules in Figure 2a. This approach lets users see how any node affects the whole model (see Figure 3).
Graph Operations Some algorithms are simpler when used on DAGs instead of general graphs, owing to topological ordering. A topological sort orders vertices in a DAG so that, if a path exists from node u to node v, then v appears after u in the sort. Here, I introduce some operations that apply this characteristic of DAGs to automatically perform complex tasks.
Rule-Set Evaluation By establishing a topological order, the DAG evaluation becomes a typical scheduling problem for a system of tasks (the node executions) with ordering constraints (a node’s dependence on previous 68
March/April 2012
node evaluations). The rule set doesn’t need to execute more than once because the DAG naturally provides the rule order. However, in the prototype, I didn’t need to implement this functionality because the underlying system already provides it. The topological sort frees me from synchronization tasks between nodes because it guarantees that all inputs to a node are evaluated before that node executes. This way, new operators such as Boolean ones don’t need special care.
Traditional Rule-Set File Compatibility Because the graph-based representation incorporates previous text-based approaches, it’s logical to expect that the system can process and generate text-based rule sets from the graph. Reading a text-based rule-set file is simple. For each rule read, I identify the nodes with the predecessor label and instantiate the nodes for each command (and shape) in the successor. I then link their inputs to the corresponding predecessor output. Although, in this case, I don’t need to maintain compatibility with the original shapegrammar proposal,2 only one use of each predecessor label is allowed. So, when reading a file, I also maintain the shape configuration as a list of pairs of active labels and references to the respective nodes. If the system doesn’t need to enforce backward compatibility, a simple graph traversal collecting all nodes with the needed output label will suffice to apply any rule.
To generate a rule-set file, I use topological node ordering to implement a graph traversal that visits each node and generates the respective rule. So, if users want backward compatibility with tools such as CityEngine, they could keep track of all commands that use the same label as a predecessor when they visit each node. Finally, I save all rules for each evaluation-ordered label at once, which guarantees compatibility.
L
L
L
÷
a
·|·
Terminal shape
Shape
a
C
Component split
—
·|·
Split Repeated splits
÷
(a)
Consistency Verification When the user loads rules from a standard representation, or during the subsequent editing, errors might occur that result in backward incompatibilities or even inconsistencies in the model. One of the first things to avoid is cycles in the graph, which are easy to identify algorithmically. To prevent further inconsistencies, users should run both local and global consistency checks. A local check verifies that every node connects correctly to its wired nodes. A connection is incorrect when a node fails to produce the predecessor label used by the node connected to its output. The system also warns users when a node produces a product that isn’t consumed by any other node. However, the system doesn’t indicate an error because this situation can happen when a node is a terminal node used to visualize the final building. Global checking is more demanding. That is, to obey the rules defined by grammar-based approaches, every node that produces a certain product must be connected to every node that has that label as a predecessor, and vice versa. These global verifications often include the results of local ones, but local verifications generally provide much more detailed information.
Initial shape
—
F
Interaction with the DAG New interaction possibilities arise when interacting with a DAG—for example, visually copying and pasting subgraphs and rewiring nodes. These operations result in new buildings that reuse a subset of existing rules, but without the hassle of the complex manual verification that a text- or tree-based rule set would require.
Legend
F
C
L
a
a
F
a
L
a
a
(b)
F
—
C L
L
÷
a
·|·
Graph Layout When users create a building using the DAG-based approach, they usually keep it tidy and well organized for editing purposes, but this doesn’t always happen. Also, if a building’s procedural model is naively or linearly translated to this representation, the resulting graph won’t be that interesting and won’t necessarily be easy to edit. To help users maintain well-ordered graphs, the
(c) Figure 2. Applying rules. (a) A visual representation of a simple rule base that selects two facades (L and F) from the input mass model, subdivides them into floors (“a” in the rule base), and applies a common window pattern and a door. (b) Applying these rules results in a production workflow, (c) whereas the rules themselves form a graph. I found this disassociation awkward and error prone; writing the rules directly on the graph was much simpler. IEEE Computer Graphics and Applications
69
Feature Article
Figure 3. Users can visualize how a node affects the whole model by selecting it at the visual interface. Here, the user has selected the node that inserts the window asset.
system incorporates graph visualization elements. It uses pydot (http://code.google.com/p/pydot instead), a wrapper around Graphviz (www.graphviz. org), to create hierarchical or layered drawings of directed graphs. The layout algorithm aims edges in the same direction (top to bottom) and then tries to avoid edge crossings and reduce edge length. The system uses the Graphviz dot tool to compute the nodes’ locations and moves them to their respective positions after adapting them to the system coordinates. In my experience, the resulting layout closely follows the one a user might do. I’ve considered using extensions that position graph branches relative to each other according to geometrical relationships (counterclockwise for Comp nodes, bottom-up for Split and Repeat nodes, and so on). However, this hasn’t been necessary in practice, even for the more complex buildings I’ve created.
However, completely new commands should be added to the tools that will execute the rules (for example, as plug-ins). Of course, new commands such as Boolean operations could be hard to accommodate in traditional grammar-based models.
Extensibility
To export to systems where extensibility isn’t direct (for example, as in Müller and his colleagues’ initial implementation2), this rule can be gracefully degraded to the original Repeat commands.
Because I propose a visual programming language for interactive graphics, users can easily generate new nodes or combine existing ones to create commands that smoothly integrate into the current pipeline (see the “Implementation” section). However, maintaining backward compatibility with existing tools will require writing extra code. For instance, if a new node was created as a compound of previously defined commands, it must be broken down into the atomic elements so that software can use it without an extensibility mechanism. 70
March/April 2012
Rule-combination extensibility. You can extend the available functionality simply and seamlessly by combining existing primitives into a new primitive. For instance, you can create the Multisplit(dimX, dimY) command from two chained Repeat commands: pred → Repeat(Y){dimY : pred_prod} pred_prod → Repeat(X){dimX : prod}. Once you’ve defined it, you can use it as pred → Multisplit(dimX, dimY){prod}.
Newly defined operator extensibility. Users can also easily add new commands by using any tool in the artist’s toolbox. For instance, they can seamlessly add simple Boolean operations such as subtraction, union, and intersection to the production workflow. This mechanism introduces some interesting possibilities, as I explain later.
Model Verification My graph-based workflow lets the user exploit graph theory’s full power to perform computations with a building graph. For example, you can perform different kinds of correctness verification on the models you build. Integrity. A model can have holes when a rule is defined using only absolute value sizes. You can easily verify that holes exist by traversing the graph, recursively checking each node for integrity. A node satisfies the verification if the sum of the successor’s product sizes is equal to its predecessor’s size. Basically, the Insert and Split commands can cause holes, but their integrity verification in each direction is trivial. Resizability. You can define a building’s resizability as when a model passes the integrity check and, after a redimensioning transform (for example, a change of scale), no new holes appear. All the command nodes that Müller and his colleagues proposed (except the Split command) are, by their definition, consistent. 2 The Split command can be problematic when it’s defined with all constant subdivision measures. That is, if the Split node receives a geometry that the upstream nodes defined as having a constant size, the input geometry’s size should be equal to or smaller than the sum of the sizes defined in the subdivision node. However, if the input geometry is relatively sized (that is, it can change when the size of the building changes) and the Split command has all constant subdivision sizes, the model simply isn’t resizable. Scalability. We’re interested in scalable buildings that we can use to continuously produce plausible buildings. In this case, the scalability depends mainly on the effect that the different commands produce on the final asset inserts. For instance, if a window asset is in a relatively sized Split node, and nothing prevents this node from being infinitely resized, the window will be stretched if the building is nonuniformly scaled. Verifying this is tricky because all paths from the root node (or nodes) to any Insert node should be tracked and verified in a top-down approach. Basically, there must be a Repeat node in every direction (for example, horizontal and vertical) between any relatively sized Split node and the Insert node. For example, the model in Figure 4b isn’t scalable, as you can see in Figure 4c. Fortunately, the system can easily solve this problem, as I explain later.
(a)
(b)
(c)
(d) Figure 4. The scalability of building models. (a) A photograph of a building. (b) A model using the assets in Figure 1. (c) The model is resizable but not scalable. (d) The model is scalable once the user has applied the graph-rewriting rules to correct the unscalability.
Detecting connected components. The graph’s connected components generally represent the graph’s pieces. Two nodes are in the same component of a graph only if there’s a path that connects them (without considering the paths’ directions). By detecting the connected components, you can easily determine whether any rules or sets of rules are unused, and then notify the user of this.
Graph Rewriting One of the new paradigm’s most important applications is the ability to apply graph rewriting to the graph-based buildings. The idea is to automate the creation of new graphs from the original graph. Formally, a graph-rewriting system consists of a set of graph-rewrite rules of the form p : L → R, IEEE Computer Graphics and Applications
71
Feature Article
with L being the pattern graph and R being the replacement graph. A graph-rewrite rule is applied to the host graph by searching for an occurrence of the pattern graph and replacing that occurrence with an instance of the replacement graph. Developing a full graph-rewriting system is beyond this article’s scope, so I now present a few examples that address the scalability issues
One of the most practical criteria is to use a filter based on the labels the geometry has received during its previous upstream processing.
sume that, starting from the bottom, the first floor is the ground floor and the last one is the roof.1 Then, this operation applies a Repeat node to the intermediate floors, if it wasn’t already applied. If the node has only two splits, repeat the topmost one: c : Split(Y){G…I|T} → Split(Y){G…Repeat{I}|T}, A Repeat node should not follow the Split(Y). The rationale behind this is that buildings tend to have unique structures for their lower floors, but the upper ones tend to be uniform. Ground floor repetition. At the ground floor, you apply a Repeat operation to everything that doesn’t have a “door” label: d : Split(X){door|A} → Split(X){door|Repeat{A}}.
I mentioned earlier. These examples have proven useful for automatically creating buildings using captured data.6,7 In particular, the following rules a and b′ are inspired by Paul Merrell’s research,8 and b and c are inspired by Daniel Aliaga and his colleagues’ research.9
Poor man’s horizontal repetition. For any floor but the first, if you can’t apply rules b or c, repeat the full floor pattern (see the rightmost building’s roof in Figure 1):
Repeated-entry generalization. If a Split node has a tag “A” repeated twice or more, you can transform it into two new operations: a Split and a Repeat node. The new Split node would be identical to the original one, but with the repeated entry replaced by a single subdivision with size equal to the sum of the original ones. The output of the Split node will be routed to the Repeat node, which will have the original size from “A” as the size for the repeated split.
The results. By applying only rules a and c, graph rewriting automatically transformed the nonscalable building in Figure 4c into the scalable building in Figure 4d. I created all the models in Figure 1 by applying user-configurable combinations of these rules to initially nonscalable buildings.
a : Split{…A|A…} → Split{…Repeat(A)…}. In the system, the assets already represent a window and part of its surrounding wall, so I can’t ignore this case, as others do.9 Repeated pairs. This case is similar to the previous one, but for repetitions of AB pairs: b : Split{…A|B|A|B…} → Split{…Repeat{A|B}…}. A simple variant is b′ : Split{…A|B|A…} → Split{…Repeat{A|B}|A…}.8 Upper-floor repetition. You can assume that, starting from the root, the first Split(Y) node you find subdivides the facade into floors. You can also as72
March/April 2012
e : Split(X){…} → Repeat{Split(X){…}}.
Local Control I added direct, persistent local control over the generated instances without breaking the graphbased streaming process.3 For that, I used the extensibility mechanism I described earlier to create an Exception node that splits the geometry into a new stream, using user-defined criteria. One of the most practical criteria is to use a filter based on the labels the geometry has received during its previous upstream processing. Besides the labels used to define the rules, the system attaches to each geometric element labels related to the Split and Repeat nodes it goes through. That is, at each of these commands, the geometry receives a label identifying the command plus the ordering number. This way, if you want all windows in a column, you only need to filter the geometry by identifying the Split(X) command in addition to the column number you’re interested in. Figure 3 shows this situation; Figure 5 shows both an example in which one window in a façade changes and the resulting graph. This mechanism
InitialShape Comp
Subdiv_Facade Repeat_Floors Insert_door Repeat_Windows
Insert_Window
InitialShape Comp
Subdiv_Facade Repeat_Floors Insert_door Repeat_Windows Exception plain_Window Insert_Window
Figure 5. You can easily add local control by using the simple Exception node, which classifies geometry on the basis of a given expression.
differs from the one that Bernd Lintermann and Oliver Deussen presented10 because my method doesn’t imply duplication of the node that’s wired to the exception, nor is it limited to a simple numbering mechanism. This allows the full hierarchical control described in other research.3 In this DAG-based context, you face the same persistence problems that Lipp and his colleagues described when the graph changes through user interaction.3 Fortunately, you can easily adapt their algorithm to the graph-based representation, so you can use their solutions. For example, you can easily implement exact in
stance locators and semantic instance locators as pathfollowing algorithms inside the DAG. To compute sequential orderings, you can implement a two-pass algorithm. The first pass recursively computes the number of shapes for each node, pulling the values from the leaves so that each upstream node knows the number of columns (or floors) for each of its outputs. For Split operations, the number of columns (or floors) is the sum of the numbers for each subdivision. Repeat nodes require special care; they have a variable number of output shapes depending on the input’s size. Keeping track of semantic instance IEEE Computer Graphics and Applications
73
Feature Article
(a)
(b)
(c)
Figure 6. A complex model (the Raccolet House) recreated by the system. (a) Different construction stages (from left to right). (b) The graph for the left wing and the whole model. (c) The textured model. Here, you can see the simplicity that the visual metaphor and the shift in the development paradigm represent for the end user compared to traditional text-based methodologies.
74
locators easily solves this. Any other node simply returns the maximum value among its outputs. The second pass is another recursive traversal that assigns each terminal shape the accumulated value of its position with respect to every upstream node. However, I’ve found that the simplicity and robustness of using an Exception node renders it almost unnecessary to define these locating mechanisms. That’s because it’s much simpler to let users manually reconfigure the node. This remains an open area of research.
Other researchers have defined an oriented bounding box in space called a scope, using it so that any nonterminal symbol in a rule will be created with the current scope.2 Because I use a streaming node-based paradigm, users must take care when tracking the affected geometry’s scope. In this implementation, each geometric element that’s streamed through any node carries its own oriented bounding box, so every subsequent operation can be executed as described in “Procedural Modeling of Buildings.”2
Implementation
Discussion
I developed a fully functional prototype for generating graph-based buildings on top of SideFX Houdini (for more on Houdini, see the sidebar). For that, I used a mixture of Houdini’s own nodes, embedded Python scripts, and external Python methods. I put a selector text box (called a filter) on every node so that users can select a predecessor label from the incoming geometry stream. I implemented labels from Houdini’s grouping mechanism. Every node also provides a labeled output stream. For Comp and Split nodes that can provide geometry with different successor labels, I labeled the resulting streamed geometry by the respective operations and merged it to provide a final, unified output. I used external Python methods to implement graph visitors, which are recursive procedures that implement either a preorder graph traversal or a topological sort-based one. Figure 6 shows a complex model I created with this prototype.
My easy-to-understand visual language for rule applications lets users interactively construct and modify buildings on the fly. Compared to editing a tree’s parameters,3 editing at the graph level is more intuitive because it frees users from browsing the tree for a shape. Also, the local control mechanism I described earlier provides the same controlling capabilities at a higher abstraction level. My approach incorporates previous grammarbased schemes because they basically consist of commands applied to geometric shapes. That is, sequential applications only require linearly chaining nodes, whereas an independent application (using a stack-based notation) simply means wiring more than one output node to a given input one. But not all graphs can be mapped to grammars (for example, a node requiring two different inputs). Users could do some of the operations I described using a traditional text-based approach. However, many of these operations would require treating
March/April 2012
the text-based rules as a graph-data structure so that the users can follow executions from one rule to another. So, the need to traverse the rules as a graph confirms the need for a user-friendly graphediting tool. For instance, a text-based representation could perform the upper-floor repetition I described, but it would need to verify whether a Repeat operation is wired to the Split node. This implies following the rule execution, which is a graph-based operation. This approach has some intrinsic limitations, though. Imagine that a user is scaling parts of a building for which no repeat operation is possible. If the shapes of certain assets are baked into the scaled asset itself (for example, the windows in the roof shape in Figure 4), preserving the shapes of the windows under the roof so that they don’t get stretched is almost impossible. The only solution in this case is to use a rule such as rule e in the “Graph Rewriting” section to make it repeatable. Another limitation of the approach, and of shape grammars in general, is that some architectural styles don’t lend themselves to shape grammars— for example, Antonio Gaudi’s and Frank Gehry’s designs.
I
barely scratched the surface of the many possibilities that graph rewriting introduces for creating procedural models of buildings. One of the most exciting lines for future research is automatic rule-set generation, which would unburden artists who must create whole cities. Instead of creating cities building by building, they could create an adequately rich initial building and then automatically create several variations. In this line of reasoning, it’s sensible to consider using graph-transformation grammars to enumerate all possible graphs from a given graph. (Sometimes “graph grammar” is a synonym for graph-rewriting systems, especially in the context of formal languages.) This way, you could describe a higher-level graph-transformation language, instead of transforming a given graph into a new graph. Also, taking ideas from compiler theory, you could perform common subgraph elimination efficiently (replacing all repetitions by a single instance). This would result in much simpler representations that would be faster to evaluate. Following the same line of reasoning, using automatic parallelization would allow parallel rule-set evaluation, making this approach suitable for modern parallel architectures such as GPUs and multiprocessor-based CPUs.
Acknowledgments I thank Ferran Roure, Remei Ridorsa, Francisco Gonzalez, CG&A’s editors, all members of the Geometry and Graphics Group at Universitat de Girona, my reviewers, and everyone who helped at the Houdini forums. The Raccolet House was modeled by DAZ 3D software (www.daz3d.com). Spain’s Minister of Science and Innovation funded this research under grant TIN2010-20590-C02-02.
References 1. P. Wonka et al., “Instant Architecture,” ACM Trans. Graphics, vol. 22, no. 3, 2003, pp. 669–677. 2. P. Müller et al., “Procedural Modeling of Buildings,” ACM Trans. Graphics, vol. 25, no. 3, 2006, pp. 614–623. 3. M. Lipp, P. Wonka, and M. Wimmer, “Interactive Visual Editing of Grammars for Procedural Architecture,” ACM Trans. Graphics, vol. 27, no. 3, 2008, article 102. 4. P.E. Haeberli, “Conman: A Visual Programming Language for Interactive Graphics,” Siggraph Computer Graphics Q., vol. 22, no. 4, 1988, pp. 103–111. 5. C. Upson et al., “The Application Visualization System: A Computational Environment for Scientific Visualization,” IEEE Computer Graphics and Applica tions, vol. 9, no. 4, 1989, pp. 30–42. 6. P. Müller et al., “Image-Based Procedural Modeling of Facades,” ACM Trans. Graphics, vol. 26, no. 3, 2007, article 85. 7. P. Koutsourakis et al., “Single View Reconstruction Using Shape Grammars for Urban Environments,” Proc. 2009 IEEE 12th Int’l Conf. Computer Vision, IEEE CS, 2009, pp. 1795–1802. 8. P. Merrell, “Example-Based Model Synthesis,” Proc. Symp. Interactive 3D Graphics and Games, ACM, 2007, pp. 105–112. 9. D.G. Aliaga, P.A. Rosen, and D.R. Bekins, “Style Grammars for Interactive Visualization of Archi tecture,” IEEE Trans. Visualization and Computer Graphics, vol. 13, no. 4, 2007, pp. 786–797. 10. B. Lintermann and O. Deussen, “Interactive Modeling of Plants,” IEEE Computer Graphics and Applications, vol. 19, no. 1, 1999, pp. 56–65. Gustavo Patow is an associate professor in Universitat de Girona’s computer science and applied math department. His research interests include modeling, texturing, GPUbased rendering, and procedural modeling. Patow has a PhD in computer science from Universitat Politècnica de Catalunya. Contact him at [email protected]. Selected CS articles and columns are also available for free at http://ComputingNow.computer.org. IEEE Computer Graphics and Applications
75
