Abstract
Computer simulation is a significant technology on making great scenes of crowd in the film industry. However, current animation making process of crowd motion requires large manual operations which are time-consuming and inconvenient. To solve the above problem, this paper presents an editing method on the basis of mesh deformation that can rapidly and intuitively edit crowd movement trajectories from the perspective of time and space. The method is applied to directly generate and adjust the crowd movement as well as avoid the crash between crowd and obstacles. As for collisions within the crowd that come along with path modification problem, a time-based solution is put forward to avoid this situation by retaining relative positions of individuals. Moreover, an experiment based on a real venue was performed and the result indicates that the proposed method can not only simplify the editing operations but also improve the efficiency of crowd motion editing.
1. Introduction
Computer simulation of crowd movement refers to setting up simulation models depending on crowd motion features in the reality, which means imitating the movement process of a bunch of people in the virtual environment. This technique is so useful that has been widely applied in the field of architecture, transportation safety, computer games, animation, and so on. In terms of film and television production, it can save time and massive money in making collective scenes that involved a large group of people, for instance, the battle scenes. Under the circumstance, it is meaningful to develop the production method of crowd animation, which would improve the productivity and make promotion to the animation industry.
Up to now, the making process of crowd animation relies on quantity of manual operations and it is difficult to edit every individual manually in large-scale crowd motion, especially when one’s appearance time needs to be precise or the speed of a certain group of people has to change over time. In addition, the influence on crowd formation and speed brought by editing is hard to eliminate. When adjusting the pace or routes of a small part of people in the group, the speed of the whole crowd often changes in a mutation which looks strange for walking people. Besides, collision matters are likely to occur after editing crowd motion and improper solutions may cause extensive crash among virtual characters. According to previous studies, crowd behavior simulation under different circumstances has been widely researched, such as simulating the crowd rushing to exits in a closed room [1] and establishing an interactive simulation model to imitate human-like behaviors among people as well as their influence to the environment [2]. Furthermore, many researchers paid attention to the calculation efficiency and large-scale crowd visualization. Wong et al. improved a model by combining clustering and spring force to govern individual movement in the virtual environment [3]. With the broad application of neural network, motion rules can be learned from real data so that human walking can be simulated in higher speed [4]. In terms of crowd formation, Hughes [5] considered crowd as flow continuum and continuum models have been set up to capture key features of pedestrian flows [6]. Nevertheless, the editing of crowd animation during the simulation process has been rarely studied. Therefore, this paper is aimed at proposing a method to effectively edit the crowd motion.
Inspired by the cage-based editing method by Kwon et al. [7], we adopt concave hull to encircle the crowd and adjust its motion as a whole, which was able to intuitively edit the crowd by dragging characters. Moreover, triangular mesh is used to establish topological relation among characters in the virtual population in order to maintain their relative positions. The modification process relies on the mesh deformation technique proposed by Liu et al. [8]. In addition, according to the editorial way mentioned by Kim et al. [9], this paper divide crowd movement into a spatial part and temporal part to adjust crowd formation and solve collision problems.
There are three main contributions of this research: (i)The proposed method can effectively edit the crowd motion both collectively and individually(ii)The method is able to retain the formation of the crowd and the relative positions between characters. Moreover, the collision problems between characters/obstacles are handled(iii)The method is suitable for simulating activities that needed fixed relationships among participants, such as the march-in ceremony and military review
2. Related Work
Crowd simulation technology mainly includes modeling and simulation of virtual people’s behaviors, group animation, and visualization of virtual crowd. Many researchers have explored the manners of virtual people. The seminal contribution to the simulation of crowd’s behaviors should be the work of Reynolds [10] in 1987. They thought flock behaviors were the dense interactions of the relatively simple manners of the individuals. In addition, a distributed model was set up to generate the simple individual behaviors and complex flock behaviors evolved from individual behaviors. Reeves [11] proposed a particle system which is a collection of minute particles, modeling irregular fuzzy object. Musse and Thalmann [12] presented a Vi-Crowd model, including a hierarchy of crowds, groups, and individuals. Each level of autonomy had its own behavioral rule, and the complex crowd behaviors were simulated by synthetic effects among all hierarchies, which took sociological aspects into consideration. On the basis of the above model, psychological effects were considered for crowd movement by Pelechano et al. [13]. Moreover, Chenney [14] described a method for representing and designing speed fields by using flow tiles, which realized the minimum impact from the mixture of social and psychological influences.
In the field of group animation, which is a branch of crowd simulation, Ji et al. [15] carried out researches on the algorithm of group movement simulation and proposed a method for group calisthenics based on group events. Takahashi et al. [16] presented a spectral-based approach to control the temporal and spatial distribution of individuals by mapping the group formation into the rotation interpolation of Laplace matrix eigenbases. Kovar et al. [17] suggested a novel method based on motion graph which was automatically constructed, and walks can be built on the graph to generate motion. Interactive manipulation of multicharacter animation was initially proposed by Kwon et al. [18]. They used a graph structure to model the spatiotemporal group behaviors of the crowd and employed a mesh-editing algorithm to manipulate the animation interactively. Ulicny et al. [19] put forward the concept of crowdbrush that users could brush in three-dimensional scenes with a brush metaphor, and then the system would generate a crowd in the brushed place. Furthermore, different brush colors form distinct crowd behaviors. Allain et al. [20] applied constraints to control the crowd. Different from other methods, they divided the constraints into macroconstraints and microconstraints. Macroconstraints were used to constrain the macroscopic characteristics of the crowd, for example, the speed field; microconstraints were used to constrain the crowd speed, location, and so on.
3. Crowd Motion Editing Based on Mesh Deformation
Crowd scenes are commonly used in animation, but group motion is complicated and difficult to modify. For specific scenes that required large crowd, such us parade and mass choreography, if we could edit people’s movement in a straightforward way, what you see is what you get, it would be a useful method to simulate crowd animation. Therefore, this paper proposed a method that can intuitively edit group motion through mesh deformation, separating the crowd movement depending on spatial and temporal attributes. The method can be divided into three portions which are the establishment of representation model, the spatial track deformation, and the temporal track deformation.
The overview of the editing model is shown in Figure 1 that is organized by the crowd motion editing method and trajectory deformation model. In Figure 1(a), the lower left corner is the generation process of the spatial track that contributes to form the representation model above. The representation model demonstrates the trajectory situation of the crowd, which can be edited according to spatial and temporal attributes, connecting two methods on the right side of this part. The editing method applied on spatial trajectory calculates new routes of group motion through the trajectory deformation model presented on Figure 1(b), and so does the model apply on the temporal track editing method. Therefore, the trajectory deformation model maintains relative positions of individuals as well as plans new paths. Then, it returns the editing results to the crowd motion editing method and rebuilds the representation model.
3.1. Representation Model of Temporal and Spatial Track
The path of a virtual character in the crowd simulation is a sequence of the motion clips arranged in a chronological order. In this thesis, the A algorithm [21] is applied to calculate crowd tracks so that the motion clips can be formed by taking sample points on the tracks.
As it is displayed in Figure 2, every spatial track is composed of a two-dimensional set of points obtained by projecting the motion clips onto the ground. More specifically, consider as the trajectories of the first person to the th one accordingly. The th trace is on behalf of a piecewise curve, and the collection of any point on this curve signifies a sequence of people’s positions in time order. The direction of a certain point on the track is represented by local coordinates: and , where is a rotation matrix of angle 90° [9].
In terms of the temporal attribute of the virtual population shown in Figure 3, regard as the temporal tracks of the first characters, respectively. Each track can be sampled and serialized into a set of one-dimensional points which show one-to-one correspondence to the points on the spatial track. Suppose as a point on the time series, then is the interval from point to . Parameter indicates the arrival time of the th person in the th place that is in accordance with point . By adjusting the time sequence of the crowd, their speed displaying on a part of the space tracks can be changed. As a result, the crowd is able to reach a specific position in a given time.
3.2. Spatial Track Deformation Model Based on Concave Hull
The model is built on the basis of concave hull that encircles the selected crowd’s space/time trajectories represented by Mean Value Coordinates (MVC) [22] with respect to the vertices in the hull (for details, refer to [22]). After generating the hull, Delaunay triangulation is performed to divide the area into numerous triangles and set up a triangular mesh. In consideration of keeping the relative positions among virtual population while editing, triangular meshes are used to establish the topological relationship. Therefore, the position of each individual is taken to form the vertices of the grid and mesh deformation with rigid characteristic [23] is applied to edit the crowd motion. After dragging a single vertex on the hull, the shape of the surrounding area is able to be modified through mesh deformation, changing the motion tracks of the crowd thereby.
When the user drags a point on the hull without fixing other points, as is shown in Figure 4, the deformation can be calculated by where is the constraint matrix to remain the dragging point unchanged during the deformation process. represents the distortion energy which is used to measure the extent of the mesh deformation. Introduce as-rigid-as-possible deformation energy [23] as where and denote the th vertex of the th triangle after and before the deformation in the triangular mesh, respectively. represents the rotation estimation matrix from the deformed triangles to the triangles before deformation. in the following function is aimed at adding the local effect of deformation: where is the coefficient matrix. The closer the other points are to the dragging one, the smaller their coefficients are and vice versa. is the position vector of the vertex before the deformation in the triangular mesh. is the vertex of the concave hull after deformation. is an matrix. means the number of points on the tracks of the selected part with the addition of —the vertexes of the concave hull. The first rows of are the MVC [9] of the points on each track; the last rows with columns of the matrix form the unit matrix: where is the number of points on the tracks of the crowd.
As for the computation of coefficient matrix , breadth-first search algorithm in the graph theory is used, namely, considering the mesh as a directed graph. The traversal begins from the dragging vertex and then goes through every layer of points around it. According to the layer combined with the distance from the start point, coefficients can be calculated. The formula of calculating the coefficient is shown below: where is the parameter of the nonstarting node in the mesh; the coefficient of the starting node is 0. and are nonzero coefficients. is the number of points in between the present vertex to the starting node. is the threshold set in advance, and is the distance covering from the current node to the starting one. It can be seen from Equation (5) that as increases, makes more influence on .
Based on the results of Equation (1), the closer the points are to the dragging vertex on the mesh, the greater the impact is; however, if the fixed point is far from the dragging vertex during the editing process, it may affect nothing to the points around the fixed point according to Equation (1). Nevertheless, often, it is the case that the mesh deformation is supposed to perform between the dragging point and the fixed point, as is presented in Figure 5. Therefore, Equation (6) is used to deal with the situation with the fixed point:
In order to expand the influence of the dragging point, is used to measure the distance variation among the vertexes of the concave hull, which is approximately presented by the change of the components on the -axis and -axis: where is an matrix:
After adjusting the tracks of the crowd by means of mesh deformation, the speed of some people in the crowd may become unnaturally fast and slow. This is because the distance between the sampling points on the character’s track varies sharply; however, the time interval between them has not changed. Equation (9) is displayed to solve this problem: where and represent the points on the tracks of the crowd before and after the deformation, respectively. is the time interval between two points on the track before deformation.
The solving process of the target equation is demonstrated in Algorithm 1.
| ||||||||||||||||||||||||||||||
3.3. Temporal Track Deformation Model Based on Concave Hull
As users are able to drag points on the space track to manipulate crowd tracks, they can also operate the temporal track as well. When crowd tracks are discretely sampled, every sampling point on each track has a corresponding time node which indicates the moment that a character appears at this position. Therefore, it is not the only advantage for modifying the speed in a section of track after adjusting the time node; the time of occurrence at a certain place is also under control. When users drag the corresponding points on temporal tracks, the following equation is applied to adjust these points: where represents the degree of temporal deformation, which is similar to. controls the local variation traits of spatial deformation. means the vertexes of the concave hull after deformation. The equation is used to ensure that the fixed points or the dragging points are not changed. where matrix is the difference vector of two adjacent time nodes and is the default time interval vector. and denote the time node vector after and before the deformation separately.
4. Collision Avoidance Method Based on Space-Time Trajectory Deformation
During the process of crowd motion editing at present, the movement tracks are manually set. However, because of the huge amount of virtual characters involved with complicated routes, it is hard to notice the collisions among the crowd, let alone avoiding the crash by manual adjustment. Thus, collision detection algorithm and collision avoidance algorithm are indispensable, which make a great difference to the crowd animation quality. Therefore, we put up the measures of avoiding the collisions between characters/obstacles.
4.1. Collision Avoidance Method among People Based on Space Track Keeping
In view of collision avoidance among characters in the crowd, a method based on space track keeping is proposed—that is, to adjust the time tracks of related people to stagger the collision points. The strategy is shown in Figure 6.
4.1.1. Collision Detection
The collision problem of the th character is detected by whether the minimum distances between the th track and other tracks in the same section are less than the threshold.
in Equation (13) displays the nearest distance between individual and :
In order to gain the relevant time information, take ’s partial with respect to after getting : and then, make to solve . The result is presented in Equations (15) and (16): where is the time that two individuals get the shortest distance in different situations. In combination with , , and the speed mentioned in the above functions, the spatial position of collision can be presented, respectively, as and .
The specific algorithm is shown in Algorithm 2.
| ||||||||||||||||
4.1.2. Collision Avoidance
The space-time trajectory is the combination of the spatial track and the time track. It can be regarded as a polyline in a three-dimensional space, comprising the spatial position coordinates of the character and the timeline. When the collision occurred, the minimum distance of two segments of the broken lines in this three-dimensional space is less than the threshold. Therefore, enlarging the distance over the threshold is able to prevent the collision.
It is illustrated in Figure 7 that adjusting the time track can affect the character’s three-dimensional trajectory to change the minimum distance between two lines. The distance, therefore, becomes greater than the threshold with no need to change their spatial locations. This method is suitable for scenes with complex terrain and large crowd density, because it can avoid the collisions in the crowd without worrying about colliding obstacles in the environment.
In order to achieve the above purpose, the scaling factor and translation coefficient of time sequence are defined. When characters and have a crash, suppose that the crash moment is in the period and for two pedestrians separately. After the scaling translation with and , the time periods turn to and , respectively. Therefore, the collision can be prevented if previous time periods are replaced with the changed one.
The scaling factor and the translation coefficient can be found according to Equation (19), while Equation (18) measures the change of the time before and after the adjustment. where , , and . is the time interval between character and, which is estimated as follows: where means the closest distance between two pedestrians. If the distance between two lines is less than, the collision will happen.
Equation (19) is solved on the basis of the Kuhn-Tucker condition, that is, calculating the gradient of ,, , , , and , respectively, to find and satisfying Equation (23). The time nodes after adjustment are, , , .
After calculating the related time nodes, it is necessary to adjust the overall time series of the two characters in order to ensure that their speeds do not suddenly become very fast or slow. Considering that the change in speed is continuous, so the time people spent in per unit length is similar, expressed by: where is the length of the th segment on the pedestrian track and is the time interval of the th segment.
In view of the similarity it should have between the speed before and after, is introduced to measure the degree of the speed change as shown in Equation (25). Nevertheless, the time node of the crowd is the only alteration during the whole process of collision avoidance, and therefore, the speed variation can be substituted by time information. Hence, is defined to represent the modification on time as well as on speed for the replacement of . where is the modified time interval. In the whole changing process, the adjustment of time node is hoped to be as little as possible. As a result, is going to measure the degree of the change of time node as described in
Associating with Equations (24), (26), and (27), the avoidance method can be demonstrated by an optimization model, as shown in where ,, and are the weight coefficients of , , and , respectively. The function of the matrix is to filter out the time nodes adjusted by Equation (18), while is a column vector consisting of the values of modified time nodes. The purpose of the constraint equation is to keep the time points that have been changed in the collision avoidance process away from changing again when the whole time track is later modified.
The specific algorithm of collision avoidance method is shown in Algorithm 3.
| ||||||||||||
4.2. Obstacle Avoidance Method Based on Spatial Track Deformation Model
When the crowd pass through a narrow channel as displayed in Figure 8, it is easy to collide with obstacles. It is very tedious for animators to manually manipulate virtual characters avoiding obstacles, which consumes a lot of time. The reason of the crash is when the crowd turn at a narrow junction, the maximum width of the crowd is bigger than the width of the intersection, as shown in Figure 9.
Conditions for the crowd that smoothly pass obstacles can be expressed by the formula , where can be calculated by where and are the th position of the th character and the th position of the th character, respectively. is a collection of characters’ positions at the narrow junction. means the angle between the vector and the moving direction. In order to solve this collision problem, a measure integrating mesh deformation and cage-based method proposed by Kim et al. [7] is put forward to adjust the spatial trajectories of the crowd. This measure can restrict virtual characters to a narrow road without crossing obstacles by adjusting the contour of the mesh.
As is shown in Figure 10, the contours with squares form an enclosing shell of the crowd. Since the shell is always on the periphery of the crowd, when the crowd is crossing the obstacles, the shell also gets through. In other words, if the shell has no intersection with barriers on the route, characters in the shell will not touch the barriers, too.
Thus, adjusting the squares on the contour line of the shell to move them out of the obstacles can effectively avoid the crash, so that the crowd will not get through the obstacles. The schematic diagram of adjusting the bounding shell is presented in Figure 11, where the squares are the vertices of the shell and the circular points are the mapping points of the vertices inside the obstacles.
As is presented in Figure 11, parts (a) to (c) describe the process of detecting whether the enclosing shell passes through the obstacles, calculating the mapping points, and obtaining movement directions, as well as moving the vertexes to the new positions. Part (d) represents the generation of mesh deformation by solving Equation (1) to get the end result. Figure 12 illustrates the overall situation when getting through the narrow road.
Mesh deformation is likely to change the trajectories of some characters in the crowd, which may lead to a sudden change in their speeds. Hence, Equation (9) is applied to adjust the speed, so that the movement becomes more realistic.
It is probably that the whole structure of original mesh will be influenced after deformation when a part of the mesh has been greatly narrowed down. As a result, the distance between each character is likely to become short and even brings track overlap, which easily causes the collision. Therefore, the collision avoidance method mentioned in Section 4.1 is used to solve this problem.
5. Experimental Results
To verify the performance of the proposed method, different experiments were designed to simulate the crowd motion according to distinct demands. This section demonstrates experiments on crowd motion editing and collision avoidance. Meanwhile, the practical use of the method has been tested by setting up an editing system for the badminton hall of Beijing University of Technology to imitate the event-holding situation. The experiments were programmed in C++, and OpenSceneGraph was used for rendering scenes. Furthermore, MATLAB was applied to compute the target equations and optimize the solutions. The hardware configuration of the experimental platform is shown in Table 1.
5.1. Crowd Motion Editing Results
According to Section 3, people’s movement in the crowd is divided into spatial trajectory and temporal trajectory, while the position relationship between each character is recorded by the triangular mesh. Therefore, experiments in this part were conducted based on space and time attributes separately, which is in accordance with the description in Section 3.
5.1.1. Spatial Track Editing Results
In terms of editing the space tracks of the crowd, the experiments are focused on the mesh deformation capacity, the smooth transition problem of trajectory after editing, and the speed mutation. Figure 13 demonstrates the process of establishing, choosing, and editing the tracks as well as presents the final routes of characters. Equation (1) is used to calculate the editing result after mesh deformation, which is shown in part (c). It can be noticed that the edge of the crowd, that is, the concave hull, is in a smooth change during the editing.
In addition, the result reflects that partial deformation is caused because of Equation (1). There are two main factors deciding the deformation range brought by the above equation. The first one is the influence on the mesh given by editing. If the final position is far away from the original point, the whole grid will be under great influence, and therefore, the range of influence will be wide according to Equation (2). In order to make sure of the smoothness of the modification, it must be a gradual process that a large range of grid is involved. The second factor is the calculation of coefficient matrix in Equation (3). The calculation result of Figure 13 is computed by Equation (4) that is set as , , and . If increase fast, the influence range will be small.
Table 2 demonstrates the comparison of two characters’ speeds before and after editing. Because there are many segments on a trajectory, we only choose sections with large speed change. It is displayed in the table that after spatial editing on the crowd, their speeds have been influenced for the distance between each point is modified in some parts of the space tracks while the original time interval is retained. Equation (9) is applied to adjust the characters’ temporal tracks to correct their speeds, and the result shows obvious effect after correction.
5.1.2. Temporal Track Editing Results
As for adjusting the time information, three characters in the crowd were chosen in this experiment. In Figure 14(a), we edit the temporal tracks of the bottom three characters to slow down their initial speeds and use Equation (9) to calculate the editing result. Parts (b) and (c) reflect the crowd formation before and after temporal track editing, respectively. It can be seen from the figure that the time modification causes variation on speed and further brings the formation change. In addition, it is shown in part (c) that the edited people remain in their relative positions during proceeding for their speeds are adjusted in an overall way.
5.2. Collision Avoidance Results
In this section, we concentrate on collision detection as well as avoiding the crash that happens on characters in the virtual environment, which are the foundation of making crowd animation and has direct relation to the animation quality. In the experiment, a standard path was calculated by the A algorithm from the starting point to the end. Then, other walks were generated by copying the standard path in accordance with the number of characters. To observe the effect of methods mentioned in Section 4, a part of the road was narrowed down to create an environment that is prone to accidents.
5.2.1. Collision Avoidance between People and Obstacles
Though the path avoided all of the obstacles during construction, it is likely for the crowd to crash on barriers when they pass a narrow alley. Hence, the mesh deformation method described in Section 4.2 is used to avoid characters getting through the obstacles.
As shown in Figure 15, part (a) is the original mesh of the crowd while part (b) presents the mesh after deformation under the constraint of the concave hull. It is illustrated in part (b) that the whole crowd with the concave hull (the outline consists of yellow points) exactly fills in or is narrower than the road. Because of the restriction from the hull, crowd tracks are bounded outside obstacles and achieve collision avoidance.
5.2.2. Collision Avoidance among People
On dealing with the crash between virtual people and the environment, collision within characters is easily involved. For the mesh is constructed from every spatial trajectory, deformation will cause track variation and also influences a character’s speed, proceeding direction, and destination, resulting in crash in high possibility. The red circle in Figure 16 reflects the collision.
The measure mentioned in Section 4.1 is applied to this problem, and Figure 17 demonstrates the effect after modifying the spatiotemporal trajectories of characters.
For overall adjustment and optimization solution are needed for the crowd track after each modification, the convergence rate is low when a considerable number of people participate, and therefore, the calculation is time-consuming. In order to improve the computational efficiency, the crowd track was compressed before collision detection. After detecting and handling the collision problems, the results calculated on the basis of compressed tracks can be converted to that before compression. Equation (30) is used to transform the result after compression: where and present the th time point and track segment of the th character, respectively. Equation (30) informs that the result conversion is actually a process of time allocation in proportion.
5.3. An Application Case on Real Stadium
For practical use of the proposed method in Section 3 and Section 4, the badminton hall of Beijing University of Technology was mapped to a virtual scene to simulate crowd motion. The scene has nearly 7000 seats, and virtual characters are supposed to walk through the aisles naturally. As it can be seen in Figures 18 and 19, the model imitates the marching process of the crowd when they enter and leave the stadium, and the mesh deformation method with spatial and temporal track adjustment was applied to the movement editing.
6. Conclusions and Future Work
This paper put forward a crowd motion editing method that can be applied on crowd animation and simulation. The method is able to retain the relative positions of people, which is suitable for activities that needed fixed orientation relationships among participants, such as the march-in ceremony and military review. Meanwhile, this mesh deformation-based method directly edits the crowd by dragging the mouse, which is easier to operate and improves the simulation productivity.
Nevertheless, there is still room for promotion. Because of large amount of data involved in the editing process, future studies can focus on the enhancement of computational efficiency. In addition, the proposed collision avoidance method is time-based, and therefore, combining collision problems with spatial attributes is worth studying in the future.
Data Availability
The data that support the findings of this study are available from the corresponding author upon reasonable request. Correspondence should be addressed to Yong Zhang: [email protected].
Conflicts of Interest
The authors declare that there is no conflict of interest regarding the publication of this paper.
Acknowledgments
This work was supported in part by the National Natural Science Foundation of China under Grant 62072015, U1811463, 61771058, and 61876012, in part by the Beijing Municipal Science and Technology Project under Grant Z171100004417023, and in part by Beijing Key Laboratory of Multimedia and Intelligent Software Technology, Beijing Artificial Intelligence Institute of Information Technology, Beijing University of Technology, Beijing, 100124, China.