Relevance Filtering in Distributed Simulation

1. Relevance Filtering in Distributed Simulation Ali Berrached ADSRC University of Houston-Downtown Arached@dt.uh.edu

2. Outline • Distributed Interactive Simulation (DIS) • DIS Scalability Problem • Relevance Filtering • The High-Level Architecture (HLA) Framework • Overview of Data Distribution Management • Approaches to Relevance Filtering in the HLA Framework.

3. Distributed Interactive Simulation(DIS) • DIS defines a framework for interconnecting • different types of simulation applications • a common and consistent virtual world • real-time interaction • battle simulation, disaster emergency situation, peace keeping/relief ops, etc. • In the DIS virtual world simulations interact by exchanging information about the objects (e.g. vehicles) they represent • DIS defines a communication protocol: a set of data messages called Protocol Data Units (PDUs) • Location X(t), Y(t), Z(t) • Speed Vx(t), Vy(t), Vz(t) • Acceleration Ax(t), Ay(t), Az(t) • DIS protocols are described in IEEE Standard 1278 http://www.ist.ucf.edu/~STDS/

4. Scalability Problem • Every simulation keeps a dead reckoning model of all objects in the distributed simulation. • DIS requires simulations to broadcast state changes to all other objects in the distributed simulation. • As the number of simulators increases the PDU traffic exchanged among simulators increases tremendously. • Effect on the underlying network • Effect on the simulation host • To achieve good scalability, the available communication bandwidth must be used as efficiently as possible.

5. Relevance Filtering • Rational: In most distributed simulation applications a simulated object is only interested in a subset of all other objects in the virtual world. • If two simulated vehicles, V1 and V2, are separated by a large distance in the simulated world, then state updates from V1 would be irrelevant to V2; and vice versa. • Example: In one STOW 4-hours training scenario with 10,000 simulated objects distributed over 24 sites, 90% of state PDUs were needed by only 5or less sites. Virtually no state PDUs were ever needed by more that a third of all sites.

6. Relevance Filtering Objective • Reduce the amount of irrelevant data exchanged among simulations • Send data only when and where it is needed. Approaches • Filtering at the destination: The receiver analyzes incoming PDUs and filters out those that are not relevant to the local simulator. • simple to implement • does not reduce network traffic • Filtering at the source: this is more difficult because it requires that the sender has enough knowledge about the state of the remote simulator (receiver) to decide if update is relevant. • There is also the risk of errors: that is filtering out data updates that are relevant.

7. Example Filtering Errors A B A and B are outside each other’s region of interest. PDUs are filtered out: they don’t exchange state information. A starts moving towards B. But because of filtering they don’t realize it. A moves into B’s region of interest. But neither realize it.

8. High-Level Architecture (HLA) • Successor of the DIS Standard. • Overall Architecture: • A group of federates (i.e. simulations): Each federate simulates a set objects. • The Run Time Infrastructure (RTI) • RTI: • provide a set of services to the federates • facilitate interaction (and interoperability) among the federates. More Information about the HLA: http://www.dmso.mil/projects/hla

9. Federate Federate Federate RTI Interface RTI Interface RTI Interface RUN TIME INFRASTRUCTURE NETWORK COMMUNICATION SERVICES

10. Data Distribution Management • Data Distribution Management: set of services that the RTI provides for federates to facilitate relevance filtering • RTI provides mechanisms that allow each federate to declare what classes of objects and object attributes it is willing to publish to other simulations, and what classes of objects and attributes of other federates it is interested in. • Example: V1 interested in receiving updates from all objects that are within a 10km radius from my current position.

11. Routing Spaces • A routing space is a multidimensional coordinate system in which federates express an interest for either receiving data or sending data • Each dimension defines a domain for an attribute value • example: a routing space for X, Y, Z coordinates a routing space for speed • each federate decides which routing spaces are useful and defines its regions of interest for subscription and/or update within those routing spaces.

12. Regions • A region is a set of bounded extents in a routing space • an extent along each dimension • by specifying a subscription region a federate tells the RTI to deliver only the data which fall within the extents of the regions • by specifying an update region (associated with an object) the federate promises the RTI that the attributes of the object instance will stay within the extents of the region (until the update region is changed again)

13. Federate B U1 S1 Federate C S2 Federate A • 2-dimensional routing space • U1: update region of Federate B • S1 and S2: Subscription regions of Federates A and C • A data update is relevant to a federate if the update region of the publishing federate overlaps the subscription region of the subscribing federate. • A connection is established between the publishing federate and the subscribing federate.

14. C 1 0 0 k m Y ( 3 0 , 7 0 ) 1 ( 7 0 , 6 0 ) A ( 6 0 , 4 0 ) 2 ( 4 0 , 3 0 ) 3 B ( 7 0 , 2 0 ) 0 0 1 0 0 k m X 1. A 2D routing space with routing variables X and Y 2. Federate C subscribes for ships in the regions specified to the RTI by the extents {{40,70},{30,60}} 3. Federates A and B publish ships 4. The RTI delivers data updates for ship 2 to federate C - updates for ships 1 and 3 are filtered

15. Message Routing • All federates whose subscription regions overlap an update region are joined to a multicast group. Data flows from publisher to subscribers through the multicast group. • Multicasting provide added efficiency.

16. Objective of HLA: simulate 100,000 entities by year 2000 Limitations: • Matching updating regions with subscription regions is computationally demanding. • This scheme requires one multicast group per update region. Assuming each simulated object has only one update region per routing space, then we would need at least as many multicast groups as objects.

17. Multicasting • Federates join and leave multicast groups dynamically as their subscription/update regions change • dynamic multicasting • Multicasting incurs a cost on the underlying network • Use of a larger number of multicast groups implies an increase in group change rate (i.e. join/leave operation). Objectives 1. Efficient matching of update regions and subscription regions 2. Use multicast groups efficiently.

18. Fixed Grid Based Approach • Developed at MIT Lincoln Lab (D. Van Hook et al.). Used in current implementation of the RTI. • A routing space is subdivided into a predefined regular grid of d-dimensional cells. • Each cell is assigned a multicast group. • Each subscribing federate matches its subscription region(s) against the grid: federate joins multicast groups associated with cells that overlap its subscription region(s). • Each publishing federate matches its update region(s) against the grid: Attribute updates are routed to multicast groups associated with the cell(s) that overlap with the update region. Idea: If a subscription and an update region intersect, then they will overlap at least one grid cell in common and thus will be connected through at least one multicast group.

19. F1 0 1 2 3 S1 S2 5 6 7 F3 8 9 10 11 F2 U 12 13 14 15 F1 Joins multicast groups 4, 5, 8, and as a subscriber F3 joins groups 6 and 7 as a subscriber F2 joins multicast group 8 as a publisher => Data updates from F2 are routed to F1 through multicast group 8

20. Fixed Grid Based Filtering Features: • Matching update regions with subscription regions is computationally efficient (done implicitly through the grid). • Scalable: every federate (publishing/subscribing) can determine which multicast group(s) to join by simply finding which grid cells its update region overlaps. • no need for global state knowledge ( layout of the grid is static) Shortcomings: • Spurious connections: update and subscription regions may overlap the same cell without intersecting each other • causes data to be delivered even though their update and subscription regions don’t intersect. • Trade off : number of multicast groups vs. filtering efficiency.

21. F1 0 1 2 3 S1 S2 5 6 7 F3 8 9 10 11 F2 U 12 13 14 15 • U and S1 do not intersect • => data updates from F2 are not relevant to F1. • Data updates from F2 are delivered to F1 because U and S1 overlap cell 8.

22. Hierarchical Grid-Based Filtering Rational: • The limit on the number of multicast groups and their associated maintenance overhead is most critical at the wide area network level. • Multi-access local area networks (e.g. Ethernet or token bus based LANs) provide hardware support for multicasting. • As the network gets wider and larger, multicasting gets more expensive. Idea: • Use a hierarchy of grid based filters at different levels of the network hierarchy with increasing levels of grid granularity (i.e. “resolution”). • Filtering is done in stages with increasing accuracy as data moves from sender to receiver.

23. LocalNetwork LocalNetwork Site 2 Gateway Site 3 Gateway Agent Agent WAN Agent Site 1 Gateway WAN-level Grid LAN-level Grid F2 subscription region F3 Subscription region F1 update region Local Network A two-level Hierarchy • Each gateway keeps a record of the subscription regions of all federates in its site. The subscription region of a gateway is the union of the subscription regions of its federates. • Gateways use the WAN-grid to join the appropriate WAN multicast groups by matching their subscription regions against the WAN grid. • Federates use the LAN-grid to join the appropriate LAN multicast groups.

24. When a federate sends a data update: 1. The data update is sent to the local multicast groups assigned to the cells of the LAN grid that overlap the update region 2. A copy of the update and the update region are also forwarded to the local gateway 3. Local gateway sends the update to the WAN multicast groups assigned to the cells of the WAN grid that overlap the update region. When a gateway receives an update from a remote site: 1. Matches the update region against its LAN grid. 2. Sends the update to the local multicast groups assigned to the cells of the LAN grid that overlap the update region

25. Dynamic Grid-Based Filtering Rational: • “Objects” tend to be clustered in small areas and tend to move together. => Heavily populated areas and sparsely populated areas • Fixed grid approach allocates multicast groups uniformly across a routing space. Idea: • use a grid with varying resolution • By using smaller cells in heavily populated areas (allocating more multicast groups), more efficient utilization of multicast groups and more efficient filtering can be achieved. How: • Use quadtrees/octrees to represent routing space.

26. 0 1 3 2 0 2 1 3 2 0 1 3 2 0 1 3 0 1 3 2 • A quadtree/octree is hierarchical data structure that is based on recursive decomposition of space. 0 2 1-level quadtree 1 3 3-level quadtree 0 3 2 1 2 1 0 3 2 Variable Resolution Grid

27. 0 1 3 2 0 • While still enough multicast groups • Subdivide quadrant with largest • number of regions • Assign a multicast group to each leaf node of the quadtree R3 R1 R2 1 3 0 2 1 3 1 3 2 0 0 1 3 2 R3 R2 R1, R2 R2 R3, R2

28. Current/Future Work • Performance analysis of grid based approaches • Investigate performance cost of multicasting • Investigate clustering approaches: Group N objects in M clusters based on proximity of their update regions in a routing space.