Implementation
The key architectural choice we faced in implement- ing FUSE was whether to route all FUSE messages using the overlay paths, or to route certain messages directly between the group members. In the topology we used, spanning trees along overlay routes, path failures involv- ing delegates can be dealt with in one of two ways. One option is to signal a failure on all FUSE groups using that path. This has the advantage of implementation simplic- ity, but can be a significant source of false positives. In- stead, we chose the second option: to attempt to repair the liveness monitoring topology for the group.
Repair will succeed if the members of the group can still communicate with each other directly, and therefore repair routes around all failures involving delegates. We chose to implement repair by routing repair messages di- rectly from the root to the group members. The over- riding factor for this choice was rapidity of failure de- tection: relying solely on overlay routes would require waiting for the overlay to attempt to repair itself before signaling a failure. Using direct root to member com- munication allows failures involving group members to be detected more rapidly. This direct communication also results in better latencies for group creation and application-signalled failure notifications. When overlay routing paths are working, we still get the scalability ben- efits of shared spanning trees using the overlay. In the remainder of this section, we first describe the functional- ity exposed by the SkipNet overlay, and we then discuss the details of implementing each FUSE operation.
