Title: Incite: Edge-Based Traffic Processing and Service Inference for High-Performance Networks

Rice University

PI: Richard Baraniuk

Department of Electrical Engineering and Computer Science

6100 Main Street, Houston, TX 77005

Tel: 713-348-5132, Email: richb@rice.edu

 

Los Alamos National Laboratory

PI: Wu-chun Feng

Los Alamos National Laboratory

P. O. Box 1663, MS D451

Los Alamos, New Mexico 87545

Tel: 505-665-2730, Email: feng@lanl.gov

 

 

Stanford Linear Accelerator Laboratory

PI: Les Cottrell

Standford Linear Accelerator Center (SLAC)

Mail Sop 97, Box 4349

Stanford, CA 94309

Tel: 650-926-2523, Email: cottrell@stanford.edu

 

Executive summary:

This proposal is a collaboration between Rice University, Los Alamos, and SLAC. Led by Rice University. The explosive growth of high-speed computer networks, combined with rapid and unpredictable developments in applications and workloads, has rendered network modeling, control, and performance prediction increasingly demanding tasks. Critical high-end applications such as remote visualization and high-capacity data transfers routinely fail to meet end-to-end performance expectations when deployed on high-speed networks. Optimizing the performance of these and other demanding applications requires that end-systems have knowledge of the internal network traffic conditions and services.

Without special-purpose network support, the only alternative is to indirectly infer dynamic network characteristics from edge-based network measurements. Furthermore, the complexity of network dynamics demands more advanced mathematical theory and methods in order to develop scalable and accurate algorithms in support of high-performance computing infrastructures, such as computational grids. The INCITE (InterNet Control and Inference Tools at the Edge) Project focuses experts from the fields of networking, supercomputing, statistical signal processing, and applied mathematics towards the goal of analyzing, modeling, and characterizing high-speed network services based solely on edge-based measurement at hosts and/or edge routers. This project develops an innovative framework for capturing the complex dynamics and controlling the performance of networks driven by real applications. Specifically, we aim to develop on-line tools to characterize and map network performance as a function of space, time, application, protocol, and service. This understanding will allow us to devise low-complexity models and measurement methodologies amenable to optimized on-line control and management. Our effort consists of four closely inter-related research thrusts that directly address the key challenges facing the DOE high-performance networks program (Network measurement and analysis, High-performance transport protocols, and Advanced traffic engineering tools and services).

Scheduled Milestones - Richard Baraniuk- Rice University

Rice will produce four classes of deliverables: (1) new theory and technologies, (2) software simulations of these technologies, (3) experimental testing and reference implementations, and (4) publications in the form of talks, technical reports, and papers. In particular, we expect to produce the following:

    1. Theory for Multifractal Network Traffic Analysis: Novel modeling of network traffic; investigating origins of multifractality; identifying candidate traffic sufficient statistics.

    2. Model-based Traffic Processing and Inference Methods: Online processing and inference algorithms that are accurate, robust, efficiently computed, and suitable for real-time implementation.

    3. Traffic Synthesis Techniques: Rapid traffic synthesis algorithms based on multifractal models.

    4. Novel Theory and Methods for Network Tomography: Probabilistic factor graph modeling and analysis of networks; tomographic inference algorithms from block-based network measurements.

    5. Real-Time Network Tomography: Sequential learning and inference strategies that track internal network parameters; network model selection methods for inferring/tracking unknown network topologies.

    1. Multifractal Analysis Toolbox: Matlab software tools for traffic modeling and processing; online implementations for multifractal analysis and inference.

    2. Network Modeling and Tomography Simulator: Matlab software tools for network tomography under arbitrary network configurations; sequential software tools for dynamic network simulation and tomography.

    3. Network Tomography Real-time Inference Engine: Software platform for online network tomography based on sequential algorithms.

    1. Multifractal Model Validation: Offline (trace-driven) and online testing of multifractal traffic modeling and inference.

    2. Network Tomography Validation: Simulation and measurement-based experimentation with network tomography and topology inference software.

    3. Prototyping: Validation and testing of integrated prototype network software system with network data provided by collaborators.

We will provide software implementations of the above technologies in a public domain simulator (ns). For the traffic generation and network inference modules, we will build prototype implementations using standard networking APIs (Berkeley sockets).

Milestones

Year 1:

Year 2:

Year 3:

Integration of Rice, LANL, and SLAC results into one coherent system.

Scheduled Milestones - Los Alamos National Laboratory

  1. Year 1: Research, development, and limited deployment of an alpha-tested MAGNeT.
  2. Year 1: Testing and performance evaluation of MAGNeT to ensure that it does not adversely perturb the stream of application-generated network traffic and that it does not drop packets.
  3. Year 1: Kernel modification to tcpdump to enable higher-fidelity network monitoring and measurement.
  4. Year 1: Monitoring, measurement, and collection of network-level traffic using our higher-fidelity tcpdump and/or PingER (depending on the actual network infrastructure that we will be monitoring, that is, the continually evolving LANL network backbone, which is currently FDDI but will soon be Gigabit Ethernet.
  5. Year 1: Deployment of our higher-fidelity tcpdump.
  6. Year 2: Beta-tested MAGNeT for wider deployment along with substantial wrapper scripts to enable the collection of traffic traces from distributed workstations and PCs.
  7. Year 2: Collection and analysis of application-generated network traffic via MAGNeT at LANL, and possibly at Rice and/or SLAC.
  8. Year 2: Analysis and characterization of network-level traffic measurements using the coefficient-of-variation model, the second-order self-similar model (fBm), and/or the multifractal models developed by our Rice collaborators that will be incorporated into PingER (using the traffic measurements collected in Year 1).
  9. Year 2: Research & develop a stochastic model for TCP Reno based on the above application-generated and network-level traffic measurements.
  10. Year 3: Electronic distribution of application-generated network traffic traces for use by the network-research community. These real traces can then be used to test new protocols or protocol enhancements.
  11. Year 3: Further collection and analysis of application-generated network traffic.
  12. Year 3: Final software-distribution release of MAGNeT.
  13. Year 3: Analysis and characterization of the interactions between TCP and active queue-management routers such as RED and BLUE.
  14. Year 3: Integration of the aforementioned characterization of TCP and active queue management into our stochastic model.

 

 

Scheduled Milestones - Stanford Linear Accelerator Center

  1. Year 1: Install, configure and customize PingER monitoring tools at LANL & Rice. Incorporate new monitoring sites into databases, update web documentation and activate the PingER data gathering.
  2. Year 1: Install/configure INCITE chirp multi-fractal measurement client/server tools on SLAC, LANL and Rice PingER monitor hosts. Extend PingER monitoring and data gathering tools to also utilize chirp tools to make multi-fractal measurements and to gather the chirp data.
  3. Year 1: Design and enhance the PingER analysis tools to analyze the chirp data and integrate presenting the results via the web into the existing PingER analysis/web reporting structure.
  4. Year 1: Work with John MacAllister (the author) to extend traceping to run on Unix and NT, and to archive the RTT measurements. Deploy new version at SLAC, evaluate and improve.
  5. Year 1: Install and configure traceping measurement probes at LANL and Rice.
  6. Year 1: Work with PSC to come up with a method and support to install NIMI probes at Rice and LANL.
  7. Year 2: Electronic distribution of chirp data from PingER.
  8. Year 2: Design and implement a client only chirp measurement tool that uses ICMP echo.
  9. Year 2: Integrate new client only chirp measurement tool into PingER starting with SLAC, and then with Rice and LANL. Integrate the collection, analysis and reporting of the data from the client only chirp measurement tool into the PingER infrastructure.
  10. Year 2: Deploy client only chirp measurement tools to PingER monitoring hosts managed by SLAC, and following this discuss and arrange deployment to other high performance PingER sites that are interested.
  11. Year 2: Install NIMIs at Rice and LANL.
  12. Year 2: Investigate and develop installing the INCITE chirp tools into NIMI. Deploy at SLAC, Rice and LANL.
  13. Year 2: Package and compress the traceping data and make available to Rice researchers for network tomography analysis.
  14. Year 3: Final software distribution of client only chirp tool.
  15. Year 3: Test, modify, make more robust and validate the NIMI chirp measurements, and install chirp tools in NIMIs at LANL and Rice.
  16. Year 3: Work with NIMI developers to decide whether to build chirp tool into stndard distribution.
  17. Year 3: Electronic distribution of chirp data from NIMI for more detailed multi-fractal etc. analysis and validation by experts at Rice and elsewhere.