NSF III-1656958: CRII: III: Scalable and Interactive Dependency Visualization to Accelerate Parallel Program Analysis
Large-scale parallel programs are used to simulate and study weather patterns,
vehicle design, combustion and energy. However, developing these and
optimizing these programs to be more accurate and efficient is difficult.
Program analysts often investigate the dependencies within the
programs. The number of dependencies in such programs is vast, taxing visual
tools representing them. The goals of this project are developing visualization
approaches to overcome these challenges as well as design guidelines for
related large and complex data.
Current Results:
- A domain-specific layout approach for control flow dependency visualization which improves interperatability.
- While our visualization supported direct manipulation to tweak layout, we found researchers no longer prioritized that functionality once this layout was in place. Instead, they wanted to preserve the layout.
- This layout is used in the stand-alone CFG drawing library, CFGConf.
- This layout has served as the basis for tools beyond this project, such as CCNav.
- An open-source interactive visualization tool for control flow dependencies and dyanmic traces.
- A goal and task analysis for visualizing control flow graphs in computing research.
- We found node-link visualizations were used first to debug the researchers' own mental models in anticipation of developing new algorithms and later in debugging their research in progress.
- Presenting and explaining their work to collaborators was another major goal in generating the visualization.
- A set of designs for scalably representing communication patterns in large-scale parallel traces.
- In a pre-registered, user study, we found these designs increased identification accuracy for common patterns.
- Guidelines for conducting design studies in rapidly moving projects such as the software and research concerns targeted by this project.
- An open-source interactive visualization tool for expression trees from an asynchronous task-based parallel runtime system.
- A library for integrating Javascript utilities with Jupyter Notebook, to pass data between in-cell Javascript visualizations with other cells in the notebook.
- A framework and integrated system for visualizing HPC performance data, included the expression trees in Atria and responsive Gantt charts for large-scale traces.
Impact:
- CFGExplorer is available publicly and has been used by a team of researchers at the University of Arizona for program analysis and optimization of scientific software and a team of researchers at Louisiana State University for loop parallelization. The layout has been incorporated in a stand-alone library, CFGConf, and adapted in a tool from Lawrence Livermore National Laboratory, CCNav.
- Roundtrip is available publicly and has been used in Jupyter notebooks at Louisiana State University, the National Renewable Energy Laboratory, and Lawrence Livermore National Laboratory.
- Atria is available publicly and has been used for performance tuning in an asynchronous task-based parallel runtime.
- Traveler is available publicly and is being used used for performancne inspection of large traces and asynchronous task-based parallel runtimes.
This material is based upon work supported by the National Science Foundation under Grant No. 1656958.
News
- 2021/07 - Looking for a stand-alone version of the CFGExplorer layout? Check out CFGConf!
- 2021/06 - Learn more about scaling communication trace visualization in our ArXiv report, Design and Evaluation of Scalable Representations of Communication in Gantt Charts for Large-scale Execution Traces!
- 2020/11 - See our talk on futher development of Jetlag, "Distributed Asynchronous Array Computing with the JetLag Environment", 4:10 PM EST on November 13th at PyHPC!
- 2020/07 - See our talk on "JetLag: An Interactive, Asynchronous Array Computing Environment," Noon Pacific Time on July 30th at PEARC 20!
- 2020/01 - We are running a study at Hack Arizona! Come by the first floor of the Gould-Simpson Building to participate!
- 2019/11 - We arre running a study at SC19! Come by the Arizona Research Computing booth to participate!
- 2019/10 - Check out Visualizing a Moving Target, a design study conducted in the presence of changing data and concerns, supported in part by this project and accepted in IEEE Transactions on Visualization and Computer Graphics, proceedings of InfoVis.
- 2019/08 - Welcome new PhD students Sayef Sakin and Connor Scully-Allision
- 2019/06 - Our design study on execution graphs and evolving concerns in computing research supported in part by this project.
- 2018/06 - Welcome undergraduate researcher Sarah Jacquot!
- 2018/06 - CFGExplorer presented at EuroVis 2018.
- 2017/11 - We are running a study at SC17! Come by the Arizona Research Computing Booth (Booth 887) to participate or to demo CFGExplorer!
- 2017/08 - Welcome to the project PhD student Sabin Devkota and new PhD Student Katy Williams!
Publications
C. Scully-Allison, K. E. Isaacs. “Design and Evaluation of Scalable
Representations of Communication in Gantt Charts for Large-scale Execution
Traces.” ArXiv:2107.00065v1, June 2021.
S. R. Brandt, B. Hasheminezhad, N. Wu, S. A. Sakin, A. R. Bigelow, K. E.
Isaacs, K. Huck, H. Kaiser. Distributed Asynchronous Array
Computing with the Jetlag Environment. Proceedings of the 9th Workshop on
Python for High-Perforrmance and Scientific Computing (PyHPC). November
2020.
S. Brink, I. Lumsden, C. Scully-Allison, K. Williams, O. Pearce, T.
Gamblin, M. Taufer, K. E. Isaacs, and A. Bhatele. Usability and
Performance Improvements in Hatchet. Proceedings of the 2020 IEEE/ACM
International Workshop on HPC User Support Tools (HUST) and Workshop on
Programming and Performance Visualization Tools (ProTools), held
in conjunction with SC20, November 2020.
S. R. Brandt, A. Bigelow, S. Sakin, K. Williams, K. Isaacs, K. Huck, R. Tohid, B. Wagle, S. Shirzad, and H. Kaiser. JetLag: An Interactive, Asynchronous Array Computing Environment. To appear in Proceedings of the 2020 Practice and Experience in Advanced Computing Research (PEARC) Conference. July 2020.
DOI
K. Williams, A. Bigelow, and K. Isaacs<.
Visualizing a Moving Target: A Design Study on Task Parallel Programs in the Presence of Evolving Data and Concerns.
IEEE
Transactions on Visualization and Computer Graphics, Proceedings of InfoVis '19. January 2020.
PDF (Arxiv) |
Code
S. Devkota and K. E. Isaacs. CFGExplorer: Designing a Visual Control
Flow Analytics System around Basic Program Analysis Operations.
Computer Graphics Forum (Proceedings of EuroVis). July 2018.
PDF |
Code
Presentations
S. R. Brandt. Distributed Asynchronous Array Computing with the JetLag Environment PyHPC 2020 (Online for Global Pandemic), November 13, 2020.
S. R. Brandt. JetLag: An Interactive, Asynchronous Array Computing Environment PEARC 2020 (Online for Global Pandemic), July 30, 2020.
K. Isaacs. Some Other Sides of Visualization in Support of Data Science. TRIPODS Seminar. University of Arizona, Tucson, AZ, USA. November 4, 2019.
K. Williams. Visualizing a Moving Target: A Design Study on Task Parallel Programs in the Presence of Evolving Data and Concerns. Paper Presentation. IEEE VIS, Vancouver, Canada. October 24, 2019.
S. Jacquot. A User Study on Visualizing Gantt Charts. Oral
Presentation. National McNair Scholars Conference. Los Angeles, CA, USA. July
26, 2018.
S. Devkota and K. E. Isaacs. CFGExplorer: Designing a Visual Control
Flow Analytics System around Basic Program Analysis Operations. Paper Presentation. EuroVis
2018, Brno, Czech Republic. June 7, 2018.
Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.
