Building a Collaboration Infrastructure: CyberWater2–A Sustainable Data/Model Integration Framework
CyberWater2 develops a collaboration-centric cyberinfrastructure for scientific and engineering communities to solve complex scientific modeling problems efficiently, accurately and in-depth and to facilitate collaboration across disciplines, platforms, organizations, and geographic boundaries. It will be built on top of CyberWater, and significantly boost it to a new level of capability and effectiveness for broader collaborations.
Intellectual Merits of CyberWater2 include:
Enabling two-way open model coupling across platforms with new task-based and in-situ hybrid workflow mechanism, with little or no coding
Enabling easy and systematic model parameter calibration and data assimilation without coding
Allowing easy access to the CyberWater2 system via browsers to construct modeling workflows that will be downloaded and run on users' local machines supporting scalability
Capability to adapt to any future changes of external data sources via automated data agent updating
Capacity to support automated heterogeneous resource planning with an intelligent HPC site recommender
The CyberWater project creates a new cyberinfrastructure with an open data, open modeling framework software.
The goal of this project is to reduce the user time and effort needed for hydrologic modeling studies, thereby expediting fundamental knowledge discoveries. In addition to facilitating diverse data and model integration while ensuring reproducible computing, CyberWater also enables on-demand access to high-performance computing facilities.
Scientists need to use diverse data and integrate models outside their own disciplines with sufficient model accuracy and predictability. This is currently difficult to accomplish because (1) a vast quantity of diverse data are not readily accessible to models; and (2) diverse models developed individually by different research groups are difficult to share an integrate between disciplines.
The goal of this project is to build an open data, open modeling framework software that enables easy and incremental integration of diverse data and models for knowledge discovery and interdisciplinary team-work, as well as reproducible computing and seamless and on-demand access to HPC resources. Our project team includes hydrologists, climate experts, meteorologists, computer scientists and CI experts, from multiple universities and CUAHSI, who collaborate closely to ensure CyberWater will engage the broad communities for domain scientists' benefits.
CyberWater can lower the learning curve for modelers with limited skills in computer and technology, and enable them to quickly employ models from other domains. It advances the state-of-the-art by making it possible to quickly, effectively and reliably connect diverse models together to form a comprehensive integrated modeling framework for tackling emerging complex interdisciplinary problems using diverse sources of data.
Currently, a large interdisciplinary team must be formed to modify model code to couple the diverse domain models together. The innovations of CyberWater include:
- Significantly reducing model integration complexity, and allowing easy coupling of diverse models without coding.
- Facilitating the use of diverse data by automatically ingesting data of heterogeneous types, formats, and access protocols, etc.
- Automating the flow from data to model to output visualization in real- or near real-time
- Supporting reproducible computing
- Enabling high power computing on demand
- Scientists can use the CyberWater framework to greatly simplify their data access, coupled model simulations, reproducible computing, and access to HPC facilities, all of which significantly enhance research productivity.
- CyberWater will be a valuable education tool to teach and train university students on how to conduct interdisciplinary and reproducible research, thus greatly enhancing university STEM education capability for future workforce.
- Salas, D., Liang, X., Navarro, M., Liang, Y., and Luna, D. An open-data open-model framework for hydrological models’ integration, evaluation and application. Environmental Modelling and Software, 126 (April 2020), 104622. https://doi.org/10.1016/j.envsoft.2020.104622
- Luna, D., Chen, R., Yuan, C., Liang, Y., Liang, X., Bales, J., Castronova, A.M., Demir, I., Hooper, R.P., Krajewski, W.F. and Lin, L., 2019. CyberWater—An open and sustainable framework for diverse data and model integration. AGU Fall Meeting, 2019, pp.IN11B-02.
- Liang, X., Liang, Y., Luna, D., Chen, R., Cao, Y., Fu, Y., Pamidighantam, S.,Song, F., Bales, J., Castronova, A., Demir, I., Hooper, R., Krajewski, W., Lin, L., Mantilla, R., Zhang, Y., 2020. Collaborative Research: CyberWater - An open and sustainable framework for diverse data and model integration with provenance and access to HPC.
- Chen, R., D. Luna, Y. Cao, Y. Liang, and X. Liang, Open data and model integration through generic model agent toolkit in CyberWater framework, Environmental Modelling and Software, 152, 105384, https://doi.org/10.1016/j.envsoft.2022.105384, 1-16, 2022.
- Chen, R., D. Luna, F. Li, R. Young, D. Bieger, F. Song, S. Pamidighantam, Y. Liang and X. Liang, 2022, CyberWater: An Open Framework for Data and Model Integration in Water Science and Engineering. In Proceedings of the 31st ACM Int’l Conference on Information and Knowledge Management (CIKM’22), Oct. 17-21, 2022, Atlanta, GA. ACM, New York, NY, USA, 5 pages. https://doi.org/10.1145/ 3511808.3557186.
- Li, F., Chen, R., Fu, Y., Song, F., Liang, Y., Ranawaka, I., Pamidighantam, S., Luna, D., & Liang, X., Accelerating complex modeling workflows in CyberWater using on-demand HPC/Cloud resources. The 17th IEEE eScience Conference, 196–205, DOI:https://doi.org/10.1109/eScience51609.2021.00030, 2021.
- Luna, D., R. Chen, R. Young, Y. Liang, and X. Liang, Towards flexible, rich, and easy model coupling environments via CyberWater’s open data and open model framework, American Geophysical Union Fall Meeting (Hybrid), Chicago, IL, Dec. 12-16, 2022. Website at: https://agu2022fallmeeting-agu.ipostersessions.com/default.aspx?s=DF-BE-CC-07-14-C5-1F-01-F2-24-10-BB-4A-64-04-8C
- Chen, R., D. Luna, A. Castronova, X. Liang, and Y. Liang, Composable National Water Model simulations using the CyberWater framework, American Geophysical Union Fall Meeting (Hybrid), Chicago, IL, Dec. 12-16, 2022.
Watch the videos below to learn more about our project. These lectures were a part of our 2021 virtual cyberseminar.
- CyberWater 2021 Workshop Welcome and Overview
- Overview of the CyberWater Project, Platform and Features
- The CyberWater Interface and Modeling Workflow
- Using the Generic Model Agent to Port Models into CyberWater
- Leveraging High Performance Computing in CyberWater
- CyberWater’s Geographic Information System Capabilities
- Overview of 2nd User Testing Workshop - August 2021
- Xu Liang (University of Pittsburgh), Project lead PI
- Yao Liang (Indiana University - Purdue University Indianapolis), IUPUI PI
- Lan Lin (Ball State University), PI
- Lai-yung Ruby Leung (Pacific Northwest National Laboratory), Collaborator
- Jeen-Shang Lin (University of Pittsburgh), Collaborator
- Deanna H. McCay (CUAHSI), Collaborator
- Robert Quick (Indiana University), Collaborator
- Fengguang Song (Indiana University - Purdue University Indianapolis), Collaborator