Photo of river with lush trees

CyberWater

Building a Collaboration Infrastructure: CyberWater2–A Sustainable Data/Model Integration Framework

CyberWater2 develops a collaboration-centric cyberinfrastructure for scientific and engineering communities to solve complex scientific modeling problems efficiently, accurately and in-depth and to facilitate collaboration across disciplines, platforms, organizations, and geographic boundaries. It will be built on top of CyberWater, and significantly boost it to a new level of capability and effectiveness for broader collaborations.

CyberWater2

Intellectual Merits of CyberWater2 include:

  • Enabling two-way open model coupling across platforms with new task-based and in-situ hybrid workflow mechanism, with little or no coding

  • Enabling easy and systematic model parameter calibration and data assimilation without coding

  • Allowing easy access to the CyberWater2 system via browsers to construct modeling workflows that will be downloaded and run on users' local machines supporting scalability
    Capability to adapt to any future changes of external data sources via automated data agent updating

  • Capacity to support automated heterogeneous resource planning with an intelligent HPC site recommender

A flow diagram illustrating Cyberwater framework as it relates to Users developed model agents, data agents, toolkits, and integration engines. The diagram is titled “Workflow Management System (WMS)” with the subtitle “Workflow Engine”.

CyberWater Architecture Diagram

A diagram divided into 3 vertical section labeled Data, Models, Investigations from left to right. The data Column contains 4 stacked boxes that contain the text: “Dynamic information in map form (e.g. forcings, satellite/radar measurements, climate model simulation results, etc.)”, “Dynamic information in vector form (e.g. point measurements from gauges for precipitation and streamflow, etc.)” “Static information in map form (e.g. elevation, river network, soil map, etc.)” and “Static information in vector form (e.g., model configuration files, model global parameter files, etc.)”. An arrow points to the second column titled models, which contains a diagram describing inputs and outputs for model calibration, leading to model parameter calibration algorithms. At the base of the column is a box containing the text “Updating model state variables and/or parameters via data assimilation based on newly available observed information (Platform: HPCs/Clusters/PCs)”. The final column is titled investigations and contains 5 boxes stacked. From top to bottom the boxes read “Coastal Flood Forecasts” , “Landslide risk assessment”, “Drought predictions”, “Model evaluations, intercomparisons”, “Model couplings with different combinations”.

CyberWater2 Use Case Diagram

The CyberWater project creates a new cyberinfrastructure with an open data, open modeling framework software.

The goal of this project is to reduce the user time and effort needed for hydrologic modeling studies, thereby expediting fundamental knowledge discoveries. In addition to facilitating diverse data and model integration while ensuring reproducible computing, CyberWater also enables on-demand access to high-performance computing facilities.

Project Description:

Scientists need to use diverse data and integrate models outside their own disciplines with sufficient model accuracy and predictability. This is currently difficult to accomplish because (1) a vast quantity of diverse data are not readily accessible to models; and (2) diverse models developed individually by different research groups are difficult to share an integrate between disciplines.

The goal of this project is to build an open data, open modeling framework software that enables easy and incremental integration of diverse data and models for knowledge discovery and interdisciplinary team-work, as well as reproducible computing and seamless and on-demand access to HPC resources. Our project team includes hydrologists, climate experts, meteorologists, computer scientists and CI experts, from multiple universities and CUAHSI, who collaborate closely to ensure CyberWater will engage the broad communities for domain scientists' benefits.

CyberWater can lower the learning curve for modelers with limited skills in computer and technology, and enable them to quickly employ models from other domains. It advances the state-of-the-art by making it possible to quickly, effectively and reliably connect diverse models together to form a comprehensive integrated modeling framework for tackling emerging complex interdisciplinary problems using diverse sources of data.

Currently, a large interdisciplinary team must be formed to modify model code to couple the diverse domain models together. The innovations of CyberWater include:

  1. Significantly reducing model integration complexity, and allowing easy coupling of diverse models without coding.
  2. Facilitating the use of diverse data by automatically ingesting data of heterogeneous types, formats, and access protocols, etc.
  3. Automating the flow from data to model to output visualization in real- or near real-time
  4. Supporting reproducible computing
  5. Enabling high power computing on demand
  • Scientists can use the CyberWater framework to greatly simplify their data access, coupled model simulations, reproducible computing, and access to HPC facilities, all of which significantly enhance research productivity.
  • CyberWater will be a valuable education tool to teach and train university students on how to conduct interdisciplinary and reproducible research, thus greatly enhancing university STEM education capability for future workforce.
  1. Salas, D., Liang, X., Navarro, M., Liang, Y., and Luna, D. An open-data open-model framework for hydrological models’ integration, evaluation and application. Environmental Modelling and Software, 126 (April 2020), 104622. https://doi.org/10.1016/j.envsoft.2020.104622
  2. Luna, D., Chen, R., Yuan, C., Liang, Y., Liang, X., Bales, J., Castronova, A.M., Demir, I., Hooper, R.P., Krajewski, W.F. and Lin, L., 2019. CyberWater—An open and sustainable framework for diverse data and model integration. AGU Fall Meeting, 2019, pp.IN11B-02.
  3. Liang, X., Liang, Y., Luna, D., Chen, R., Cao, Y., Fu, Y., Pamidighantam, S.,Song, F., Bales, J., Castronova, A., Demir, I., Hooper, R., Krajewski, W., Lin, L., Mantilla, R., Zhang, Y., 2020. Collaborative Research: CyberWater - An open and sustainable framework for diverse data and model integration with provenance and access to HPC.
  4. Chen, R., D. Luna, Y. Cao, Y. Liang, and X. Liang, Open data and model integration through generic model agent toolkit in CyberWater framework, Environmental Modelling and Software, 152, 105384, https://doi.org/10.1016/j.envsoft.2022.105384, 1-16, 2022.
  5. Chen, R., D. Luna, F. Li, R. Young, D. Bieger, F. Song, S. Pamidighantam, Y. Liang and X. Liang, 2022, CyberWater: An Open Framework for Data and Model Integration in Water Science and Engineering. In Proceedings of the 31st ACM Int’l Conference on Information and Knowledge Management (CIKM’22), Oct. 17-21, 2022, Atlanta, GA. ACM, New York, NY, USA, 5 pages. https://doi.org/10.1145/ 3511808.3557186.
  6. Li, F., Chen, R., Fu, Y., Song, F., Liang, Y., Ranawaka, I., Pamidighantam, S., Luna, D., & Liang, X., Accelerating complex modeling workflows in CyberWater using on-demand HPC/Cloud resources. The 17th IEEE eScience Conference, 196–205, DOI:https://doi.org/10.1109/eScience51609.2021.00030, 2021.
  7. Luna, D., R. Chen, R. Young, Y. Liang, and X. Liang, Towards flexible, rich, and easy model coupling environments via CyberWater’s open data and open model framework, American Geophysical Union Fall Meeting (Hybrid), Chicago, IL, Dec. 12-16, 2022. Website at: https://agu2022fallmeeting-agu.ipostersessions.com/default.aspx?s=DF-BE-CC-07-14-C5-1F-01-F2-24-10-BB-4A-64-04-8C
  8. Chen, R., D. Luna, A. Castronova, X. Liang, and Y. Liang, Composable National Water Model simulations using the CyberWater framework, American Geophysical Union Fall Meeting (Hybrid), Chicago, IL, Dec. 12-16, 2022.

Please contact Xu Liang (email hidden; JavaScript is required) or Deanna McCay (email hidden; JavaScript is required) if you want to be involved or have any questions about the project.

Contributors

  • Xu Liang (University of Pittsburgh), Project lead PI
  • Yao Liang (Indiana University - Purdue University Indianapolis), IUPUI PI
  • Lan Lin (Ball State University), PI
  • Lai-yung Ruby Leung (Pacific Northwest National Laboratory), Collaborator
  • Jeen-Shang Lin (University of Pittsburgh), Collaborator
  • Deanna H. McCay (CUAHSI), Collaborator
  • Robert Quick (Indiana University), Collaborator
  • Fengguang Song (Indiana University - Purdue University Indianapolis), Collaborator
image description

Join our mailing list!

Keep up with CUAHSI news and events

Subscribe