OOI data team members have developed a new toolbox for downloading data from the Machine-to-Machine (M2M) interface using MATLAB, making data access as easy as 1, 2, 3. The work expands on the existing tools available on OOI. This new toolbox covers data from moorings, gliders, and profilers in the OOI Coastal Arrays and can easily be extended to other research platforms. A video tutorial on how to use this new tool is provided above. (A pdf of the tutorial is provided below).
In three short steps – define the mooring, node, and instrument – users can easily and quickly access relevant OOI data. “This new approach eliminates some steps and makes it easy for anyone to find and access data they are looking for, “said Jonathan Fram, OOI Endurance Array Project Manager at Oregon State University (OSU). “We found this new approach really streamlined data access using the M2M interface and we are certain other OOI data users will find it similarly helpful.”
Developed by OOI OSU team members, Craig Risien and Russ Desiderio, these new tools are part of our ongoing effort to simplify access to OOI data to encourage its broader integration into scientific research and classroom content. A tutorial showing how to use MATLAB to access OOI data can be viewed here.
Other members of the OOI OSU team, Chris Wingard and Ian Black, are testing Python and R tools to explore OOI data, as well. Once the tools have been tested and vetted, they will be available along with other community-generated tools here.
This new tutorial adds to others that have been developed to help users integrate OOI data into their science.Read More
OOI’s data teams have just completed an extensive, year-long review of critical metadata to ensure the quality and usability of data for OOI data users. The review covered data collected through the end of 2019 and included instrument calibration coefficients, instrument deployment assignments, and deployment dates. Moving forward, all metadata verification will conform to the standards established during the review.
“Our reason for undertaking this review was no more complicated than to make the data better for our data users,” explains Jeffrey Glatstein, Senior Manager of Cyberinfrastructure and OOI Data Delivery Lead. “It is the first time since the inception of the program that we’ve really gone in and looked at the metadata from top to bottom. If there was a calibration that was off, a depth missing, or something misspelled, we found it.
“This intense and deliberative review process brought historic metadata up to current standards to ensure continuity, completeness of records, and consistency in how metadata are reported now and moving forward.”
The data teams used a combination of human review and an automated script development process to identify and correct data issues. The human-in-the-loop (HITL) process ensured that two sets of eyes verified each metadata product, whenever possible, while the scripts performed automated verification and generated reports to pass back into the HITL workflow.
“This initiative is part of ongoing OOI efforts to make its data more accessible, user friendly, and integrated into ongoing science,” adds Glatstein.
Check Previously Downloaded Data
The OOI Data Portal operates on a process-on-demand model, which means that data downloaded prior to the end of 2019 should be checked to see if relevant metadata has been modified.
Users can check to see if changes were made to relevant metadata by clicking here. This link provides a searchable database by array, platform, and instrument to help ensure that previously downloaded data are correct or if they need to be re-downloaded so users are working with the best available data. The OOI data teams are continuing to verify the historical deployment assignments/dates, and the results will be updated accordingly[feature]
A Gargantuan Effort
As part of the transition of OOI to 2.0 in October 2018, the RCA data team initiated a comprehensive audit of all critical metadata to ensure that data products served by the OOI Cyberinfrastructure system meet Quality Assurance/Quality Control standards set by the program and expected from the user community. This daunting task included the examination of over 700 calibration files from 2013 to the present. The results of this audit were used to aid in evaluation of current processes and guide in adapting workflows to improve QA/QC efforts and communication to the users, a vital component to building confidence in the OOI datasets as reliable and valuable resources that can be used in scientific research and education.
Wendi Ruef, Research Scientist, Regional Cabled Array
The CGSN Data Team worked carefully and methodically through thousands of files containing over 30,000 calibration coefficients and other critical metadata. We now have a high level of confidence in past metadata and a strong process for continued review going forward.
Al Plueddemann, Chief Scientist, Coastal Global Scale Nodes
[/feature] Read More
On Thursday, September 27th at 1938 PT, one of two chillers supporting power feed equipment at the Regional Cabled Array Shore Station had a fault which resulted in an automatic protective shutdown of the high voltage power supply and loss of power to both submarine cable lines. Power was restored to the system on Tuesday, October 2, 2018 at 1115 PT. Due to loss of power to the Cabled Array marine infrastructure, data were not collected by the instruments during this outage period.Read More
On Friday 8 December 2017 at 1:30 PM ET an error occurred that prevented the OOI Cyberinfrastructure (CI) system from loading new live data. On Monday 11 December 2017 the issue was fixed; no data were lost. We are sorry for any inconvenience.
If you have any questions, please contact the HelpDesk.Read More
A data anomaly came to our attention during the Portland workshop last month regarding some of the OOI oxygen data.Read More