Data Store¶
CyVerse Data Store runs the Integrated Rule-Oriented Data System (iRODS) Open Source Data Management Software.
iRODS helps researchers, archivists, and others manage large, geographically dispersed computer files by providing a virtual filesystem, metadata catalog, and a rule engine to automate data management and enforce policies.
The Data Store is designed for storing, managing, and sharing your data throughout its entire lifecycle.
Integrated across all CyVerse platforms, the Data Store has high accessibility and connectivity.
The Data Store's features are aimed at helping you maintain data integrity and value, while making your data more FAIR (Findable, Accessible, Interoperable, and Reusable) with minimal effort.
This guide will walk you through the essential steps to get started, assuming you’ve already created a CyVerse account.
Data Management¶
There are several ways to access the Data Store. These methods vary in speed, flexibility, and technical knowledge required. Different methods may suit your needs for different projects at different times.
| Method | Access Point | OS | Upload/Download | Installation/Setup Required | Account Required | Max File Size |
|---|---|---|---|---|---|---|
| Discovery Environment | Web | Any | Both | No | Yes | 2GB/file upload, no limit for import |
| WebDAV | Web & Command line | Any | Both | No | Yes (No for public data) | No limit |
| GoCommands | Command line | Any | Both | Yes | Yes (No for public data) | No limit |
| iCommands | Command line | Linux & macOS | Both | Yes | Yes (No for public data) | No limit |
| SFTP | Desktop App & Command line | Any | Both | No (Yes for desktop app) | Yes (No for public data) | No limit |
This section covers each of the following data management methods:
- Discovery Environment: A comprehensive web-based platform for data analysis and management
- GoCommands: A lightweight, portable command-line tool for efficient data operations on any OS
- iCommands: A powerful command-line suite for advanced data management tasks on Linux
- SFTP: A secure file transfer protocol accessible via command-line or GUI applications on any OS
- WebDAV: A protocol extending HTTP for collaborative file management over the internet, usable on any OS
Additional resources for managing your data and team collaboration:
- Getting a DOI: Obtain a Digital Object Identifier (DOI) for a permanent and stable link to your data
- Checking Data Usage: Monitor your data usage and storage limits
- Team Access Management: Create and manage teams in the Data Store for collaborative work
Manage Your Data in Discovery Environment¶
The Discovery Environment has a fully managed interface for uploading, downloading, sharing, and adding metadata to the Data Store through your browser.
Manage Your Data with GoCommands¶
GoCommands is a lightweight and portable command-line tool designed for efficient data management within the Data Store. This section provides guidance on how to set up and use GoCommands to interact with the Data Store.
Manage Your Data with iCommands¶
iCommands is a powerful command-line tool designed for data management in iRODS. It enables you to manage data in the Data Store efficiently via the command line. This section covers how to install and use iCommands for streamlined data transfers and management.
Transferring Data with SFTP¶
SFTP (Secure File Transfer Protocol) is a widely adopted network protocol for secure file access, transfer, and management. This section explains how to use SFTP to move your data to and from the Data Store.
HTTP Access with WebDAV¶
WebDAV (Web Distributed Authoring and Versioning) is a protocol built on top of HTTP/HTTPS, allowing users to manage data on web servers. This section demonstrates how to use WebDAV for seamless data access and management within the Data Store.
Getting a DOI¶
A Digital Object Identifier (DOI) offers a permanent and stable link to your data. Learn how to obtain a DOI for your datasets stored in the Data Store, ensuring long-term citation and accessibility.
Checking Data Storage¶
Keep track of your data usage and storage limits. This section covers tools and techniques for monitoring your data storage within the Data Store.
Managing Data within a Team¶
Efficiently manage team access and permissions for your data. Learn how to create and manage teams in the Data Store to facilitate collaborative data work.
