A data lake is a centralized repository that allows you to store vast amounts of structured, semi-structured, and unstructured data in its raw format. Unlike traditional data warehouses where data is stored in a structured manner, a data lake retains the data in its native format until it's needed for analysis or processing. ONES provide the capability to store the RAW data of all the Metrics to Cloud and then user will be able to use that RAW data for any deployment or any other use cases.
Storage of Diverse Data Types: A data lake can store various types of data, including structured data (like relational databases), semi-structured data (like JSON, XML), and unstructured data (like documents, images, videos). This flexibility allows organizations to ingest and store data from different sources without the need for extensive preprocessing.
Scalable and Cost-Effective Storage: Data lakes are typically built on scalable storage systems, such as cloud-based object storage (e.g., Amazon S3, Azure Data Lake Storage) or Splunk . These systems can efficiently handle large volumes of data and offer cost-effective storage solutions.
Schema-on-Read Approach: In contrast to traditional data warehouses that use a schema-on-write approach (where data must be structured and conform to a predefined schema before storage), data lakes adopt a schema-on-read approach. This means that data is stored in its original form, and the schema is applied at the time of data retrieval or analysis. This flexibility allows users to apply different schemas and interpretations to the same dataset based on their analytical needs.
Support for Big Data Processing and Analytics: Data lakes serve as a foundational component for big data analytics and processing. Users can perform various analytics tasks, including exploratory data analysis, data mining, machine learning, and real-time analytics, directly on the data lake. Tools like Apache Spark, Apache Hive, and Presto are commonly used for querying and processing data stored in data lakes.
Support for Data Discovery and Self-Service Analytics: Data lakes enable data discovery and self-service analytics, empowering users to explore and analyze data without extensive dependencies on IT teams. Data scientists, analysts, and business users can access relevant data directly from the data lake, speeding up insights generation and decision-making processes.
In summary, ONE DL provides a flexible and scalable platform for storing, managing, and analyzing diverse data types at scale. By leveraging a schema-on-read approach and supporting various analytics tools, ONES DL facilitate advanced data analytics and enable organizations to derive valuable insights from their data assets. However, proper governance, security, and metadata management are crucial to ensure the usability, reliability, and integrity of data lakes.
As of now ONES support 2 different platforms where customer can get the RAW data
Splunk
Amazon S3
Users will have the option to tune the frequency of streaming the metric to the cloud platform, user will have the option to tune frequency starting from 1 minute to 60 minutes.
Users can select/unselect the Network state metrics using the above catalogue option
