Six V

• Big Data ist mit 6 englischen Begriffen definiert, die mit “V” beginnen. Schreiben alle Begriffe (auswendig lernen)
  • Volume: ****The amount of data generated and stored
    • Volume refers to the massive amounts of data generated from various sources, including business transactions, social media, sensors, and more. Big data involves handling petabytes to exabytes of data.
    • Example: A social media platform storing terabytes of user-generated content daily.
  • Velocity: The speed at which new data is generated and processed
    • Velocity denotes the rapid rate at which data flows from various sources like IoT devices, online transactions, and social media feeds, necessitating real-time or near-real-time processing and analysis.
    • Example: Real-time credit card fraud detection based on transaction data as it occurs.
  • Variety: The different types of data (structured, semi-structured, unstructured).
    • Variety refers to the diverse forms of data, including structured data (databases), semi-structured data (XML, JSON), and unstructured data (text, images, videos), all of which are part of big data.
    • Example: Combining customer data from CRM systems (structured), emails (semi-structured), and call center transcripts (unstructured) for a comprehensive view.
  • Veracity: The accuracy and reliability of data.
    • Veracity concerns the quality, credibility, and accuracy of data. In the context of big data, ensuring the data is clean, trustworthy, and not misleading is crucial for effective decision-making.
    • Example: Filtering out noise and errors from sensor data to accurately monitor environmental conditions.
  • Value: The usefulness of the data in making decisions.
    • Value represents the actionable insights and benefits that can be derived from processing and analyzing big data. It's about turning data into valuable information that can drive strategic decisions and actions.
    • Example: Analyzing consumer behavior data to personalize marketing strategies and improve sales.
  • Variability: The inconsistency of the data set (changes in data flow rates, distributions).
    • Variability refers to the fluctuation in data flow rates, formats, and structures. Big data sources often produce data that is inconsistent, making processing and analysis challenging.
    • Example: During a marketing campaign, the volume of social media posts mentioning a brand may spike suddenly, and the sentiment expressed in these posts (positive, negative, neutral) can vary widely. The analysis system must accommodate these fluctuations in volume and the changing nature of the data (from text to emojis to video mentions) to provide accurate sentiment analysis.
• Variety vs Variability:
  • Variety of formats, is about the type of data
  • Variability of data flow and change in the data, is about the inconsistency and change in data
• Law School Beispiel fuer Veracity geben
  • Law schools in US reported that thei student have high employment rate 10 months after graduation of 14% → affects ranking
  • Self-reported by school, no checks possible, nothing known about nature of employment - people can get senior positions or work part time in McDonalds
  • If a metric is a target, it can be “optimized”

Data Stewardship

• Was ist Data Stewardship?

  • Data stewardship promotes access and use of digital data today and data preservation promotes the access and use of digital data tomorrow

• What are the goals of Data Stewardship?

  • Creating and maintaining consistent metadata
  • Handling data ownership
  • Ensuring data preservation
  • Facilitating data curation

• Data Preservation und Curation erklaeren

  • Data Preservation: Make data available online long-term, accessible, provide needed sofware, VMs
    • Actions undertaken to ensure the long term viability and availability of digital assets - how to make the data usable tomorrow
    • Ensuring materials Authenticity (e.g. digital logbook), Reliability (trusted source), Usability (e.g. format, VMs, emulation), Integrity (records complete & unaltered)
  • Data Curation: Explain the data (e.g columns meaning), explain how to evaluate the data, explain how it was gathered
    • Improving and updating documentation, linking to related data, rights management
    • Maintaining and adding value to a trusted body of digital information for current and future use

• Explain Bit-stream Preservation

  • Physical preservation of data objects on the ‘bit level’

    

  https://www.youtube.com/watch?v=N2zK3sAtr-4&ab_channel=NYUHealthSciencesLibrary