Course Outline
Introduction
Understanding the Fundamentals of Apache Flume
- About Apache Flume
- Understanding How Flume Works
- Overview of the Important Components of Apache Flume
- Architecture of Apache Flume
- Data Flow Mode
- Reliability
- Recoverability
Setting Up Apache Flume
- Setting up and Configuring an Agent
- Starting an Agent
- Using Environment Variables
- Logging Raw Stream of Data
- Installing Third-Party Plugins
Ingesting Data from External Resources
- Using Avro RPC Mechanism
- Executing Commands
- Exploring Network Streams
Setting Multi-Agent Flow
Consolidating Events into a Single Channel
Defining a Flow Multiplexer
Flow Configuration
- Defining the Flow
- Setting Up Individual Components
- Adding Multiple Flows in an Agent
- Setting Up a Multi-Tier Flow
- Fanning Out the Flow from a Single Source to Multiple Channels
Implementing a Flume Source
- Using Avro Source
- Using Thrift Source
- Using Exec Source
- Using JMS Source
- Using Spooling Directory Source
- Using Taildir Source
- Using Twitter 1% firehose Source
- Using Kafka Source
- Using NetCat TCP Source
- Using NetCat UDP Source
- Using Sequence Generator Source
- Using Syslog TCP Source
- Using Multiport Syslog TCP Source
- Using Syslog UDP Source
- Using HTTP Source
- Using Stress Source
- Using Legacy Sources
- Using Custom Source
- Using Scribe Source
Implementing a Flume Sink
- Using HDFS Sink
- Using Hive Sink
- Using Logger Sink
- Using Avro Sink
- Using Thrift Sink
- Using IRC Sink
- Using File Roll Sink
- Using Null Sink
- Using HBaseSinks
- Using MorphlineSolrSink
- Using ElasticSearchSink
- Using Kite Dataset Sink
- Using Kafka Sink
- Using HTTP Sink
- Using Custom Sink
Implementing a Flume Channel Interface
- Using Memory Channel
- Using JDBC Channel
- Using Kafka Channel
- Using File Channel
- Using Spillable Memory Channel
- Using Pseudo Transaction Channel
- Using a Custom Channel
Using Flume Channel Selectors
- Using the Replicating Channel Selector
- Using the Multiplexing Channel Selector
- Using a Custom Channel Selector
Implementing Flume Sink Processors
- Using the Defauult Sink Processor
- Using the Failover Sink Processor
- Using the Load balancing Sink Processor
- Using a Custom Sink Processor
Using Event Serializers
Using Flume Interceptors
- Using the Timestamp Interceptor
- Using the Host Interceptor
- Using the Static Interceptor
- Using the Remove Header Interceptor
- Using the UUID Interceptor
- Using the Morphline Interceptor
- Using the Search and Replace Interceptor
- Using the Regex Filtering Interceptor
- Using the Regex Extractor Interceptor
Understanding Flume Properties
Security Configurations on Apache Flume
Monitoring and Reporting in Apache Flume
Using Tools in Apache Flume
- Using the File Channel Integrity Tool
- Using the Event Validator Tool
Understanding Topology Design Considerations
Handling Agent Failures
Handling Compatibility
Troubleshooting
Summary and Conclusion
Closing Remarks
Requirements
- Programming experience
Testimonials (5)
Theory and excises not separated but after every bigger Topic there were exercises to remember theory in practice
Krzysztof - Nordea
Course - XQuery
Practical examples.
Mateusz - Aplikacje Krytyczne
Course - BizTalk for Developers
interesting content
Mike Bassi - SRC Aus
Course - OpenCL
The speed and focus on the target system, I could maintain focus well
Kevin - Phoenix Contact B.V.
Course - gRPC
Conducting training so that everyone could follow. Not leaving the student with questions asked by him.