Flume source
Flume agent can have multiple sources, but it is mandatory to have at least one source for it to function. The source is managed by Source Runner which controls the threading aspect and execution models namely:
- Event-driven and
- Polling
In event-driven execution model the source listens and consumes events. In polling execution model the source keeps polling for events and then deal with it.
The event (as detailed earlier) can take a variety of content satisfying the event schema (header and payload). The source, complying with the architecture principle of extensibility, works on plugin approach. The source requires mandated name and type. According to the type, source will demand additional parameters and accordingly configurations have to set for it to work fine. The source can accept single event or a batch of event (mostly and in ideal case micro-batch as opposed to regular batch). Built-in sources in Flume can be broadly classified as:
- Asynchronous sources: Client sending the...
